How to use io.Reader and io.Writer

This was confusing for me when I started using Go after coming from other languages.

Most of the technical articles and tutorials I came across made numerous references to io.Reader and io.Writer, interfaces provided by the io package, describing them as excellent and easy-to-use but seemed to assume that the person reading already knew how to use them.

Those abstractions are indeed powerful and convenient so here's how to actually use them in some common scenarios.

io.Reader

Say you have some reader which implements the io.Reader interface and you want to get the data out of it.

You could make a slice of bytes with a capacity, then pass the slice to Read().

b := make([]byte, 512)
reader.Read(b)

Go slices are reference types which means when Read() is called, b will be modified to contain the data from the reader.

However, Read() will only copy as many bytes as the passed slice has capacity for.

Most of the time, gophers will reach for ioutil.ReadAll when they simply want all of the data at once.

You could also use a byte buffer and io.Copy.

For example, here's a function I made and use all the time to get the contents of text files as a string (inspired or perhaps spoiled by php's file_get_contents())

func fileGetContents(filename string) string {
    contents := new(bytes.Buffer)
    f, err := os.Open(filename)
    checkErr(err)
    _, err = io.Copy(contents, f)
    if err != io.EOF {
        checkErr(err)
    }
    checkErr(f.Close())
    return contents.String()
}

checkErr() above is just if err != nil { panic(err) }

But what if reader is a stream of data with indeterminate (or neverending) length?

In that case you want to reach for a bufio.Scanner:

scanner := bufio.NewScanner(os.Stdin)
for scanner.Scan() {
    fmt.Println(scanner.Text())
}

io.Writer

To write data is very straightforward:

someBytes := []byte("Hello world")
f, err := os.Open("someFile.txt")
checkErr(err)
f.Write(someBytes)
f.Close()

Of course io.WriteString is less typing:

f, err := os.Open("someFile.txt")
checkErr(err)
io.WriteString(f, "Hello world")
f.Close()

A simple byte buffer

There's nothing stopping you from implementing io.Reader and io.Writer in your own user-defined types.

For example, here's a very useful, simple byte buffer:

type buf []byte

func (b *buf) Read(p []byte) (n int, err error) {
    n = copy(p, *b)
    *b = (*b)[n:]
    if n == 0 {
        return 0, io.EOF
    }
    return n, nil
}

func (b *buf) Write(p []byte) (n int, err error) {
    *b = append(*b, p...)
    return len(p), nil
}

func (b *buf) String() string { return string(*b) }

I made it at first just to capture the text output of shell commands in go.

ʕ◔ϖ◔ʔ

io.Reader and io.Writer are used all over the standard library from shell commands to networking even the http package!

Hopefully now you know the basics of how to use readers and writers and maybe you can even try to implement io.Reader and io.Writer for your own user-defined types.