Understanding Slices And The Internals In Go

For many languages designed over the years, we have seen alot of them take lessons from the existing to improve the experience and generally make it better. The Go design team, has also done alot of research in this area.

One area that is not appreciated as much are slices in Go. Slices provide a very simple abstraction over Arrays in Go. Slices allow you to have a dynamic window overview on an array.

So, we know an array is basically "contiguous" blocks of memory. For many languages when you allocate an array you have to define the size, this is to allow a memory safe reserved space for that array.

In Go, an array can be defined like this

var array [5]int

This would allocate 5 blocks of either 64bits or 32bits in memory (depending on the cpu architecture). so for example

64*5 = 320bits = 40bytes

This is nice, assuming i know ahead of time the size of what i want to store. What if in the cause of my program i needed to add more int values ?

Well since, we already allocated fixed size of blocks, then we might need to allocate another with sufficient size and copy the old.

Well, this is where Go shines 😁. Slices provide a "dynamic window" over an array, internally there is an array, but all the magic of copying, sizing etc is abstracted away by slices.
So in that example, we could write like this with slices

slice := make([]int,5)
slice[4] = 23
slice = append(slice,5)

So yea, in this example, make internally allocates an array with a size of 5, and then when we needed to add another item to make it 6, append allocated a larger array and simply copied over the items in the first array, and we didnt have to do anything.

But, from the code above, when we added a new element that was clearly out of bounds, and append did all that magic what is the new length ?

if we print out the length of that slice it would simply be 6, does that mean append only created a 6 blocks array ?

So one thing to note here, is to dig into the internals or the components that make up a slice, A slice is a dynamic view over an array and internally it can be represented as this

type Slice struct {
    Array unsafe.Pointer
    cap   int
    len   int
}

Array is a pointer to the internal array, cap keeps the real length of the array, len is the size of the view which is the length we get after append. When we called append the cap size changed obviously because it allocated a larger array.

Going a little more low level, can we prove these claims ? yes we can get access back to the internal array.

  slice := []int{1,2,3,4}
  sliceStruct := (*Slice)(unsafe.Pointer(&slice))
  arrayPtr := (*[4]int)(sliceStruct.Array)
  array := *arrayPtr

If we run the code above, we see that we can see the internals of the slice when casting the pointer to a pointer of our Slice struct.

Im sure people with maybe java experience, would think, well isnt this just ArrayList or .Net engineers List. Yes infact it is, but done correctly with native language syntax support. For example, you can slice a slice to get a different view

 slice := []int{1,2,3,4,5}
 sliceB := slice[1:2]

In the example above, the sliceB slices the slice from the second index, with a length of 2. You would think, ok this might allocate another array ? since we are getting a slice from the second position of another.

Well no, the Array pointer in Slice, points to the memory block of the first element in the array, so when we replaced the slice, sliceB just pointed to the second block, and len and cap followed suite no expensive memory copying anywhere, so slices are very very efficient.

if we do

sliceB[0] = 7
fmt.Println(slice[1])

We notice, that the value is changed across, because like we said, slices is just a view over the memory nothing more. C# engineers might see something similar with Span<T>, same thing.

In summary, a slice is a simple efficient solution to a general problem in many other languages, which provides an efficient dynamic view over memory.