DEV Community

loading...
Cover image for Memory efficient parsing in GO

Memory efficient parsing in GO

moniquelive profile image Cyber ・1 min read

Instead of greedily reading the whole stream in memory and passing it to json.Unmarshal:

json.Unmarshal(bigBlobOfData, &myStruct)
Enter fullscreen mode Exit fullscreen mode

We can delegate the stream reading to our dear masters of golang:

json.NewDecoder(myIOReader).Decode(&myStruct)
Enter fullscreen mode Exit fullscreen mode

And read multi giga(tera?)bytes files with no sweat. Also works with xml.NewDecoder.

Discussion (0)

Forem Open with the Forem app