DEV Community

Cover image for Memory efficient parsing in GO
MoniqueLive
MoniqueLive

Posted on

Memory efficient parsing in GO

#go #json #xml #stream

Instead of greedily reading the whole stream in memory and passing it to json.Unmarshal:

json.Unmarshal(bigBlobOfData, &myStruct)
Enter fullscreen mode Exit fullscreen mode

We can delegate the stream reading to our dear masters of golang:

json.NewDecoder(myIOReader).Decode(&myStruct)
Enter fullscreen mode Exit fullscreen mode

And read multi giga(tera?)bytes files with no sweat. Also works with xml.NewDecoder.

Top comments (0)

Read next

basskibo profile image

Update JSON file using Terminal or bash script

Bojan Jagetic -

daviducolo profile image

Building an Efficient Text Compression Algorithm Inspired by Silicon Valley’s Pied Piper

Davide Santangelo -

douglasmakey profile image

Playing with Rust: Building a Safer rm and Having Fun Along the Way

Douglas Makey Mendez Molero -

neelp03 profile image

Handling Large File Uploads in Go with AWS S3: Stream Like a Pro

Neel Patel -