DEV Community 👩‍💻👨‍💻

Arsham Arya
Arsham Arya

Posted on

How to Unmarshal JSON in a custom way in Golang

#go

When I was working on gocoinex library for the first time, I needed some custom configuration on JSON unmarshalling process, I want to share my learnings with you here, but before that, we're going to have a recap on how we can unmarshall JSON in golang.

These are real world examples. (coinex exchange API)

First, we make our request like this:

raw_response, _ := http.Get("https://api.coinex.com/v1/market/list")
Enter fullscreen mode Exit fullscreen mode

And we'll get a json object like this:

{
    "code": 0,
    "data": [
        "LTCBCH",
        "ETHBCH",
        "ZECBCH",
        "DASHBCH"
    ],
    "message": "Ok"
}
Enter fullscreen mode Exit fullscreen mode

But we should parse it,
There are different ways to parse it,
You can use NewDecoder or Unmarshal functions of json package,
and you can decode it to a struct or to a map[string]interface{}

It depends on your preferences, but in this case, I prefer NewDecoder and struct combination.

So we should make a struct like this:

type AllMarketList struct {
    Code    int    `json:"code"`
    Message string `json:"message"`
    Data    []string
}
Enter fullscreen mode Exit fullscreen mode

Also we can have embedded structs, for example, we break last struct to two:

type GeneralResponse struct {
    Code    int    `json:"code"`
    Message string `json:"message"`
}

type AllMarketList struct {
    GeneralResponse
    Data    []string
}
Enter fullscreen mode Exit fullscreen mode

And there is no difference.

Finally, use NewDecoder to decode the raw_response to AllMarketList struct:

var allMarketList AllMarketList
json.NewDecoder(raw_response.Body).Decode(&allMarketList)
Enter fullscreen mode Exit fullscreen mode

Completed code

package main

import (
    "encoding/json"
    "fmt"
    "net/http"
)

type AllMarketList struct {
    Code    int    `json:"code"`
    Message string `json:"message"`
    Data    []string
}

func main() {
    raw_response, _ := http.Get("https://api.coinex.com/v1/market/list")

    var allMarketList AllMarketList
    if err := json.NewDecoder(raw_response.Body).Decode(&allMarketList); err != nil {
        fmt.Println(err)
    }
    defer raw_response.Body.Close()
    fmt.Printf("%+v\n", allMarketList)
}
Enter fullscreen mode Exit fullscreen mode

Example Two

Think we have a json like this:

{
  "code": 0,
  "data": {
    "date": 1513865441609, # server time when returning
    "ticker": {
        "open": "10", # highest price
        "last": "10.00", # latest price 
        "vol": "110" # 24H volume
    }
  },
  "message" : "Ok"
}
Enter fullscreen mode Exit fullscreen mode

We're going to improve some things in decoding process

  1. In this case, Unix timestamp can't be parsed, we should provide it a way.
  2. We want to remove "ticker" key and access to "open", "last", "vol" directly from "data".
  3. "last" should be exported to field named "Close"! (same happens with "vol" and "Volume".
  4. "open", "last", "vol" should be float, not string, but we'll leave them for next example.

Problems 1 and 2, can be solved by implementing UnmarshalJSON method on any struct we want to decode.
problem 3 will be easily solved with json tags. (I've mentioned it in the code below)

Our final struct should be like this:

// Final struct
type SingleMarketStatistics struct {
    Code    int    `json:"code"`
    Message string `json:"message"`
    Data    TickerData
}

// Inner struct that we should implement to solve problem 2
type TickerData struct {
    ServerTime CTime   `json:"date"` // CTime is short for CustomTime
    Open       float64 `json:"open"`
    Close      float64 `json:"last"` // Different Attribute name and tag name
    Volume     float64 `json:"vol"` // Different Attribute name and tag name
}

// Custome time
// Inner struct that we should implement to solve problem 1
type CTime struct {
    time.Time
}
Enter fullscreen mode Exit fullscreen mode

Custom time implementation

func (t *CTime) UnmarshalJSON(data []byte) error {
    // Ignore null, like in the main JSON package.
    if string(data) == "null" || string(data) == `""` {
        return nil
    }
    // Fractional seconds are handled implicitly by Parse.
    i, err := strconv.ParseInt(string(data), 10, 64)
    update := time.UnixMilli(i)
    *t = CTime{update}
    return err
}
Enter fullscreen mode Exit fullscreen mode

And we don't get an error anymore!
This method will be automatically used (thanks to interfaces!) whenever we want to decode a time to CTime!

Custom Data implementation

func (t *TickerData) UnmarshalJSON(data []byte) error {
    if string(data) == "null" || string(data) == `""` {
        return nil
    }

    // This is how this json really looks like.
    var realTicker struct {
        ServerTime CTime `json:"date"`
        Ticker     struct {
            // tags also can be omitted when we're using UnmarshalJSON.
            Open   string `json:"open"`
            Close  string `json:"last"`
            Volume string `json:"vol"`
        } `json:"ticker"`
    }

    // Unmarshal the json into the realTicker struct.
    if err := json.Unmarshal(data, &realTicker); err != nil {
        return err
    }

    // Set the fields to the new struct,
    // with any shape it has,
    // or anyhow you want.
    *t = TickerData{
        ServerTime: realTicker.ServerTime,
        Open:       realTicker.Ticker.Open,
        Close:      realTicker.Ticker.Close,
        Volume:     realTicker.Ticker.Volume,
    }

    return nil
}
Enter fullscreen mode Exit fullscreen mode

Now, just use NewDecoder as before, no change is needed.

var singleMarketStatistics SingleMarketStatistics 
json.NewDecoder(raw_response.Body).Decode(&allMarketList)
Enter fullscreen mode Exit fullscreen mode

Example Three

Imagine a JSON like this:

{
  "asks": [ // This is a array of asks
    [ // This is a array of ONE ask
      "10.00", // Price of ONE ask
      "0.999", // Amount of ONE ask
    ]
  ],
  "bids": [ // Same structure as asks
    [
      "10.00",
      "1.000",
    ]
  ]
}
Enter fullscreen mode Exit fullscreen mode

As it's totally clear, in the non professional way, we should decode "asks" to [][]string and access first ask price like this asks[0][0] and amount asks[0][1]
Who remembers 0 is price and 1 is amount? What was which? 😄
So we'll manage them on the UnmarshalJSON method.
Also, we'll solve problem 4 of previous example that also exists here.

type BidAsk struct {
    // Tags are not needed.
    Price  float64 `json:"price"`  // Bid or Ask price
    Amount float64 `json:"amount"` // Bid or Ask amount
}

func (t *BidAsk) UnmarshalJSON(data []byte) error {
    // Ignore null, like in the main JSON package.
    if string(data) == "null" || string(data) == `""` {
        return nil
    }

    // Unmarshal to real type.
    var bisask []string
    if err := json.Unmarshal(data, &bisask); err != nil {
        return err
    }

    // Change value type from string to float64.
    price, err := strconv.ParseFloat(bisask[0], 64)
    if err != nil {
        return err
    }
    amount, err := strconv.ParseFloat(bisask[1], 64)
    if err != nil {
        return err
    }

    // Map old structure to new structure.
    *t = BidAsk{
        Price:  price,
        Amount: amount,
    }
    return err
}

type MarketDepth struct {
    Asks   []BidAsk `json:"asks"` // Ask depth
    Bids   []BidAsk `json:"bids"` // Bid depth
}
Enter fullscreen mode Exit fullscreen mode

Again, we simply use:

var marketDepth MarketDepth 
json.NewDecoder(raw_response.Body).Decode(&marketDepth)
Enter fullscreen mode Exit fullscreen mode

And enjoy the beauty of your result:

for i, ask := range data.Data.Asks {
    fmt.Printf("Ask %v\n", i)
    fmt.Printf("  Price: %v\n", ask.Price) // here is the beauty
    fmt.Printf("  Amount: %v\n", ask.Amount) // here is the beauty
    fmt.Println()
}
for i, bid := range data.Data.Bids {
    fmt.Printf("Bid %v\n", i)
    fmt.Printf("  Price: %v\n", bid.Price) // here is the beauty
    fmt.Printf("  Amount: %v\n", bid.Amount) // here is the beauty
    fmt.Println()
}
Enter fullscreen mode Exit fullscreen mode

Top comments (0)

Let's Get Wacky


Use any Linode offering to create something unique or silly in the DEV x Linode Hackathon 2022 and win the Wacky Wildcard category

Join the Hackathon <-