Introduction
In this section of the series, we will be exploring how to send a GET
HTTP request in golang. We will be understanding how to send a basic GET request, create an HTTP request and customize the client, add headers, read the response body, etc in the following sections of this post.
What is a GET method?
A GET method in the context of an HTTP request is an action that is used to obtain data/resources. The GET
method is used in a web application to get a resource like an HTML page, image, video, media, etc.
Some common use cases of the GET
method are:
- Loading a webpage
- Getting an image, file or other resource
- API requests to retrieve data
- Search queries sending filters and parameters
Basic GET Request
To use the HTTP method GET
in golang, the net/http package has a Get method. This method simply takes in a URL as a string and returns the response or an error. Let's look at how to send a basic HTTP GET request in golang.
// web/methods/get/main.go
package main
import (
"fmt"
"net/http"
)
func main() {
reqURL := "https://www.google.com"
resp, err := http.Get(reqURL)
if err != nil {
panic(err)
}
fmt.Println(resp)
fmt.Println(resp.Status)
fmt.Println("Status Code:", resp.StatusCode)
}
$ go run main.go
&{200 OK 200 HTTP/2.0 2 0 map[Alt-Svc:[h3=":443"; ma=2592000,h3-29=":443"; ma=2592000] Cache-Control:[private, max-age=0] Content-Security-Policy-Report-Only:[object-src 'none';base-uri 'self';script-src 'nonce-pdz_s8Gr0owwMbX8I9qNEQ' 'strict-dynamic' 'report-sample' 'unsafe-eval' 'unsafe-inline' https: http:;report-uri https://csp.withgoogle.com/csp/gws/other-hp] Content-Type:[text/html; charset=ISO-8859-1] Date:[Fri, 27 Oct 2023 09:37:04 GMT] Expires:[-1] P3p:[CP="This is not a P3P policy! See g.co/p3phelp for more info."] Server:[gws] Set-Cookie:[1P_JAR=2023-10-27-09; expires=Sun, 26-Nov-2023 09:37:04 GMT; path=/; domain=.google.com; Secure AEC=Ackid1Q5FARA_9d7f7znggUdw6DoJA1DBpI17Z0SWxN519Dc64EqmYVHlFg; expires=Wed, 24-Apr-2024 09:37:04 GMT; path=/; domain=.google.com; Secure; HttpOnly; SameSite=lax NID=511=EToBPqckCVRE7Paehug1PgNBKqe7lFLI9d12xJrGbvP9r8tkFIRWciry3gsy8FZ8OUIK4gE4PD-irgNzg4Y1fVePLdyu0AJdY_HcqL6zQYok-Adn-y5TDPmMCNuDnrouBfoxtqVjYY_4RFOe8jalkYto5fQAwzWnNJyw8K0avsw; expires=Sat, 27-Apr-2024 09:37:04 GMT; path=/; domain=.google.com; HttpOnly] X-Frame-Options:[SAMEORIGIN] X-Xss-Protection:[0]] 0xc000197920 -1 [] false true map[] 0xc0000ee000 0xc0000d8420}
200 OK
Status Code: 200
In the above code, we have defined a URL string as reqURL
and used the Get method to send a GET request. The Get
is parsed with the reqURL
string and the return values are stored as resp
and err
. We have added an error check after calling the Get
method in order to avoid errors later in the code.
The Get
method as seen from the output has returned a *http.Response
object, we can use the Status
and StatusCode
properties to get the status code of the response. In this case, the status code of the response was 200
. The response resp
is an object of type http.Response
i.e. it has fields like Body
, StatusCode
, Headers
, Proto
, etc. We can get each individual field from the resp
object. We will later look into how to read the Body
field from the response, it is not directly read as a string nor it is stored in other forms, rather it is streamed from the requested URL.
Creating a GET request
We can even construct a GET request using the NewRequest method. This is a low-level way of creating a GET
request. We mention the method
, URL
, and the body
, in the case of GET
request, there is nobody. So, the NewRequest
is a general way of creating a http
request.
// web/methods/get/newreq.go
package main
import (
"fmt"
"net/http"
)
func main() {
reqURL := "https://www.google.com"
req, err := http.NewRequest(http.MethodGet, reqURL, nil)
if err != nil {
panic(err)
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
panic(err)
}
fmt.Println(resp)
}
As we can see, we construct a GET
request using the NewRequest
method and then use the Do method to send the request to the server. The http.DefaultClient is used as a client to send the request, if we want to customize this we can create a new instance object of http.Client and use it to send requests. We will be taking a look at clients in another part of this series when we want to persist a connection or avoid connecting multiple times to the same application/URL.
For now, we will go ahead with the DefaultClient. This will trigger the request, in this case, a GET
request to the specified URL in the reqURL
string. The Do
method returns either a http.Response
or an error
just like the Get
method did.
Reading the Response Body
We saw some different ways to send a GET
request, now the below example will demonstrate how to read the body of the response. The response body is read from a buffer rather than loading the entire response into memory. It makes it flexible to parse the data efficiently and as per the needs. We will see how we use the io package's ReadAll method can be used to read from the response buffer.
// web/methods/get/body.go
package main
import (
"fmt"
"io"
"net/http"
)
func main() {
reqURL := "https://httpbin.org/html"
resp, err := http.Get(reqURL)
if err != nil {
panic(err)
}
// close the body object before returning the function
// this is to avoid the memory leak
defer resp.Body.Close()
// stream the data from the response body only once
// it is not buffered in the memory
body, err := io.ReadAll(resp.Body)
if err != nil {
panic(err)
}
fmt.Println(string(body))
}
In the above example, we are trying to get the body from the response to the request sent at https://httpbin.org/html
. We have used the simple Get
method instead of NewRequest
and Do
for simplicity. The response is stored in resp
, we also have added defer resp.Body.Close()
which is to say that we will close the body reader object when the function is returned/closed. So, this means that the Body
is not readily available data, we need to obtain/stream the data from the server. We have to receive the body in bytes as a TCP request, the body is streamed in a buffer.
The response body is streamed from the server, which means that it's not immediately available as a complete data set. We read the body in bytes as it arrives over the network, and it's stored in a buffer, which allows us to process the data efficiently.
Reading Body in bytes
We can even read the body in bytes i.e. by reading a chunk of the buffer at a time. We can use the bytes.Buffer container object to store the body. Then we can create a slice of bytes as []bytes
of a certain size and read the body into the chunk. By writing the chunk into the buffer, we get the entire body from the response.
// web/methods/get/body.go
package main
import (
"bytes"
"fmt"
"net/http"
)
func main() {
reqURL := "https://httpbin.org/html"
resp, err := http.Get(reqURL)
if err != nil {
panic(err)
}
defer resp.Body.Close()
// create a empty buffer
buf := new(bytes.Buffer)
// create a chunk buffer of a fixed size
chunk := make([]byte, 1024)
for {
// Read into buffer
n, err := resp.Body.Read(chunk)
if err != nil {
break
}
// append the chunk to the buffer
buf.Write(chunk[:n])
fmt.Printf("%s\n", chunk[:n])
}
// entire body stored in bytes
fmt.Println(buf.String())
}
In the above example, the body is read chunkwise buffers and obtained as a slice of bytes. We define the request as a GET
request to the httpbin.org/html
. We create a new Buffer as a slice of bytes with bytes.Buffer, then we define chunk as a container to stream the response body with a particular size. We have taken 1024
bytes as the size of the chunk. Then inside an infinite for loop, we read the body as n, err :=
resp.Body.Read
(chunk)
. The code will read the body into the chunk(slice of byte) and the return value will be the size of the bytes read or an error. Then we check if there is no error, and if there is an error, we break the loop indicating we have completed reading the entire body or something went wrong. Then we write the chunk into the buffer that we allocated earlier as buf
. This is a slice of bytes, we are basically populating the buffer with more slices of bytes.
The entire body is then stored in the buffer as a slice of bytes. So, we have to cast it into a string to see the contents. So, this is how we can read the contents of a body in a response in chunks.
Parsing the JSON body with structs
If we have the structure of the response body already decided, then we can define a struct for the response body and then we can Unmarshal / deserialize/unpickle. This means we can convert the bytes representation of the data into a Golang-specific structure which is called a high-level representation of the data. We can parse the JSON body into a defined struct using Unmarshal or Decode methods in the json package.
Let's look at both the methods.
Using Unmarshal
The Unmarshal
method takes in two parameters i.e. the body in bytes and the reference of the object that we want to unmarshal into. The method returns an error if there is a discrepancy in the returned JSON or the structure defined it is unable to deserialize the JSON object into the defined structure.
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
)
type Product struct {
ID int `json:"id"`
Title string `json:"title"`
Description string `json:"description"`
Price float64 `json:"price"`
DiscountPercentage float64 `json:"discountPercentage"`
Rating float64 `json:"rating"`
Stock int `json:"stock"`
Brand string `json:"brand"`
Category string `json:"category"`
Thumbnail string `json:"thumbnail,omitempty"`
Images []string `json:"-"`
}
func main() {
reqURL := "https://dummyjson.com/products/1"
resp, err := http.Get(reqURL)
if err != nil {
panic(err)
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
if err != nil {
panic(err)
}
var data Product
if err := json.Unmarshal(body, &data); err != nil {
panic(err)
}
fmt.Println(data)
fmt.Println(data.Title)
}
$ go run main.go
{1 iPhone 9 An apple mobile which is nothing like apple 549 12.96 4.69 94 Apple smartphones https://cdn.dummyjson.com/product-images/1/thumbnail.jpg []}
iPhone 9
In the above example, we have defined a structure called Product with fields such as ID
, Title
, Description
, etc. We use the JSON tag to specify how each field should be encoded to or decoded from JSON. These tags guide the Golang JSON encoders and decoders to correctly map JSON data to struct fields and vice versa. The omitempty
option in a struct tag instructs the encoder to omit the field from the JSON output if the field's value is the zero value for its type (e.g., 0 for integers, "" for strings, nil for pointers, slices, and maps). This is useful for reducing the size of the JSON output by excluding empty or default-valued fields.
Conversely, the -
option in a struct tag tells the encoder and decoder to completely ignore the field. It will not be included in encoded JSON, nor will it be populated when decoding JSON into a struct. This is particularly useful for excluding fields that are meant for internal use only and should not be exposed through JSON.
Therefore, omitempty
is used to control the inclusion of fields in the JSON output based on their values, while -
is used to exclude fields from both encoding and decoding from JSON.
We send the GET
request to the api https://dummyjson.com/products/1
. The response from the request is read into a slice of bytes with io.ReadAll that takes in a io.Reader object in this case it is the resp.Body
and it returns a slice of byte and error if any issue arises while reading in the body. Further, we can use the Unmarshal method to parse the slice of body body
into the struct Product
with the variable data
, the reference to &data
indicates that the method will directly mutate/change this variable and populate the object with the fields from the body.
So in a gist, to convert the JSON body into a golang native structure with Unmarshal
with the following steps:
- Read the body into a slice of bytes using
io.ReadAll
- Create an object of the struct
- Pass the body as a slice of bytes and the reference of that object (struct instance) into the Unmarshal method
- Access the object with the fields in the struct
In the output response, we can see the object is populated with the fields from the body. The Title
field is accessed using the data.Title
as we do with a normal golang struct. The Images
field is not populated because we have always ignored/omitted from the json tag with -
.
Using Decoder
Similar to the Unmarshal
we can use the Decoder to parse the body into a struct. However, the parameters it takes are a bit different and it is a two-step process. We first create a Decoder object using the NewDecoder method, which takes in a io.Reader
object, luckily the body from the response is already in that structure, so we can directly pass that resp.Body
into the NewDecoder
method. The second step is to decode the data into the object, here as well, we need to create the object of the struct and parse the reference to the object to the Decode method. The Decode
method converts the bytes parsed in the resp.Body
from the Decoder
object and populates the fields of the object provided in the reference struct.
So the steps for deserializing the json object into the struct with the decode method are:
- Create a decoder with
NewDecoder
method and pass theresp.Body
as the parameter which is anio.Reader
object - Create an object of the struct
- Decode the body into that object using the
decoder.Decode
method - Access the object with the fields in the struct
package main
import (
"encoding/json"
"fmt"
"net/http"
)
type Product struct {
ID int `json:"id"`
Title string `json:"title"`
Description string `json:"description"`
Price float64 `json:"price"`
DiscountPercentage float64 `json:"discountPercentage"`
Rating float64 `json:"rating"`
Stock int `json:"stock"`
Brand string `json:"brand"`
Category string `json:"category"`
Thumbnail string `json:"thumbnail,omitempty"`
Images []string `json:"-"`
}
func main() {
reqURL := "https://dummyjson.com/products/1"
resp, err := http.Get(reqURL)
if err != nil {
panic(err)
}
defer resp.Body.Close()
var data Product
decoder := json.NewDecoder(resp.Body)
err = decoder.Decode(&data)
if err != nil {
panic(err)
}
fmt.Println(data)
fmt.Println(data.Title)
}
$ go run main.go
{1 iPhone 9 An apple mobile which is nothing like apple 549 12.96 4.69 94 Apple smartphones https://cdn.dummyjson.com/product-images/1/thumbnail.jpg []}
iPhone 9
We have first defined the struct Product
with the json:"id"
tag. As explained earlier, we have used the json tags to identify the fields from the json data to the structures while encoding and decoding.
In the above example, we have sent a GET
request to the api endpoint https://dummyjson.com/products/1
, and we have created a new decoder with the NewDecoder
method with the resp.Body
as the parameter. The data
is created as a Product
instance. The reference to data
is parsed to the Decode
method from the decoder
instance as &data
. This method will either return nil
or an error
. Thereafter, we can check for errors and then only access the data object with its populated fields from the response body.
There is a certain difference between the Unmarshal
and Decode
methods. The difference is just a slight performance improvement in the NewDecoder
and Decode
methods. Though it is not significant, having a little info about it might be handy in your use case. Read here for more info : To Unmarshal or Decode
Adding Headers to a GET Request
We can even add headers before sending a GET
request to a URL. By creating a Request
object with the NewRequest
method and adding a Header with the Add method. The Add
method will take in two parameters i.e. the key of the header, and the value of the header both as strings.
// web/methods/get/header.go
package main
import (
"fmt"
"io"
"net/http"
)
func main() {
req, err := http.NewRequest(http.MethodGet, "https://api.github.com/users/mr-destructive", nil)
if err != nil {
panic(err)
}
req.Header.Add("Authorization", "token YOUR_TOKEN")
resp, err := http.DefaultClient.Do(req)
if err != nil {
panic(err)
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
if err != nil {
panic(err)
}
fmt.Println(string(body))
}
$ go run web/methods/get/header.go
{"message":"Bad credentials","documentation_url":"https://docs.github.com/rest"}
In the above example, we have created a GET
request to the https://api.github.com/users/mr-destructive
the last portion is the username, it could be any valid username. The request is to the GitHub API, so it might require API Key/Tokens in the headers, however, if there are certain endpoints that do not require Authorization headers might work just fine.
So, the above code will give 401
error indicating the request has wrong or invalid credentials, if we remove the header, the request will work fine. This is just an example, but headers are useful in working with APIs.
Without adding the header:
$ go run web/methods/get/header.go
{"login":"Mr-Destructive","id":40317114,"node_id":"MDQ6VXNlcjQwMzE3MTE0","avatar_url":"https://avatars.githubusercontent.com/u/40317114?v=4","gravatar_id":"","url":"https://api.github.com/users/Mr-Destructive",
...
"updated_at":"2023-10-10T17:57:22Z"}
That's it from the 33rd part of the series, all the source code for the examples are linked in the GitHub on the 100 days of Golang repository.
Mr-Destructive / 100-days-of-golang
Scripts and Resources for the 100 days of golang series (In progress)
100 Days of Go lang
Go lang is a programming language which is easier to write and even provides type and memory safety, garbage collection and structural typing. It is developed by the Google Team in 2009 and open sourced in 2012. It is a programming language made for Cloud native technologies and web applications. It is a general purpose programming lanugage and hence there are no restrictions on the type of programs you wish to make in it.
Resources to learn Golang
- Exercism - Go track
- Go by example
- Go Time - Podcast
- Boot.dev Golang Course
- Zetcode Blog
- daily.dev Golang articles
Some famous applications made with GO!
Web apps | DevOps tools | CLI tools |
---|---|---|
SoundCloud - Music System |
Prometheus - Monitoring system and time series database |
gh-cli - Official Github CLI |
Uber - Ride Sharing/Cab booking Webapp |
References
Conclusion
From this article, we explored the GET
HTTP method in golang. By using a few examples for creating a get request, adding headers, and reading the response body, the basic use cases were demonstrated.
Hopefully, you found this section helpful, if you have any comments or feedback, please let me know in the comments section or on my social handles. Thank you for reading. Happy Coding :)
Top comments (0)