The article is originally published on HackerNoon
Golang recently turned 10 and the success this language has created over the last 10 years is overwhelming. Today, many of the most popular DevOps tools are now written in Go. This proves that Go is a language that has a great future in the DevOps industry. It is meant for cloud network infrastructure practitioners - this was one of the primary goals of the Go creators. As a result, Every major cloud provider today has turned to Go for their core cloud infrastructure, to name some - Docker, Kubernetes, Istio, Etcd, Prometheus, and Terraform.
Go Modules
Even though Go has achieved a lot of attention recently, it is still considered a fairly new language. Thus, dependency management for Go has gone through a few stages. At Golang’s inception, there was no dependency management system. go get was the only way to download dependencies, but this posed a serious issue because this pulled the code from the master branch of a repository and put the files into your GOPATH.
In case there were any significant changes to any of your dependencies, there was a high probability the code would break. After some time, tools like dep, godep, govendor came into the scene to try and solve this problem with varying levels of success.
Go has had many different dependency management considerations, but all of them were lacking, and the biggest deficiency they had was this concept of vendoring. What made it tough is that you had a source for your dependencies and that source would be in GitHub, Bitbucket, GitLab, etc. and if you needed one of those dependencies, the vendoring tool would just pull it down - you as a developer had no idea where it came from.
This blob of code just appeared and this posed a severe security concern. This is a major reason Go Modules were introduced. Now you have official dependency management within the language that takes care of your security concerns in your dependencies right from the beginning.
In this context, providing better security options in Go became a major point of conversation. Creating tools to actually help developers know which modules and which versions of those modules are up-to-date and safe to use is now a big part of the Golang roadmap. This all started with Go 1.11.
Starting with Go 1.11, Go Modules became an official effort (but only with 1.13 is when they became the standard) to standardize dependency behavior within Go applications. A major effort was made to support and encourage Go developers to use semantic versioning for their Go packages to make the evolution of Go modules easy to follow. This also addressed a security concern allowing developers to find fixed version of the same module faster since there was a standard way for authors to tag new versions.
More Features and Security aspects in Go
Golang usage is increasing fast and the Go community is more concerned about security than ever before. Today, there is an ongoing effort to make sure Go modules are even more secure.
One major security feature introduced in Golang version 1.13 is a checksum database. This database stores checksums that are used to verify that a committed Go Module collection of files haven’t changed. This feature creates trust and is a major reason why Go Modules and versioning within Go has become a hot topic.
Go Proxy
In Go modules, if you add a new dependency, it’ll download all the dependencies based on the instructions in your go.mod file and will cache it for additional operations. You can bypass the cache by using a vendor/ folder, but that is not the proper approach.
Go proxy allows by default caching and storing all the dependencies forever (in immutable storage). This means you don’t have to use any vendor/ folder anymore. You don't require VCS tools to download the dependencies when you have Go proxy as a default, and it’s significantly faster to download and build your Go module this way because Go proxy serves the source code and go.mod independently over HTTP.
Using GOPROXY has a number of benefits worth taking note of: it's fabulous and works seamlessly with the go command and is secure, fast, storage efficient, etc.
Module Mirror
A module mirror is a distinct module proxy that is introduced in Golang v.1.13 that caches metadata and source code in its own storage system, allowing the mirror to continue to serve source code that is no longer accessible from the original locations and hence speeding up downloads and protecting users from the disappearing dependencies by posing a strong security factor while working with Go projects.
Go.mod file
A module is a set of numerous Go packages that are stored in a file tree with a go.mod file at its root level. The go.mod file represents the module's module path, and it is also the import path applied for the root directory, and its dependency requirements, which are the other modules needed for a successful build while working in a Go project.
The problem of dependency management has been much improved. As a result, most developers are embracing the usage of go.mod. This helps Go developers track dependencies and versioning within a project through the go.mod file unlocking the door for future dependency scanning efforts.
Again, the go.mod file and the go command generally use semantic import versioning as the standard form for specifying module versions; hence versions can be compared and analyzed to determine which should be considered earlier or later.
Go Center
GoCenter is a free, community-centric GOPROXY. It has over 80,000 modules with over 500,000 versions and is a world standard platform to discover Go modules. It is supported by JFrog Artifactory and was created in January 2019 to meet the needs of Golang developers. GoCenter will proxy any publicly accessible module - it searches for and adds public modules regularly from the web - or when users request a specific one from the Go client as long as the proxy is set to GoCenter.
In GoCenter, with the "Add Module" button, Go Module authors can add their own modules anytime and don't need to wait for the GoCenter to find it on the web.
Go Center Includes JFrog Xray Vulnerability Scanning
A really important new security consideration has been added to GoCenter.io. It is a free vulnerability scanning feature for Go modules. By adding this security feature, GoCenter now has the power to automatically scan for known vulnerabilities recognized in the public vulnerability database. The results are stored in GoCenter and exposed on the Security page of the UI which lists all vulnerabilities found in that module version.
You can see an example here: https://search.gocenter.io/github.com/moby/moby?version=v1.13.2-0.20170601211448-f5ec1e2936dc&tab=security
In the recent official Go blog, the announcement about 'Module Mirror and Checksum Database Launched' has introduced a great improvement, and the community is excited about these features. As described in the official blog, A module mirror is a special kind of module proxy that caches metadata and source code in its own storage system, enabling the mirror to continue to serve source code that is no longer available from the original locations. This helps in speeding up downloads and protects from disappearing dependencies, which was the case previously.
Checksum Database
With 1.13, a checksum database was introduced keeping reliability and trust in mind. This ensures that the go command always adds the same lines to everyone's go.sum file in the project. Whenever the go command receives any new source code, it will verify the hash of that code against this checksum database and makes sure the hashes match, and the trust is authenticated accordingly. Again, this ensures everyone is using the same code for a given version.
The checksum database enables the go command to use an otherwise untrusted proxy safely. There is an auditable security layer on top of it, a proxy or origin server can’t intentionally start giving you the wrong code without getting caught.
The issue with the checksum db is that it is not a complete solution because all it does is verify that files haven’t been changed - if the files were created with harmful intentions from the start and committed to the checksum db - they will remain that way when accessed later. In this context, the vulnerability scanning capabilities in GoCenter actually helps developers know which modules are secure and safe to use by providing vulnerability information. You can even drill down on the dependency tab on the GoCenter page for each module and see if there are vulnerabilities in the dependency tree and which versions of modules have and don’t have vulnerabilities.
JFrog Xray in GoCenter
JFrog’s Xray is an excellent security tool from JFrog; it is a universal binary analysis product that works with Artifactory to analyze software components, giving you complete and transparent visibility about your metadata, issues with the software, and unveils a variety of vulnerabilities at any stage of the software application lifecycle. Xray's impact analysis automatically carries a security check and enhances the confidence of developers to produce software with confidence. You can learn more by reading 'GoCenter Reveals Go Module Vulnerabilities With Xray'
The Go community is now more in the DevSecOps aspects of Go projects than ever before. Recent security features in GoCenter have aimed to enrich the overall working experience of the Golang community by providing a dedicated standard and secure platform for Go module information. Using GoCenter as a private proxy benefits organizations with more security and control over their builds. Checkout GoCenter at Gocenter.io and make sure to keep your modules secure!
Top comments (0)