Overview
ginvalidator is a set of Gin middlewares that wraps the extensive collection of validators and sanitizers offered by my other open source package validatorgo. It also uses the popular open-source package gjson for JSON field syntax, providing efficient querying and extraction of data from JSON objects.
It allows you to combine them in many ways so that you can validate and sanitize your Gin requests, and offers tools to determine if the request is valid or not, which data was matched according to your validators.
It is based on the popular js/express library express-validator
Support
This version of ginvalidator requires that your application is running on Go 1.16+.
It's also verified to work with Gin 1.x.x.
Rationale
Why not use?
- Handwritten Validators: You could write your own validation logic manually, but that gets repetitive and messy fast. Every time you need a new validation, you’re writing the same kind of code over and over. It’s easy to make mistakes, and it’s a pain to maintain.
- Gin's Built-in Model Binding and Validation: Gin has validation built in, but it’s not ideal for everyone. Struct tags are limiting and make your code harder to read, especially when you need complex rules. Plus, the validation gets tied too tightly to your models, which isn't great for flexibility.
- Other Libraries (like Galidator): There are other libraries out there, but they often feel too complex for what they do. They require more setup and work than you’d expect, especially when you just want a simple, straightforward solution for validation.
Installation
Make sure you have Go installed on your machine.
Step 1: Create a New Go Module
- Create an empty folder with a name of your choice.
- Open a terminal, navigate (
cd
) into that folder, and initialize a new Go module:
go mod init example.com/learning
Step 2: Install Required Packages
Use go get to install the necessary packages.
- Install Gin:
go get -u github.com/gin-gonic/gin
- Install ginvalidator:
go get -u github.com/bube054/ginvalidator
Getting Started
One of the best ways to learn something is by example! So let's roll the sleeves up and get some coding happening.
Setup
The first thing that one needs is a Gin server running. Let's implement one that says hi to someone; for this, create a main.go
then add the following code:
package main
import (
"net/http"
"github.com/gin-gonic/gin"
)
func main() {
r := gin.Default()
r.GET("/hello", func(ctx *gin.Context) {
person := ctx.Query("person")
ctx.String(http.StatusOK, "Hello, %s!", person)
})
r.Run() // listen and serve on 0.0.0.0:8080
}
Now run this file by executing go run main.go
on your terminal.
go run main.go
The HTTP server should be running, and you can open http://localhost:8080/hello?person=John to salute John!
💡 Tip:
You can use Air with Go and Gin to implement live reload. These automatically restart the server whenever a file is changed, so you don't have to do this yourself!
Adding a validator
So the server is working, but there are problems with it. Most notably, you don't want to say hello to someone when the person's name is not set.
For example, going to http://localhost:8080/hello will print "Hello, "
.
That's where ginvalidator
comes in handy. It provides validators
, sanitizers
and modifiers
that are used to validate your request.
Let's add a validator and a modifier that checks that the person query string cannot be empty, with the validator named Empty and modifier named Not:
package main
import (
"net/http"
gv "github.com/bube054/ginvalidator"
"github.com/gin-gonic/gin"
)
func main() {
r := gin.Default()
r.GET("/hello", gv.NewQuery("person", nil).
Chain().
Not().
Empty(nil).
Validate(), func(ctx *gin.Context) {
person := ctx.Query("person")
ctx.String(http.StatusOK, "Hello, %s!", person)
})
r.Run()
}
📝 Note:
For brevity,gv
is used as an alias forginvalidator
in the code examples.
Now, restart your server, and go to http://localhost:8080/hello again. Hmm, it still prints "Hello, !"
... why?
Handling validation errors
ginvalidator
validation chain does not report validation errors to users automatically.
The reason for this is simple: as you add more validators
, or for more fields
, how do you want to collect the errors
? Do you want a list of all errors
, only one per field
, only one overall...?
So the next obvious step is to change the above code again, this time verifying the validation result with the ValidationResult
function:
package main
import (
"net/http"
gv "github.com/bube054/ginvalidator"
"github.com/gin-gonic/gin"
)
func main() {
r := gin.Default()
r.GET("/hello",
gv.NewQuery("person", nil).
Chain().
Not().
Empty(nil).
Validate(),
func(ctx *gin.Context) {
result, err := gv.ValidationResult(ctx)
if err != nil {
ctx.AbortWithStatusJSON(http.StatusInternalServerError, gin.H{
"message": "The server encountered an unexpected error.",
})
return
}
if len(result) != 0 {
ctx.AbortWithStatusJSON(http.StatusUnprocessableEntity, gin.H{
"errors": result,
})
return
}
person := ctx.Query("person")
ctx.String(http.StatusOK, "Hello, %s!", person)
})
r.Run()
}
Now, if you access http://localhost:8080/hello again, you’ll see the following JSON content, formatted for clarity:
{
"errors": [
{
"location": "queries",
"message": "Invalid value",
"field": "person",
"value": ""
}
]
}
Now, what this is telling us is that
- there's been exactly one error in this request;
- this field is called person;
- it's located in the query string (location:
"queries"
); - the error message that was given was
"Invalid value"
.
This is a better scenario, but it can still be improved. Let's continue.
Creating better error messages
All request location validators accept an optional second argument, which is a function used to format the error message. If nil
is provided, a default, generic error message will be used, as shown in the example above.
package main
import (
"net/http"
gv "github.com/bube054/ginvalidator"
"github.com/gin-gonic/gin"
)
func main() {
r := gin.Default()
r.GET("/hello",
gv.NewQuery("person",
func(initialValue, sanitizedValue, validatorName string) string {
return "Please enter your name."
},
).Chain().
Not().
Empty(nil).
Validate(),
func(ctx *gin.Context) {
result, err := gv.ValidationResult(ctx)
if err != nil {
ctx.AbortWithStatusJSON(http.StatusInternalServerError, gin.H{
"message": "The server encountered an unexpected error.",
})
return
}
if len(result) != 0 {
ctx.AbortWithStatusJSON(http.StatusUnprocessableEntity, gin.H{
"errors": result,
})
return
}
person := ctx.Query("person")
ctx.String(http.StatusOK, "Hello, %s!", person)
})
r.Run()
}
Now if you access http://localhost:8080/hello again, what you'll see is the following JSON content, with the new error message:
{
"errors": [
{
"location": "queries",
"message": "Please enter your name.",
"field": "person",
"value": ""
}
]
}
Accessing validated/sanitized data
You can use GetMatchedData
, which automatically collects all data that ginvalidator
has validated and/or sanitized. This data can then be accessed using the Get
method of MatchedData
:
package main
import (
"net/http"
gv "github.com/bube054/ginvalidator"
"github.com/gin-gonic/gin"
)
func main() {
r := gin.Default()
r.GET("/hello",
gv.NewQuery("person",
func(initialValue, sanitizedValue, validatorName string) string {
return "Please enter your name."
},
).Chain().
Not().
Empty(nil).
Validate(),
func(ctx *gin.Context) {
result, err := gv.ValidationResult(ctx)
if err != nil {
ctx.AbortWithStatusJSON(http.StatusInternalServerError, gin.H{
"message": "The server encountered an unexpected error.",
})
return
}
if len(result) != 0 {
ctx.AbortWithStatusJSON(http.StatusUnprocessableEntity, gin.H{
"errors": result,
})
return
}
data, err := gv.GetMatchedData(ctx)
if err != nil {
ctx.AbortWithStatusJSON(http.StatusInternalServerError, gin.H{
"message": "The server encountered an unexpected error.",
})
return
}
person, ok := data.Get(gv.QueryLocation, "person")
if !ok {
ctx.AbortWithStatusJSON(http.StatusInternalServerError, gin.H{
"message": "The server encountered an unexpected error.",
})
return
}
ctx.String(http.StatusOK, "Hello, %s!", person)
})
r.Run()
}
open http://localhost:8080/hello?person=John to salute John!
The available locations are BodyLocation
, CookieLocation
QueryLocation
, ParamLocation
and HeaderLocation
.
Each of these locations includes a String
method that returns the location where validated/sanitized data is being stored.
Sanitizing inputs
While the user can no longer send empty person names, it can still inject HTML into your page! This is known as the Cross-Site Scripting vulnerability (XSS).
Let's see how it works. Go to http://localhost:8080/hello?person=<b>John</b>
, and you should see "Hello, John!".
While this example is fine, an attacker could change the person query string to a <script> tag which loads its own JavaScript that could be harmful.
In this scenario, one way to mitigate the issue with ginvalidator is to use a sanitizer, most specifically Escape
, which transforms special HTML characters with others that can be represented as text.
package main
import (
"net/http"
gv "github.com/bube054/ginvalidator"
"github.com/gin-gonic/gin"
)
func main() {
r := gin.Default()
r.GET("/hello",
gv.NewQuery("person",
func(initialValue, sanitizedValue, validatorName string) string {
return "Please enter your name."
},
).Chain().
Not().
Empty(nil).
Escape(). // Added sanitizer
Validate(),
func(ctx *gin.Context) {
result, err := gv.ValidationResult(ctx)
if err != nil {
ctx.AbortWithStatusJSON(http.StatusInternalServerError, gin.H{
"message": "The server encountered an unexpected error.",
})
return
}
if len(result) != 0 {
ctx.AbortWithStatusJSON(http.StatusUnprocessableEntity, gin.H{
"errors": result,
})
return
}
data, err := gv.GetMatchedData(ctx)
if err != nil {
ctx.AbortWithStatusJSON(http.StatusInternalServerError, gin.H{
"message": "The server encountered an unexpected error.",
})
return
}
person, ok := data.Get(gv.QueryLocation, "person")
if !ok {
ctx.AbortWithStatusJSON(http.StatusInternalServerError, gin.H{
"message": "The server encountered an unexpected error.",
})
return
}
ctx.String(http.StatusOK, "Hello, %s!", person)
})
r.Run()
}
Now, if you restart the server and refresh the page, what you'll see is "Hello, <b>John<b>!". Our example page is no longer vulnerable to XSS!
⚠️ Caution:
ginvalidator
does not modifyhttp.Request
values during sanitization. To access sanitized data, always use theGetMatchedData
function.
The Validation Chain
The validation chain is one of the main concepts in ginvalidator, therefore it's useful to learn about it, so that you can use it effectively.
But don't worry: if you've read through the Getting Started guide, you have already used validation chains without even noticing!
What are validation chains?
Validation chains are created using the following functions, each targeting a specific location in the HTTP request:
-
NewBody
: Validates data from thehttp.Request
body. Its location isBodyLocation
. -
NewCookie
: Validates data from thehttp.Request
cookies. Its location isCookieLocation
. -
NewHeader
: Validates data from thehttp.Request
headers. Its location isHeaderLocation
. -
NewParam
: Validates data from the Gin route parameters. Its location isParamLocation
. -
NewQuery
: Validates data from thehttp.Request
query parameters. Its location isQueryLocation
.
They have this name because they wrap the value of a field with validations (or sanitizations), and each of its methods returns itself.
This pattern is usually called method chaining, hence why the name validation chain.
Validation chains not only have a number of useful methods for defining validations, sanitizations and modifications but they also have methods Validate
which returns the Gin
middleware handler function.
This is an example of how validation chains are usually used, and how you can read one:
r.Get(
"newsletter",
// For the `email` field in ctx.GetRawData()...
gv.NewBody("email", nil)
// the actual validation chain
.Chain()
// ...mark the field as optional
.Optional()
// ...and when it's present, trim its value, then validate it as an email address
.Trim("")
.Email(nil),
maybeSubscribeToNewsletter,
)
Features
A validation chain has three kinds of methods: validators
, sanitizers
and modifiers
.
Validators
determine if the value of a request field is valid. This means checking if the field is in the format that you expect it to be. For example, if you're building a sign up form, your requirements could be that the username must be an e-mail address, and that passwords must be at least 8 characters long.
If the value is invalid, an error is recorded for that field using some error message. This validation error can then be retrieved at a later point in the Gin route handler and returned to the user.
They are:
- CustomValidator
- Contains
- Equals
- AbaRouting
- After
- Alpha
- Alphanumeric
- Ascii
- Base32
- Base58
- Base64
- Before
- Bic
- Boolean
- BTCAddress
- ByteLength
- CreditCard
- Currency
- DataURI
- Date
- Decimal
- DivisibleBy
- EAN
- Empty
- EthereumAddress
- Float
- FQDN
- FreightContainerID
- FullWidth
- HalfWidth
- Hash
- Hexadecimal
- HexColor
- HSL
- IBAN
- IdentityCard
- IMEI
- In
- Int
- IP
- IPRange
- ISIN
- ISO4217
- ISO6346
- ISO6391
- ISO8601
- ISO31661Alpha2
- ISO31661Alpha3
- ISO31661Numeric
- ISRC
- ISSN
- JSON
- LatLong
- LicensePlate
- Locale
- LowerCase
- LuhnNumber
- MacAddress
- MagnetURI
- MailtoURI
- MD5
- MimeType
- MobilePhone
- MongoID
- Multibyte
- Numeric
- Octal
- PassportNumber
- Port
- PostalCode
- RFC3339
- RgbColor
- SemVer
- Slug
- StrongPassword
- TaxID
- SurrogatePair
- Time
- ULID
- UpperCase
- URL
- UUID
- VariableWidth
- VAT
- Whitelisted
- Matches
Sanitizers
transform the field value. They are useful to remove noise from the value and perhaps even to provide some basic line of defense against threats.
Sanitizers persist the updated fields value back into the Gin Contexts, so that it's usable by other ginvalidator functions, your own route handler code, and even other middlewares.
They are:
- CustomSanitizer
- Blacklist
- Escape
- LTrim
- NormalizeEmail
- RTrim
- StripLow
- ToBoolean
- ToDate
- ToFloat
- ToInt
- Trim
- Unescape
- Whitelist
Modifiers
define how validation chains behave when they are run.
They are:
📝 Note:
These methods are thoroughly documented using GoDoc within the pkg.go.dev ginvalidator documentation. If any details are unclear, you may also want to refer to related functions within thevalidatorgo
package for additional context, which I’ll be explaining below.
Standard validators/sanitizers
All of the functionality exposed by the validation chain actually comes from validatorgo, one of my other open source go packages which specializes in string validation/sanitation. Please check it out, star and share 🙏🙏🙏, Thank You.
This includes all of validatorgo
validators and sanitizers, from commonly used IsEmail
, IsLength
, and Trim
to the more niche IsISBN
, IsMultibyte
and StripLow
!
These are called standard validators
and standard sanitizers
in ginvalidator. But without the Is
prefix from validatorgo
.
Chaining order
The order in which you call methods on a validation chain usually matters.
They are almost always run in the order that they are specified, therefore you can tell what a validation chain will do just by reading its definition, from first chained method to last.
Take the following snippet as an example:
// Validate if search_query is not empty, then trim it
NewQuery("search_query", nil).Chain().Not().Empty().Trim("").Validate();
In this case, if the user passes a "search_query"
value that is composed of whitespaces only, it won't be empty, therefore the validation passes. But since the .Trim()
sanitizer is there, the whitespace's will be removed, and the field will become empty, so you actually end up with a false positive.
Now, compare it with the below snippet:
// Trim search_query, then validate if it's not empty
NewQuery("search_query", nil).Chain().Trim("").Not().Empty().Validate();
This chain will more sensibly remove whitespace's, and then validate if the value is not empty.
One exception to this rule is .Optional()
: It can be placed at any point in the chain and it will mark the chain as optional.
Reusing validation chains
If you wish to reuse the same chain, it's a good idea to return them from functions:
func createEmailValidator() gin.HandlerFunc {
return gv.NewBody("email", nil).Chain().Email(nil).Validate()
}
func handleLoginRoute(ctx *gin.Context) {
// Handle login route
}
func handleSignupRoute(ctx *gin.Context) {
// Handle signup route
}
r.POST("/login", createEmailValidator(), handleLoginRoute)
r.POST("/signup", createEmailValidator(), handleSignupRoute)
Field Selection
In ginvalidator
, a field is any value that is either validated or sanitized and it is string.
Pretty much every function or value returned by ginvalidator reference fields in some way. For this reason, it's important to understand the field path syntax both for when selecting fields for validation, and when accessing the validation errors or validated data.
Syntax
-
Body
fields are only valid for the following Content-Types:-
application/json
: This uses GJSON path syntax for extracting values. Please refer to the linked documentation for details. - Example:
{ "user": { "name": "John", "email": "john.doe@example.com" } }
With path
user.name
, the extracted value would be"John"
.-
application/x-www-form-urlencoded
: Typically used for HTML form submissions. Fields are submitted as key-value pairs in the body. - Example:
Content-Type: application/x-www-form-urlencoded
Body:
name=John&email=john.doe@example.com
Field
"name"
would have the value"John"
, and"email"
would have the value"john.doe@example.com"
.-
multipart/form-data
: Commonly used for file uploads or when submitting form data with files. - Example:
Content-Type: multipart/form-data
Body:
--boundary Content-Disposition: form-data; name="name" John --boundary Content-Disposition: form-data; name="file"; filename="resume.pdf" Content-Type: application/pdf [binary data] --boundary--
Field
"name"
would have the value"John"
, and"file"
would be the uploaded file. -
-
Query
fields correspond to URL search parameters, and their values are automatically url unescaped by Gin.
Examples:- Field:
"name"
, Value:"John"
/hello?name=John
- Field:
"full_name"
, Value:"John Doe"
/hello?full_name=John%20Doe
- Field:
-
Param
fields represent URL path parameters, and their values are automatically unescaped byginvalidator
.
Example:- Field:
"id"
, Value:"123"
/users/:id
- Field:
-
Header
fields are HTTP request headers, and their values are not unescaped. A log warning will appear if you provide a non-canonical header key.
Example:- Field:
"User-Agent"
, Value:"Mozilla/5.0"
Header: "User-Agent", Value: "Mozilla/5.0"
- Field:
-
Cookies
fields are HTTP cookies, and their values are automatically url unescaped by Gin.
Example:- Field:
"session_id"
, Value:"abc 123"
Cookie: "session_id=abc%20123"
- Field:
Customizing express-validator
If the server you're building is anything but a very simple one, you'll need validators
, sanitizers
and error messages beyond the ones built into ginvalidator sooner or later.
Custom Validators and Sanitizers
A classic need that ginvalidator can't fulfill for you, and that you might run into, is validating whether an e-mail address is in use or not when a user signing up.
It's possible to do this in ginvalidator by implementing a custom validator.
A CustomValidator
is a method available on the validation chain, that receives a special function CustomValidatorFunc, and have to returns a boolean that will determine if the field is valid or not.
A CustomSanitizer
is also a method available on the validation chain, that receives a special function CustomSanitizerFunc, and have to returns the new sanitized value.
Implementing a custom validator
A CustomValidator
can be asynchronous by using goroutines and a sync.WaitGroup
to handle concurrent operations. Within the validator, you can spin up goroutines for each asynchronous task, adding each task to the WaitGroup. Once all tasks complete, the validator should return a boolean.
For example, in order to check that an e-mail is not in use:
func isUserPresent(email string) bool {
return email == "existing@example.com"
}
r.POST("/create-user",
gv.
NewBody("email", nil).
Chain().
CustomValidator(
func(req *http.Request, initialValue, sanitizedValue string) bool {
var exists bool
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
exists = isUserPresent(sanitizedValue)
}()
wg.Wait()
return !exists
},
).
Validate(),
func(ctx *gin.Context) {
// Handle the request
},
)
Or maybe you could also verify that the password matches the repeat:
type createUser struct {
Password string `json:"password"`
PasswordConfirmation string `json:"passwordConfirmation"`
}
r.POST("/create-user",
gv.NewBody("password", nil).
Chain().
Matches(regexp.MustCompile(`^[A-Za-z\d]{8,}$`)).
Validate(),
gv.NewBody("passwordConfirmation", nil).
Chain().
CustomValidator(func(req *http.Request, initialValue, sanitizedValue string) bool {
data, err := io.ReadAll(req.Body)
if err != nil {
return false
}
// Refill the request body to allow further reads, if needed.
req.Body = io.NopCloser(bytes.NewBuffer(data))
var user createUser
json.Unmarshal(data, &user)
return sanitizedValue == user.PasswordConfirmation
}).
Validate(),
func(ctx *gin.Context) {
// Handle request
},
)
⚠️ Caution:
If the request body will be accessed multiple times—whether in the same validation chain, in another validation chain for the same request context, or in subsequent handlers—ensure you reset the request body after each read. Failing to do so can lead to errors or missing data when the body is read again.
Implementing a custom sanitizer
CustomSanitizer
don't have many rules. Whatever the value that they return, is the new value that the field will acquire.
Custom sanitizers can also be asynchronous by using goroutines and a sync.WaitGroup
to handle concurrent operations.
r.POST("/user/:id",
gv.NewParam("id", nil).
Chain().
CustomSanitizer(
func(req *http.Request, initialValue, sanitizedValue string) string {
return strings.Repeat(sanitizedValue, 3) // some string manipulation
},
).
Validate(),
func(ctx *gin.Context) {
// Handle request
},
)
Error Messages
Whenever a field value is invalid, an error message is recorded for it.
The default error message is "Invalid value"
, which is not descriptive at all of what the error is, so you might need to customize it. You can customize by
gv.NewBody("email",
func(initialValue, sanitizedValue, validatorName string) string {
switch validatorName {
case gv.EmailValidatorName:
return "Email is not valid."
case gv.EmptyValidatorName:
return "Email is empty."
default:
return gv.DefaultValChainErrMsg
}
},
).
Chain().
Not().Empty(nil).
Email(nil).
Validate()
-
initialValue
is the original value extracted from the request (before any sanitization). -
sanitizedValue
is the value after it has been sanitized (if applicable). -
validatorName
is the name of the validator that failed, which helps identify the validation rule that did not pass.
For a complete list of validator names, refer to the ginvalidator constants.
Maintainers
- bube054 - Attah Gbubemi David (author)
Top comments (1)
So what did u think? Let me know 👇👇👇