09 - Token Creation

We just implemented a Signup method in our UserService and a Create method in our UserRepository. We can see these completed features in the diagram below!

Referencing the diagram, we can see that we have two tasks to perform in order to send POST requests to our endpoint (/api/account/signup):

1. Create an implementation of the NewPairFromUser method of the TokenService.
2. Establish a connection to our development Postgres server from inside of our application, and perform dependency injection so that we can use concrete implementations of our services and repository. We should then be able to run docker-compose up to get our reload server environment running and make calls to our signup endpoint.

We'll complete number 1 today!

If at any point you are confused about file structure or code, go to Github repository and check out the branch for the previous lesson to be in sync with me!

If you prefer video, check out the video version below!

Create NewPairFromUser in TokenService

Let's go back to the Signup handler in ~/model/interfaces.go to remind ourselves of the call signature of NewPairFromUser.

type TokenService interface {
    NewPairFromUser(ctx context.Context, u *User, prevTokenID string) (*TokenPair, error)
}

The second parameter is a User. This user will be created in the Signup handler before we call NewPairFromUser. The final parameter, prevTokenID is for passing the ID of the user's current refresh token so that it can be invalidated before providing a new refresh token. We pass an empty string for this parameter when signing up a user as the user won't yet have a token.

We're not going handle this token ID for a couple of tutorials, as it requires setting up a Redis database and a TokenRepository.

Let's create a file, ~/service/token_service.go, for our TokenService implementation.

Inside, create a TokenService struct to define this service's dependencies.

// TokenService used for injecting an implementation of TokenRepository
// for use in service methods along with keys and secrets for
// signing JWTs
type TokenService struct {
    // TokenRepository model.TokenRepository
    PrivKey       *rsa.PrivateKey
    PubKey        *rsa.PublicKey
    RefreshSecret string
}

// TSConfig will hold repositories that will eventually be injected into this
// this service layer
type TSConfig struct {
    // TokenRepository model.TokenRepository
    PrivKey       *rsa.PrivateKey
    PubKey        *rsa.PublicKey
    RefreshSecret string
}

// NewTokenService is a factory function for
// initializing a UserService with its repository layer dependencies
func NewTokenService(c *TSConfig) model.TokenService {
    return &TokenService{
        PrivKey:       c.PrivKey,
        PubKey:        c.PubKey,
        RefreshSecret: c.RefreshSecret,
    }
}

// NewPairFromUser creates fresh id and refresh tokens for the current user
// If a previous token is included, the previous token is removed from
// the tokens repository
func (s *TokenService) NewPairFromUser(ctx context.Context, u *model.User, prevTokenID string) (*model.TokenPair, error) {
    panic("Not implemented")
}

Note: I've been making a mistake in creating the struct containers for the various services and repository. The TokenService should be package private, and therefore lowercase. The factory will return a model.TokenService interface. Making the TokenService struct package private will make it so the only way to instantiate the service is through the factory, and not by directly creating a struct. I'll clean this up in two tutorials!

What are the public and private keys, and refresh secret, in the TokenService? We'll be using JSON Web Tokens as our authorization mechanism. Let's do a little primer about what these are so we can then fill in the implementation details for the NewPairFromUser method.

JSON Web Token (JWT) Explanation

I highly recommend you check out the jwt.io Introduction as that's what I'm using for this explanation.

How They Are Used

For authorizing users to use resources in this application, or other applications/services within our organization. From jwt.io: "Once the user is logged in, each subsequent request will include the JWT, allowing the user to access routes, services, and resources that are permitted with that token."

Token Structure

It's just a 3-part string, xxxxx.yyyyy.zzzzz, where:

• xxxxx represents a "header".
• yyyyy represents a "payload".
• zzzzz represents a "signature".

The header and payload are written in JSON, and then encoded as Base64URLs. When the JWT is encoded and signed, it looks something like:
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c.

Header

The header tells what signing algorithm was used.

{
  "alg": "HS256",
  "typ": "JWT"
}

We're going to use RS256 (RSA) for signing the idToken and HS256 (HMAC) for signing the refreshToken.

Payload

The payload's JSON keys provide information about the user. There are some registered claims which most JWT libraries support by default (like "exp," the expiration time). You can add your own set of claims, some of which are standardized, or registered.

Signature

From jwt.io:

"The signature is used to verify the message wasn't changed along the way, and, in the case of tokens signed with a private key, it can also verify that the sender of the JWT is who it says it is."

As an example, let's say the payload has an isAdmin boolean key that is set to false. If someone were to steal this token and attempt to change isAdmin to true, the token would no longer be valid with the given signature and private signing key.

Note however, that the JWT is easily readable since Base64Urls can easily be read (for example, by copying it into a converter online or in libraries in most programming languages). Therefore, you do not want to include sensitive information in the token (like a National ID or SSN, date of birth, etc).

ID and RefreshTokens in Our App

Remember that we have defined a TokenPair to return upon successful signup. Let's take a look at another diagram of our authorization flow to understand how these are used.

One of the two purposes of this account application is to authorize users to access various services or applications in our domain.

When a user supplies a valid email and password combination (authenticated), the account application provides two tokens. Both of these tokens are JWTs, but they serve different purposes.

idToken

The idToken provides the information from the model.User we've been using in our application. Be careful not to return any sensitive user information as I previously mentioned.

The idToken is "short-lived." We'll make this token valid for 15 minutes to limit the time frame over which damage can be done if this token is stolen. You can make this longer or shorter in your application.

The idToken will be signed with an rsa private key, and verified with an rsa public key. We'll create this "key pair" soon. The public key will be provided to the various applications in our organization that need to verify that a user is authorized.

refreshToken

The refreshToken will be valid for a longer time frame. We'll set it to 3 days in our application. The purpose of this token is to improve the user-experience of the application so that they need not continually log in. If the user's idToken is expired, or if a client application loads for the first time, it may send its refreshToken to our Account Application to request a new idToken.

There are some major differences between these 2 tokens.

• The refreshToken will not hold any user information beyond an id.
• The refreshToken is only verified in the account application. We don't provide the secret to other applications or services. We'll also sign this token with a secret string (HMAC), instead of an RSA key. This is a symmetric signing algorithm, meaning the same key is used for signing and verifying JWTs.
• We'll eventually store a list of valid refresh tokens in Redis. If a user feels their account is compromised, they can authenticate by logging in, and choose to log out of all devices. This will invalidate all of their refresh tokens. idTokens are not stored anywhere, but expire in 15 minutes.

An Important Note

There is a "friendly" debate about the failings of JWTs as well as how to store them client-side. I previously addressed this at the end of Tutorial 07.

Beyond using JWTs for authorization, you might consider requiring re-authentication for important actions like changing sensitive user details or performing financial transactions. There are also considerations such as using 2-factor authentication, which we won't implement.

Adding Token Utility Functions

We'll create some token utility functions in the same way we did for passwords.

Create ~/service/tokens.go.

We'll be working with a new package called github.com/dgrijalva/jwt-go. So make sure to include this in your imports and/or run go get github.com/dgrijalva/jwt-go.

Let's now add code for generating tokens. We'll add some functions for verifying tokens when we add signin and token refreshing features to our API.

idToken Generation

package service

import (
    "crypto/rsa"
    "log"
    "time"

    "github.com/dgrijalva/jwt-go"
    "github.com/google/uuid"
    "github.com/jacobsngoodwin/memrizr/account/model"
)

// IDTokenCustomClaims holds structure of jwt claims of idToken
type IDTokenCustomClaims struct {
    User *model.User `json:"user"`
    jwt.StandardClaims
}

// generateIDToken generates an IDToken which is a jwt with myCustomClaims
// Could call this GenerateIDTokenString, but the signature makes this fairly clear
func generateIDToken(u *model.User, key *rsa.PrivateKey) (string, error) {
    unixTime := time.Now().Unix()
    tokenExp := unixTime + 60*15 // 15 minutes from current time

    claims := IDTokenCustomClaims{
        User: u,
        StandardClaims: jwt.StandardClaims{
            IssuedAt:  unixTime,
            ExpiresAt: tokenExp,
        },
    }

    token := jwt.NewWithClaims(jwt.SigningMethodRS256, claims)
    ss, err := token.SignedString(key)

    if err != nil {
        log.Println("Failed to sign id token string")
        return "", err
    }

    return ss, nil
}

Inside we do the following:

1. Define the structure of the payload portion the the JWT. If you recall, there are some predefined claims that JWTs can use, which our imported JWT library implements with jwt.StandardClaims. We also add properties of our User model as "public" or "custom" claims.
2. We set an expiry time of 15 minutes, using Unix time in seconds. (Don't let me forget to set this with an environment variable at some point 🤣).
3. We then create a Token, which is a type from the JWT library, then call the SignedString method to sign it with our RSA private key.
4. We return this signed string, ss.

refreshToken Generation

Refresh tokens will be generated in a similar manner with a few differences.

// RefreshToken holds the actual signed jwt string along with the ID
// We return the id so it can be used without re-parsing the JWT from signed string
type RefreshToken struct {
    SS        string
    ID        string
    ExpiresIn time.Duration
}

// RefreshTokenCustomClaims holds the payload of a refresh token
// This can be used to extract user id for subsequent
// application operations (IE, fetch user in Redis)
type RefreshTokenCustomClaims struct {
    UID uuid.UUID `json:"uid"`
    jwt.StandardClaims
}

// generateRefreshToken creates a refresh token
// The refresh token stores only the user's ID, a string
func generateRefreshToken(uid uuid.UUID, key string) (*RefreshToken, error) {
    currentTime := time.Now()
    tokenExp := currentTime.AddDate(0, 0, 3) // 3 days
    tokenID, err := uuid.NewRandom()         // v4 uuid in the google uuid lib

    if err != nil {
        log.Println("Failed to generate refresh token ID")
        return nil, err
    }

    claims := RefreshTokenCustomClaims{
        UID: uid,
        StandardClaims: jwt.StandardClaims{
            IssuedAt:  currentTime.Unix(),
            ExpiresAt: tokenExp.Unix(),
            Id:        tokenID.String(),
        },
    }

    token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
    ss, err := token.SignedString([]byte(key))

    if err != nil {
        log.Println("Failed to sign refresh token string")
        return nil, err
    }

    return &RefreshToken{
        SS:        ss,
        ID:        tokenID.String(),
        ExpiresIn: tokenExp.Sub(currentTime),
    }, nil
}

1. We will not return the signed string directly. Instead, we return a RefreshToken that we define in a struct. We do this to make reaching out to a TokenRepository (we'll implement this later) a bit easier. We also return an ExpiresIn field as we will use this for expiring old refresh tokens in Redis. We'll still only send the "signed string" in the HTTP response, as shown in our previous diagram.
2. We make use of the ID property (a stringified UUID) in standard claims. This id will be used to identify valid tokens in Redis.
3. We sign the token using the HS256 algorithm by using passing in our secret as a byte slice.

Now let's see how we can generate the RSA key pair!

Generating Public and Private Keys

I'm going to use the technique recommended by Google Cloud for generating 2048-bit RSA Key pairs. We'll then add these lines to a Makefile at our project root as follows:

You may need to download Make and OpenSSL to get this to work in Windows. I did it once, and it was a pain in the butt. Sorry I'm too lazy to write a tutorial now 😢. But it is possible, and probably easier than these tutorials!

Create a "Makefile" at project root (one directory above "account").

.PHONY: create-keypair

PWD = $(shell pwd)
ACCTPATH = $(PWD)/account

create-keypair:
    @echo "Creating an rsa 256 key pair"
    openssl genpkey -algorithm RSA -out $(ACCTPATH)/rsa_private_$(ENV).pem -pkeyopt rsa_keygen_bits:2048
    openssl rsa -in $(ACCTPATH)/rsa_private_$(ENV).pem -pubout -out $(ACCTPATH)/rsa_public_$(ENV).pem

Make sure to add *.pem to your .gitignore file at the project root to exclude all keys from being committed to your repository!

.env.dev
*.pem

We can then create keys with make create-keypair ENV=test, where you can replace the value of ENV with the environment for which you want to create the keypair.

Call Utility Methods in TokenService

With our utility functions completed, we can complete the implementation of NewPairFromUser! We'll generate both of our tokens and check for any errors. If everything works, we return the TokenPair.

func (s *TokenService) NewPairFromUser(ctx context.Context, u *model.User, prevTokenID string) (*model.TokenPair, error) {
    // No need to use a repository for idToken as it is unrelated to any data source
    idToken, err := generateIDToken(u, s.PrivKey)

    if err != nil {
        log.Printf("Error generating idToken for uid: %v. Error: %v\n", u.UID, err.Error())
        return nil, apperrors.NewInternal()
    }

    refreshToken, err := generateRefreshToken(u.UID, s.RefreshSecret)

    if err != nil {
        log.Printf("Error generating refreshToken for uid: %v. Error: %v\n", u.UID, err.Error())
        return nil, apperrors.NewInternal()
    }

    // TODO: store refresh tokens by calling TokenRepository methods

    return &model.TokenPair{
        IDToken:      idToken,
        RefreshToken: refreshToken.SS,
    }, nil
}

Add Test for NewTokenPairFromUser

Let's create the test file for this service, ~/service/token_service_test.go, and add a single test case to make sure we get a token pair when we call NewPairFromUser!

package service

import (
    "context"
    "io/ioutil"
    "testing"
    "time"

    "github.com/stretchr/testify/assert"

    "github.com/dgrijalva/jwt-go"
    "github.com/google/uuid"
    "github.com/jacobsngoodwin/memrizr-tutorial-script/account/model"
)

func TestNewPairFromUser(t *testing.T) {
    priv, _ := ioutil.ReadFile("../rsa_private_test.pem")
    privKey, _ := jwt.ParseRSAPrivateKeyFromPEM(priv)
    pub, _ := ioutil.ReadFile("../rsa_public_test.pem")
    pubKey, _ := jwt.ParseRSAPublicKeyFromPEM(pub)
    secret := "anotsorandomtestsecret"

    // instantiate a common token service to be used by all tests
    tokenService := NewTokenService(&TSConfig{
        PrivKey:       privKey,
        PubKey:        pubKey,
        RefreshSecret: secret,
    })

    // include password to make sure it is not serialized
    // since json tag is "-"
    uid, _ := uuid.NewRandom()
    u := &model.User{
        UID:      uid,
        Email:    "bob@bob.com",
        Password: "blarghedymcblarghface",
    }

    t.Run("Returns a token pair with values", func(t *testing.T) {
        ctx := context.TODO()
        tokenPair, err := tokenService.NewPairFromUser(ctx, u, "")
        assert.NoError(t, err)

        var s string
        assert.IsType(t, s, tokenPair.IDToken)

        // decode the Base64URL encoded string
        // simpler to use jwt library which is already imported
        idTokenClaims := &IDTokenCustomClaims{}

        _, err = jwt.ParseWithClaims(tokenPair.IDToken, idTokenClaims, func(token *jwt.Token) (interface{}, error) {
            return pubKey, nil
        })

        assert.NoError(t, err)

        // assert claims on idToken
        expectedClaims := []interface{}{
            u.UID,
            u.Email,
            u.Name,
            u.ImageURL,
            u.Website,
        }
        actualIDClaims := []interface{}{
            idTokenClaims.User.UID,
            idTokenClaims.User.Email,
            idTokenClaims.User.Name,
            idTokenClaims.User.ImageURL,
            idTokenClaims.User.Website,
        }

        assert.ElementsMatch(t, expectedClaims, actualIDClaims)
        assert.Empty(t, idTokenClaims.User.Password) // password should never be encoded to json

        expiresAt := time.Unix(idTokenClaims.StandardClaims.ExpiresAt, 0)
        expectedExpiresAt := time.Now().Add(15 * time.Minute)
        assert.WithinDuration(t, expectedExpiresAt, expiresAt, 5*time.Second)

        refreshTokenClaims := &RefreshTokenCustomClaims{}
        _, err = jwt.ParseWithClaims(tokenPair.RefreshToken, refreshTokenClaims, func(token *jwt.Token) (interface{}, error) {
            return []byte(secret), nil
        })

        assert.IsType(t, s, tokenPair.RefreshToken)

        // assert claims on refresh token
        assert.NoError(t, err)
        assert.Equal(t, u.UID, refreshTokenClaims.UID)

        expiresAt = time.Unix(refreshTokenClaims.StandardClaims.ExpiresAt, 0)
        expectedExpiresAt = time.Now().Add(3 * 24 * time.Hour)
        assert.WithinDuration(t, expectedExpiresAt, expiresAt, 5*time.Second)
    })
}

In the test we do the following:

1. Add setup code which reads in our test RSA keys and secret, initializes a TokenService, and creates an example User.
2. We assert that the returned idToken is a string.
3. We then parse the JWT (decode the Base64-encoded value), and verify that the token's "claims" contain the fields from the user.
4. However, we also assert that the password is not part of the claims as that would be very, very bad! Our User model's JSON tags are set so that the password should never be sent as JSON.
5. We check that the expiration time is within 5 seconds of 15 minutes.
6. We make the same steps for the refreshToken, except we don't have as many fields to verify, and assert an expiry time of 3 days.

Conclusion

Next time, we're going to create to run our Postgres database and create a users table with columns corresponding to the fields of our User model. After we successfully migrate create this table, we'll initialize a database connection and perform dependency injection.

With any luck, we'll then be able to fire up our application and sign up an actual user!

After that, I think we'll do a little bit of cleanup and maintenance before moving on to adding more functionality.

To be honest, it you get this part, the rest should begin to move a bit more quickly!

See you next time!