DEV Community

loading...
Cover image for Go Interval job timer, with centralized Lock for Distributed Systems

Go Interval job timer, with centralized Lock for Distributed Systems

Jay Ehsaniara
Software Architect
Updated on ・3 min read

goInterLock

known as: ⏰ Interval (Cron / Job / Task / Scheduler) ⏱️

GoInterval job timer, with centralize Lock

goInterLock is schedule tasks (cron-job) with a centralized locking mechanism for Go (AKA Golang). In the distributed systems, the lock prevents the tasks to been executed in every instant that has the scheduler.

For example, if your application has a scheduled where it's calling some external API or doing some expensive DataBase querying every 10 minutes, the lock prevents the process to be run in every instance of that application, and you ended up running that task multiple times every 10 minutes. 

Ideal use cases:

  • Kubernetes
  • Docker Swarm
  • AWS ECS Fargate Quick Start
go get github.com/ehsaniara/gointerlock
Enter fullscreen mode Exit fullscreen mode

Supported Lock

Local Scheduler (Single App)

(Interval every 2 seconds)

var job = gointerlock.GoInterval{
    Interval: 2 * time.Second,
    Arg:      myJob,
}
err := job.Run(ctx)
if err != nil {
        log.Fatalf("Error: %s", err)
}
Enter fullscreen mode Exit fullscreen mode

Examples

Basic Local Task: Simple Task Interval (Single App).

Application Cache: An example of periodically cached value update on http server.


Distributed Mode (Scaled Up)

Redis

Existing Redis Connection

you should already configure your Redis connection and pass it into the GoInterLock. Also make sure you are giving the
unique name per job

Step 1: configure redis connection redisConnection.Rdb from the existing application and pass it into the Job. for example:

var redisConnector = redis.NewClient(&redis.Options{
    Addr:     "localhost:6379",
    Password: "myRedisPassword", 
    DB:       0,               
})
Enter fullscreen mode Exit fullscreen mode

Step 2: Pass the redis connection into the GoInterval

var job = gointerlock.GoInterval{
    Interval: 2 * time.Second,
    Arg:      myJob,
    Name:     "MyTestJob",
    RedisConnector: redisConnector,
}
err := jobTicker.Run(ctx)
if err != nil {
    log.Fatalf("Error: %s", err)
}
Enter fullscreen mode Exit fullscreen mode

in both examples myJob is your function, for example:

func myJob() {
    fmt.Println(time.Now(), " - called")
}
Enter fullscreen mode Exit fullscreen mode

Note: currently GoInterLock does not support any argument for the job function

Built in Redis Connector

another way is to use an existing redis connection:

var job = gointerlock.GoInterval{
    Name:          "MyTestJob",
    Interval:      2 * time.Second,
    Arg:           myJob,
    RedisHost:     "localhost:6379",
    RedisPassword: "myRedisPassword", //if no pass leave it as ""
}
err := job.Run(context.Background())
if err != nil {
    log.Fatalf("Error: %s", err)
}
Enter fullscreen mode Exit fullscreen mode
GoInterLock is using go-redis for Redis Connection.

Examples

Basic Distributed Task: Simple Task Interval with Redis Lock.


AWS DynamoDb

Basic Config (Local Environment)

This ia sample of local DynamoDb (Docker) for your local test.

var job = gointerlock.GoInterval{
    Name:                       "MyTestJob",
    Interval:                   2 * time.Second,
    Arg:                        myJob,
    LockVendor:                 gointerlock.AwsDynamoDbLock,
    AwsDynamoDbRegion:          "us-east-1",
    AwsDynamoDbEndpoint:        "http://127.0.0.1:8000",
    AwsDynamoDbSecretAccessKey: "dummy",
    AwsDynamoDbAccessKeyID:     "dummy",
}
err := job.Run(cnx)
if err != nil {
    log.Fatalf("Error: %s", err)
}
Enter fullscreen mode Exit fullscreen mode

task:

func myJob() {
    fmt.Println(time.Now(), " - called")
}
Enter fullscreen mode Exit fullscreen mode

Note: you can get the docker-compose file from AWS DynamoDB Docker compose or you can get it from: docker-compose.yml.

Using AWS Profile

goInterLock will get credentials from the AWS profile

var job = gointerlock.GoInterval{
    Name:                       "MyTestJob",
    Interval:                   2 * time.Second,
    Arg:                        myJob,
    LockVendor:                 gointerlock.AwsDynamoDbLock,
}
err := job.Run(cnx)
if err != nil {
    log.Fatalf("Error: %s", err)
}
Enter fullscreen mode Exit fullscreen mode

Examples

Basic Distributed Task: Simple Task Interval with DynamoDb Lock.

Discussion (0)