Microservices with pebl — a complete and free cloud platform

Overview

In this guide I want to introduce pebl, a complete and free cloud platform.

Unlike the normal approach to Infrastructure as Code, pebl doesn't rely on declaring a set of yaml files to control your cloud. Instead pebl embeds cloud capabilities within your application through language specific SDKs.

Getting Started

Signing Up

So first we need to sign up for a free account at pebl.io.

Choose your desired identity provider and follow the steps. And make sure to go through claiming your free *.pebl.rocks subdomain! We'll be using this for the rest of the tutorial.

Docker & pebl CLI

There are only two system requirements for pebl, which is Docker and the pebl CLI. If you don't have Docker installed, head over to the docker website and follow their directions.

For pebl CLI, choose to correct link on the pebl's setup docs. Make sure you select the correct download link for your own system! If you are using a Mac you can check if your system is an Intel or M1 by going through the Apple Menu then About This Mac.

Verify that the CLI was installed correctly by running pebl without any args:

$ pebl
usage:

    pebl {command}

available commands:

       up  start the local pebl cluster.
     down  stop the local pebl cluster.
     info  attach and show info about the local pebl cluster.
      run  execute a pebl program locally.
   deploy  execute a pebl program in the cloud.
   tunnel  creates a tunnel to a redis instance.
     auth  manages the credentials needed for certain commands.
  version  print the version of the cli.

$

Simple Service

The core building block that pebl provides is a service. On other platforms it might be referred to as a serverless application. In essence pebl's service is a server that can respond to requests.

Now let's dive into pebl by creating a very simple service using Python.

Hello World!

Create a scratch project folder somewhere on your system. In our examples we will be using ~/work/scratch:

$ mkdir -p ~/work/scratch

But feel free to choose another scheme that makes sense for your own setup. This folder will act as the root of our project.

Now let's create a subfolder that will contain our simple hello world service.

mkdir ~/work/scratch/hello

Create a main.py (~/work/scratch/hello/main.py) that will hold the main method and paste in the following:

from flask import Flask

app = Flask(__name__)

@app.route("/")
def root():
    return "hello, world!\n"

import pebl
pebl.service(app, "hey.pebl.rocks")  # put your own domain here

The first section of main.py should look very familiar if you've used Flask before. It's a very simple Flask definition, with a single handler on the root path. The last two lines are where we declare that this Flask application should be a service, hosted at your *.pebl.rocks endpoint.

At this point you might expect that we have to create some separate pebl specific yaml. But really this single method call is all it takes to define a service with pebl! So to recap:

1. Cloud capabilities (pebl.service) are incorporated into the application as regular code
2. You can rely on any other Python library without additional configuration, like Flask

Now to run this, we will be relying on pebl's base Docker image to create a re-usable container. Go ahead and create a Dockerfile (~/work/scratch/hello/Dockerfile) with the following content:

FROM peblcloud/python
COPY . .
RUN pip install flask
ENTRYPOINT python -u main.py

Local Runtime

Pebl let's you execute your code locally to ensure correctness and to provide a super fast iteration cycle. First start a local cluster that will contain all our local workloads with pebl up:

$ pebl up
 :: initializing new pebl cluster...
 :: done!
 :: run `pebl info` to see logs, endpoints, and other info about the running cluster!
$

Once the cluster is configured, navigate to the hello subfolder and execute pebl run:

$ cd ~/work/scratch/hello
$ pebl run
 :: building docker project...
 :: starting container...
$

Then run pebl info to see the state of the running cluster.

You should see something like this, but with your own *.pebl.rocks subdomain instead of hey.pebl.rocks. Go ahead and try sending a request to your server.

$ curl localhost:32807
hello, world!
$

Make sure you put in the correct port as shown on your info pane!

Cloud Runtime

By embedding your cloud infrastructure usage into your application, pebl unlocks a key benefit: The ability to deploy the exact same application to different runtimes, without any modifications.

In other words, there's no longer a need to do any environment specific checks or define multiple config files for local vs. production deploys.

So pebl makes it super easy to migrate your applications to the cloud once you are done testing them locally. Just run pebl deploy in a similar fashion to pebl run:

$ cd ~/work/scratch/hello
$ pebl deploy

If you get authentication errors, make sure to run pebl auth and follow the steps!

Once the deploy is successful, try sending a request with curl. Make sure to use https as all services in the cloud get TLS by default, and pebl disallows non-TLS traffic.

$ curl https://hey.pebl.rocks
hello, world!
$

You can also head over to the pebl console to get a report of all incoming traffic to your endpoints.

Microservices

A common way to structure cloud workloads is to break them into small services, each serving a disparate part of the larger system. With pebl we can achieve that by utilizing the internal services capability.

Internal services are similar to pebl.service that we used previously, with one key difference. They are only exposed to your other workloads, and are not configured for external traffic.

So in this section we will explore adding an internal service, and utilizing Go instead of Python to highlight the language agnostic feature of pebl.

Go Setup

Let's create a subfolder within our scratch folder to hold our Go setup.

$ mkdir ~/work/scratch/go

Then initialize a Go project using go mod init:

$ cd ~/work/scratch/go
$ go mod init scratch

And unlike Python, since Go is compiled we actually don't need a Dockerfile. Instead we need to download the pebl package for Go:

$ cd ~/work/scratch/go
$ go get github.com/peblcloud/go

User Service

And to motivate a real usage, we will be creating an internal Go service that acts as a user service, one that handles managing user data. Create a main.go (~/work/scratch/go/main.go):

package main

import (
    "github.com/peblcloud/go"
    "net/http"
)

func main() {
    mux := http.NewServeMux()
    mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        w.Write([]byte("this is an internal service!\n"))
    })

    pebl.InternalService(mux, "go.internal")
}

So just as we had done previously with Python, we are creating a server object based on the net/http. Then passing configured server object into pebl's SDK with the pebl.InternalService call.

Note that unlike the serivce we defined previously, we are providing a seemingly invalid domain of go.internal. For internal services the endpoint can be anything, though we recommend you use an easy-to-remember scheme, such as a .local or .internal TLD.

Let's try running this! Same as before execute pebl run to run it locally:

$ cd ~/work/scratch/go
$ pebl run
 :: building golang project...
 :: staging build at: /var/folders/92/_vg8jtqx7kncxb_9jlpw8rmm0000gn/T/3019178735
 :: containerizing the build...
 :: starting container...
$

Now when you inspect the info pane, you should see another entry tagged as internal:

You can send requests to this new internal endpoint with curl:

$ curl localhost:32808
this is an internal service!
$

Note that the ability to send requests like this to internal service is provided out of convenience while running locally. When deployed to the cloud runtime, you won't be able to access the internal services from outside the cluster.

Redis

Now our internal Go service isn't very interesting because it doesn't really do anything based on our request. In order to support a much wider set of applications, pebl provides persistence through Redis. And just as we have done so far, we unlock this capability through the SDK.

So let's update the Go service (~/work/scratch/go/main.go) by incorporating Redis.

package main

import (
    "context"
    "encoding/json"
    "fmt"
    "net/http"
    "net/url"

    "github.com/peblcloud/go"
    "github.com/redis/go-redis/v9"
)

func main() {
    connInfo, _ := pebl.Redis("users")
    client := redis.NewClient(&redis.Options{
        Addr: fmt.Sprintf("%s:%d", connInfo.Host, connInfo.Port),
    })

    mux := http.NewServeMux()
    mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        query, _ := url.ParseQuery(r.URL.RawQuery)
        name := query.Get("name")
        userInfo, _ := client.HGetAll(context.Background(), name).Result()
        payload, _ := json.Marshal(userInfo)
        w.Write(payload)
    })

    pebl.InternalService(mux, "go.internal")
}

Then update our module dependencies by running go mod tidy, which should automatically pull down github.com/redis/go-redis/v9. If for some reason this isn't working, you can manually run go get github.com/redis/go-redis/v9.

Now run this Go service by executing pebl run inside ~/work/scratch/go/. Your info pane should now list the redis instance we are using named users.

If you try and send requests to your internal Go service at this point, you'll always get an empty result because our redis instance has no user data yet.

$ curl 'localhost:32808?name=alice'
{}
$ curl 'localhost:32808?name=bob'
{}

But you can quickly manage the data in your local redis by using the exposed port shown on the info pane.

$ redis-cli -p 32809
127.0.0.1:32809> hset alice name alice email alice@example.org
(integer) 2
127.0.0.1:32809> hset bob name bob email bob@example.org
(integer) 2

Now when you send requests with either alice or bob as the query, you'll get their user info json in the response:

$ curl 'localhost:32808?name=alice'
{"email":"alice@example.org","name":"alice"}
$

Proxying

Now to utilize this internal service, we will be proxying a subset of requests from our external Python service to our intenal Go service. Let's update our main.py in the ~/work/scratch/hello/ folder:

from flask import Flask
import requests

app = Flask(__name__)

@app.route("/")
def root():
    return "hello, world!\n"

@app.route("/user/<name>")
def user_service(name):
    r = requests.get(f"http://go.internal?name={name}")
    return r.text, r.status_code

import pebl
pebl.service(app, "hey.pebl.rocks")  # put your own domain here

So here we are relying on the popular requests library in order to send /user requests to the internal Go service at go.internal:80. Once again the power of pebl is this ability to incorporate other Python libraries at ease without any external configuration. But before we can run this locally, we need to ensure that the requests library is available for use within the container. So update the Dockerfile:

FROM peblcloud/python
COPY . .
RUN pip install flask requests
ENTRYPOINT python -u main.py

Go ahead and try running this! You'll be able to query the external endpoint at the /user path to get access to the internal service:

$ curl localhost:32807/user/alice
{"email":"alice@example.org","name":"alice"}
$ curl localhost:32807/user/bob
{"email":"bob@example.org","name":"bob"}
$

Deploying

Go ahead and deploy these changes! First deploy the internal Go service:

$ cd ~/work/scratch/go
$ pebl deploy

Then invoke pebl deploy once again in the python subproject:

$ cd ~/work/scratch/hello
$ pebl deploy

Pebl automatically implements zero downtime updates to services. This means that there won't be any dropped requests during this transition. Once the deploy is complete you can try sending the same requests to the cloud runtime:

$ curl https://hey.pebl.rocks/user/alice
{}
$ curl https://hey.pebl.rocks/user/bob
{}

And as expected you should be seeing empty results as our cloud runtime doesn't have the data we filled for our local runtime.

Redis Management

In order to access redis that's in the cloud runtime, we can utilize pebl CLI's pebl tunnel command:

$ pebl tunnel
 :: tunnel established at: 127.0.0.1:59039
 :: use redis-cli to connect:
 :: redis-cli -p 59039

This tunnel proxies local traffic to your redis instances in the cloud. You can use the redis-cli with the -p argument to connect to the proxied port and set the same data we used locally:

$ redis-cli -p 59039
127.0.0.1:59039> hset alice name alice email alice@example.org
(integer) 2
127.0.0.1:59039> hset bob name bob email bob@example.org
(integer) 2

Now your requests to the cloud will return the same results as before:

$ curl https://hey.pebl.rocks/user/alice
{"email":"alice@example.org","name":"alice"}
$ curl https://hey.pebl.rocks/user/bob
{"email":"bob@example.org","name":"bob"}

Next Steps

So we've explored three capabilities of pebl — services, internal services, and redis. You can explore the rest of pebl's offerings by digging into the SDK reference.

You can also follow our many guides on how to accomplish common cloud tasks.