You don't learn to walk by following rules. You learn by doing, and by falling over. - Richard Branson
The Inspiration API is the start of my journey down the path of trying to improve my Scala skills. Having coded very little in this language I decided to attempt to build a basic API using the language alongside the Play Framework. Rather than take the textbook approach I often find at times you just need to get your hands dirty and learn from your mistakes. I mean... how hard can it be?
Up to recently my primary coding focus has been on Frontend or more JavaScript heavy services. For work and personal purposes I wanted to expand my skillset to encompass a more broad scope of programming practices, and so I've found my way to Scala. I've studied the OOP basics of Java in college and taken some Scala training on the job but being rusty with using the JVM and blind to the more complex usages of the functional programming paradigm I've been ill prepared to adopt it.
The Inspiration API
The Codebase: Available Here!
I guess the name of the API is self evident -> I needed some sort of project to keep me inspired and interested in learning a new language. As many of you are aware it can be difficult at the best of times to find energy coding after a full days work!
So I was a little lazy at this point... I basically googled 'inspirational quotes' and grabbed the first 10 I came across that didn't sound entirely cheesy or ridiculous and decided to use them for the basis of the API's data.
Note that I say data here as my first attempt at making the API was focused on setting up a basic GET endpoint that return a response. Rather than relying on a DB I started with a basic array of JSON objects, I used a similar trick in another project to populate a dropdown menu on a static site instead of relying on a DB. Start small and work my way up I figured...
I set up a bunch of github issues which linked to an overall RESTful API milestone, essentially it was more or less 1 issue per CRUD feature:
When starting out I cloned a Play/Scala seed from the play website -> in retrospect this featured quite a lot of what I'll playfully call Bash bloatware that wasn't necessary for just building the project locally with SBT - however it did the job!
The main considerations for using play was that it should be as simple as modifying your routes configuration to determine your GET, POST, PUT, DELETE endpoints to point to a specific controller.
Handling JSON in Scala
Immediately it became evident that there is a big difference in the level of Google Fu required between searching for how to do something in JS versus how to do something in Scala. And to be fair I can accept that JSON is native to JavaScript... We can all agree that Stack Overflow development isn't an optimal way of programming however it can prove a useful tool when starting out and wanting to build something with a tool you have had very limited use with!
So the basic logic of my JSON solution (and prior to this an array implementation) featured a Play controller which looked a little like this (forgive the placeholder variable names... I never do this in production code... honest!):
Routes:
# Routes
# This file defines all application routes (Higher priority routes first)
# https://www.playframework.com/documentation/latest/ScalaRouting
# ~~~~
# An example controller showing a sample home page
GET / controllers.HomeController.index
# Map static resources from the /public folder to the /assets URL path
GET /assets/*file controllers.Assets.versioned(path="/public", file: Asset)
# Inspiration CRUD endpoints
GET /inspiration controllers.InspirationController.index
Inspiration Controller:
package controllers
import javax.inject._
import play.api._
import play.api.mvc._
import play.api.libs.json._
import scala.collection.mutable.ArrayBuffer
/**
* This controller creates an `Action` to handle HTTP requests to the
* application's home page.
*/
@Singleton
class InspirationController @Inject()(cc: ControllerComponents) extends AbstractController(cc) {
def index() = Action { implicit request: Request[AnyContent] =>
Ok(generateQuote(y, scala.util.Random.nextInt(10)))
}
// json method of generating quotes
// (y(scala.util.Random.nextInt(10))\"quote").get
var y: JsValue = Json.arr(
Json.obj("quote" -> "Make your life a masterpiece, imagine no limitations on what you can be, have or do.", "author" -> "Brian Tracy"),
Json.obj("quote" -> "We may encounter many defeats but we must not be defeated.", "author" -> "Maya Angelou"),
Json.obj("quote" -> "I am not a product of my circumstances. I am a product of my decisions.", "author" -> "Stephen Covey"),
Json.obj("quote" -> "We must let go of the life we have planned, so as to accept the one that is waiting for us.", "author" -> "Joseph Campbell"),
Json.obj("quote" -> "Believe you can and you're halfway there.", "author" -> "Theodore Roosevelt"),
Json.obj("quote" -> "We know what we are, but know not what we may be.", "author" -> "William Shakespeare"),
Json.obj("quote" -> "We can't help everyone, but everyone can help someone.", "author" -> "Ronald Reagan"),
Json.obj("quote" -> "When you have a dream, you've got to grab it an never let go.", "author" -> "Carol Burnett"),
Json.obj("quote" -> "Your present circumstances don't determine where you can go; they merely determine where you start.", "author" -> "Nido Quebein"),
Json.obj("quote" -> "Thinking: the talking of the soul with itself.", "author" -> "Plato")
)
// Function that returns a random string include quote & author
def generateQuote( quotes:JsValue, random:Int) : String = {
var quote:JsValue = (quotes(random)\"quote").get
var author:JsValue = (quotes(random)\"author").get
return (author.as[String] + ": " + quote.as[String])
}
// array method of generating quotes
// quotes(scala.util.Random.nextInt(10))
// var quotes = ArrayBuffer[String]()
// quotes += "Make your life a masterpiece, imagine no limitations on what you can be, have or do. - Brian Tracy"
// quotes += "We may encounter many defeats but we must not be defeated. - Maya Angelou"
// quotes += "I am not a product of my circumstances. I am a product of my decisions. - Stephen Covey"
// quotes += "We must let go of the life we have planned, so as to accept the one that is waiting for us. - Joseph Campbell"
// quotes += "Believe you can and you're halfway there. - Theodore Roosevelt"
// quotes += "We know what we are, but know not what we may be. - William Shakespeare"
// quotes += "We can't help everyone, but everyone can help someone. - Ronald Reagan"
// quotes += "When you have a dream, you've got to grab it an never let go. - Carol Burnett"
// quotes += "Your present circumstances don't determine where you can go; they merely determine where you start. - Nido Quebein"
// quotes += "Thinking: the talking of the soul with itself. - Plato"
}
Wahey - just like that we have our GET /inspiration up and running!
The Database Hookup
This was the MOST painful part of the project for several reasons.
Did you know that you can't run Docker natively on Windows 10 Home edition? You have to use Docker Toolbox combined with vagrant or some virtualbox setup?
I wanted to use Docker to run a simple postgres database that my service would connect to. I had used Docker before and figured this would be the easiest solution! This shouldn't have caused me so many issues (and time) but it did. I usually code on a MacBook in work but on a Windows PC at home for personal projects - I find its a good way to stay ambidextrous across platforms. In the end I called it quits and settled with using my trusty MacBook (which worked almost instantly once I cloned the Dockerfile locally).
To try and keep things self contained I included a basic dbsetup.sql file in the repository that users could load into their Docker container using a simple command.
dbsetup.sql (comment below if you have any better default quotes I should include!):
\c inspiration_db
CREATE TABLE quotations(
index serial,
author varchar(255) NOT NULL,
quote varchar(1000) NOT NULL
);
INSERT INTO quotations (author, quote) VALUES ('Brian Tracy', 'Make your life a masterpiece, imagine no limitations on what you can be, have or do.');
INSERT INTO quotations (author, quote) VALUES ('Maya Angelou', 'We may encounter many defeats but we must not be defeated.');
INSERT INTO quotations (author, quote) VALUES ('Stephen Covey', 'I am not a product of my circumstances. I am a product of my decisions.');
INSERT INTO quotations (author, quote) VALUES ('Joseph Campbell', 'We must let go of the life we have planned, so as to accept the one that is waiting for us.');
INSERT INTO quotations (author, quote) VALUES ('Theodore Roosevelt', 'Believe you can and you''re halfway there.');
INSERT INTO quotations (author, quote) VALUES ('William Shakespeare', 'We know what we are, but know not what we may be.');
INSERT INTO quotations (author, quote) VALUES ('Ronald Reagan', 'We can''t help everyone, but everyone can help someone.');
INSERT INTO quotations (author, quote) VALUES ('Carol Burnett', 'When you have a dream, you''ve got to grab it an never let go.');
INSERT INTO quotations (author, quote) VALUES ('Nido Quebein', 'Your present circumstances don''t determine where you can go; they merely determine where you start.');
INSERT INTO quotations (author, quote) VALUES ('Plato', 'Thinking: the talking of the soul with itself.');
Docker Commands for setting up the Docker DB & Inserting above rows:
docker-compose up -d
psql -h localhost -U user inspiration_db -f dbsetup.sql
At this point I had to start the Google searching: 'how to connect to a postgres db with scala' and there are a bunch of different libraries and results that showed up such as jdbc, postgres-scala, doobie and many more... It was a little overwhelming and difficult to just get a few lines of documentation for a super simple implementation. In the end I went with a library called Slick.
After settling on Slick I setup a basic Class to represent the quote entries for the DB. I had to do some tinkering before figuring out how the heck to handle serial (auto incrementing) postgres values but thats a story best unmentioned! Just enjoy the completed functioning code!
Class matching quotes structure form the postgres DB:
// Matches schema of the docker-compose psql DB quotations table
class Quotes(tag: Tag) extends Table[(Int, String, String)](tag, "quotations") {
def index = column[Int]("index")
def author = column[String]("author")
def quote = column[String]("quote")
def * = (index, author, quote)
}
At this point the rest of the 'hard' work came down to figuring out how to translate postgresSQL to this Slick style syntax. Heres a rough breakdown of how the various endpoints worked:
GET /inspiration
import scala.slick.driver.PostgresDriver.simple._
def index() = Action { implicit request: Request[AnyContent] =>
Ok(generateQuote(scala.util.Random.nextInt(10)))
}
val connectionUrl = "jdbc:postgresql://localhost:5432/inspiration_db?user=user"
def generateQuote(random:Int): String = {
var output = ""
// connecting to postgres db for accessing data
Database.forURL(connectionUrl, driver = "org.postgresql.Driver") withSession {
implicit session =>
val quotes = TableQuery[Quotes]
// SELECT * FROM quotations WHERE id=randomInt
quotes.filter(_.index === random+1).list foreach { row =>
output = row._2 + ": " + row._3
}
}
output
}
POST /inspiration
import scala.slick.driver.PostgresDriver.simple._
def add() = Action { request =>
val body: AnyContent = request.body
val json: Option[JsValue] = body.asJson
val author = json.get("author").toString.stripPrefix("\"").stripSuffix("\"").trim
val quote = json.get("quote").toString.stripPrefix("\"").stripSuffix("\"").trim
addQuote(author, quote)
Ok("Successfully updated quotations DB")
}
val connectionUrl = "jdbc:postgresql://localhost:5432/inspiration_db?user=user"
def addQuote(author:String, quote:String): Unit ={
var index = 0
Database.forURL(connectionUrl, driver = "org.postgresql.Driver") withSession {
implicit session =>
val quotes = TableQuery[Quotes]
// getting id of last element in table
quotes.sortBy(_.index.desc).take(1).list foreach { row =>
index = row._1 + 1
}
quotes += (index, author, quote)
}
}
PUT /inspiration
import scala.slick.driver.PostgresDriver.simple._
def replace() = Action { request =>
val body: AnyContent = request.body
val json: Option[JsValue] = body.asJson
val index: Int = json.get("index").toString.toInt
val author = json.get("author").toString.stripPrefix("\"").stripSuffix("\"").trim
val quote = json.get("quote").toString.stripPrefix("\"").stripSuffix("\"").trim
updateQuote(index, author, quote)
Ok("Successfully updated quotations DB")
}
val connectionUrl = "jdbc:postgresql://localhost:5432/inspiration_db?user=user"
def updateQuote(index:Int, author:String, quote:String) = {
Database.forURL(connectionUrl, driver = "org.postgresql.Driver") withSession {
implicit session =>
val quotes = TableQuery[Quotes]
quotes.filter(_.index === index).update(index, author, quote)
}
}
DELETE /inspiration/:index
import scala.slick.driver.PostgresDriver.simple._
def delete(index: Int) = Action { request =>
deleteQuote(index)
Ok(s"Successfully deleted entry $index")
}
val connectionUrl = "jdbc:postgresql://localhost:5432/inspiration_db?user=user"
def deleteQuote(index:Int): Unit = {
Database.forURL(connectionUrl, driver = "org.postgresql.Driver") withSession {
implicit session =>
val quotes = TableQuery[Quotes]
quotes.filter(_.index === index).delete
}
}
The routes:
GET /inspiration controllers.InspirationController.index
POST /inspiration controllers.InspirationController.add
PUT /inspiration controllers.InspirationController.replace
DELETE /inspiration/:index controllers.InspirationController.delete(index: Int)
And there you have it - a basic RESTful API in Scala & Play!
The Codebase: Available Here!
Lessons Learned
The project was a nice way to go off book and try learn by developing something directly. There were some minor pain points I definitely learned from including:
- How to connect to a Postgres DB with scala
- How to handle Array & JSON data structures with scala
- How to handle routes with the Play Framework
- How to setup a basic frontend for paths in the Play Framework
- Handling Docker on Windows... this one still irks me for the time wasted!
Improvements
Theres a bunch of additions I plan to make to this project over time including:
- Adjusting the GET endpoint to return items in the DB with an index higher than 10 -> currently this is hardcoded but it should be easy to swap it out for a DB count amount
- Setup an API swagger definition for the generated API -> this would just be some useful additional experience
- Deploy the API somewhere -> Heroku is the lead favourite for now... Once deployed implementing some endpoint tracking & analytics would be interesting -> potentially some oAuth as well but that's generally a headache to setup.
- Develop basic SDKs from the generated Swagger using a service like Swagger Codegen
- Add Tests
- Modify OK section on routes to return correct response i.e. 200, 201, 202 etc.
As always if you have any feedback, suggestions or thoughts feel free to share below.
'Till Next Time!
Top comments (3)
Hey Daniel thanks for sharing. I would like to suggest one thing in your improvement list. It's good practice to have the responsibility well divided (looks for S.O.L.I.D.). So, you could separate your database repository logic from your controller. What do you think?
Many thanks for the feedback Samuel. There is definitely an opportunity to improve the logic here, the reasoning I had it all contained together was purely convenience as my main goal was to try get the API working initially. I'll definitely look into improving upon the logic there!
In the organization I work for we use Symfony and I decided to learn a new framework (and a language)
I started working on a personal project/idea and I got into Scala and I like it. This article is definitely and must read for me.
Thanks for sharing.