Streamlining Infrastructure: Installing Backstage Developer Portal and Managing Azure Resources with Terraform (Part-1)

Introduction
In today's fast-paced software development landscape, managing complex projects and coordinating diverse teams can be a daunting task. Developers often find themselves juggling multiple tools and platforms, leading to inefficiencies, delays, and frustration. Enter Backstage – an open-source platform built by Spotify to streamline the developer experience and supercharge productivity. In this blog post, we'll explore the transformative capabilities of Backstage and how it can revolutionize your development workflow.

Understanding Backstage
Backstage is an open-source platform developed by Spotify to address the challenges of managing the software development lifecycle at scale. It serves as a unified developer portal, providing a single point of entry for accessing various tools, services, and information needed by engineering teams.

Some key features and components of the Backstage Developer Portal:

Catalog: Backstage features a centralized catalog that acts as a directory of all the services, components, and resources available within an organization. This catalog provides a holistic view of the entire software ecosystem, making it easier for developers to discover and consume resources.

Plugins: Backstage is highly extensible, allowing organizations to customize and extend its functionality through plugins. Plugins can be developed to integrate with existing tools, automate workflows, and provide additional features tailored to specific use cases.

Integration: Backstage integrates seamlessly with a wide range of third-party tools and services commonly used in the software development process. This includes version control systems (e.g., GitHub, GitLab), CI/CD pipelines (e.g., Jenkins, CircleCI), cloud providers (e.g., AWS, Azure, Google Cloud), and more.

Developer Experience: Backstage prioritizes the developer experience, providing a user-friendly interface that promotes collaboration, transparency, and productivity. Developers can easily navigate the portal, access relevant information, and perform common tasks without context switching between multiple tools.

Self-Service: Backstage empowers developers with self-service capabilities, allowing them to provision resources, manage configurations, and deploy applications independently. This reduces reliance on centralized IT teams and accelerates the pace of development.

Customization: Organizations can customize the look and feel of the Backstage portal to align with their branding and preferences. Custom themes, layouts, and styling options enable organizations to create a tailored experience that reflects their unique culture and identity.

Getting Started with Backstage

1) Install nvm ( Version should be above 18)

root@devmaster:~# curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash

root@devmaster:~# nvm install 20

2) Install yarn

root@devmaster:~# npm install -g yarn@1

3) Create Backstage App

root@devmaster:~# npx @backstage/create-app@latest

4) Move to the App Dir

root@devmaster:~# cd chicco
root@devmaster:~# vi app-config.yaml (update your system Ip)

5) Start the Developer Portal

root@devmaster:~/chicco# yarn dev
yarn run v1.22.21
$ concurrently "yarn start" "yarn start-backend"
$ yarn workspace backend start
$ yarn workspace app start
$ backstage-cli package start
$ backstage-cli package start
[0] Loaded config from app-config.yaml
[0] <i> [webpack-dev-server] Project is running at:
[0] <i> [webpack-dev-server] On Your Network (IPv4): http://192.168.0.117:3000/
[0] <i> [webpack-dev-server] Content not from webpack is served from '/root/chicco/packages/app/public' directory
[0] <i> [webpack-dev-server] 404s will fallback to '/index.html'
[0] <i> [webpack-dev-middleware] wait until bundle finished: /
[1] Loaded config from app-config.yaml
[1] 2024-02-07T12:40:51.076Z backstage info Found 2 new secrets in config that will be redacted
[1] 2024-02-07T12:40:51.088Z backstage info Created UrlReader predicateMux{readers=azure{host=dev.azure.com,authed=false},bitbucketCloud{host=bitbucket.org,authed=false},github{host=github.com,authed=false},gitlab{host=gitlab.com,authed=false},awsS3{host=amazonaws.com,authed=false},fetch{}
[1] 2024-02-07T12:40:51.114Z catalog info Performing database migration type=plugin
[1] 2024-02-07T12:40:51.679Z auth info Configuring "database" as KeyStore provider type=plugin
[1] 2024-02-07T12:40:51.698Z techdocs info Creating Local publisher for TechDocs type=plugin
[1] 2024-02-07T12:40:51.703Z search info Added DefaultCatalogCollatorFactory collator factory for type software-catalog type=plugin
[1] 2024-02-07T12:40:51.704Z search info Added DefaultTechDocsCollatorFactory collator factory for type techdocs type=plugin
[1] 2024-02-07T12:40:51.705Z search info Starting all scheduled search tasks. type=plugin
[1] 2024-02-07T12:40:51.833Z backstage info Listening on :7007
[1] 2024-02-07T12:40:51.862Z backstage info Task worker starting: search_index_software_catalog, {"version":2,"cadence":"PT10M","initialDelayDuration":"PT3S","timeoutAfterDuration":"PT15M"} type=taskManager task=search_index_software_catalog
[1] 2024-02-07T12:40:51.864Z backstage info Task worker starting: search_index_techdocs, {"version":2,"cadence":"PT10M","initialDelayDuration":"PT3S","timeoutAfterDuration":"PT15M"} type=taskManager task=search_index_techdocs
[1] 2024-02-07T12:40:54.867Z search info Collating documents for software-catalog via DefaultCatalogCollatorFactory type=plugin documentType=software-catalog
[1] 2024-02-07T12:40:54.900Z search info Collating documents for techdocs via DefaultTechDocsCollatorFactory type=plugin documentType=techdocs
[1] 2024-02-07T12:40:54.938Z backstage info ::1 - - [07/Feb/2024:12:40:54 +0000] "GET /api/catalog/entities?limit=500&filter=metadata.annotations.backstage.io%2Ftechdocs-ref&offset=0 HTTP/1.1" 200 2 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)" type=incomingRequest
[1] 2024-02-07T12:40:54.943Z search warn Index for techdocs was not created: indexer received 0 documents type=plugin documentType=techdocs
[1] 2024-02-07T12:40:54.944Z search info Collating documents for techdocs succeeded type=plugin documentType=techdocs
[1] 2024-02-07T12:40:54.947Z backstage info ::1 - - [07/Feb/2024:12:40:54 +0000] "GET /api/catalog/entities?limit=500&offset=0 HTTP/1.1" 200 - "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)" type=incomingRequest
[1] 2024-02-07T12:40:54.957Z search info Collating documents for software-catalog succeeded type=plugin documentType=software-catalog
[0] webpack compiled successfully
[1] 2024-02-07T12:41:07.203Z backstage info ::ffff:192.168.0.102 - - [07/Feb/2024:12:41:07 +0000] "GET /api/catalog/entity-facets?facet=relations.ownedBy HTTP/1.1" 200 118 "http://192.168.0.117:3000/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36" type=incomingRequest
[1] 2024-02-07T12:41:07.303Z backstage info ::ffff:192.168.0.102 - - [07/Feb/2024:12:41:07 +0000] "GET /api/catalog/entity-facets?facet=kind HTTP/1.1" 200 225 "http://192.168.0.117:3000/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36" type=incomingRequest
[1] 2024-02-07T12:41:07.306Z backstage info ::ffff:192.168.0.102 - - [07/Feb/2024:12:41:07 +0000] "GET /api/catalog/entity-facets?facet=metadata.tags HTTP/1.1" 200 31 "http://192.168.0.117:3000/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36" type=incomingRequest
[1] 2024-02-07T12:41:07.310Z backstage info ::ffff:192.168.0.102 - - [07/Feb/2024:12:41:07 +0000] "GET /api/catalog/entity-facets?facet=metadata.namespace HTTP/1.1" 200 65 "http://192.168.0.117:3000/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36" type=incomingRequest

6) Open The Browser and access the Portal

Getting Started, configuring Backstage

Backstage to use a PostgreSQL database

root@devmaster:~/chicco# apt-get install postgresql

Test if your database is working

root@devmaster:~/chicco# sudo -u postgres psql
could not change directory to "/root/chicco": Permission denied
psql (15.5 (Ubuntu 15.5-0ubuntu0.23.10.1))
Type "help" for help.

postgres=#

For testing going to use the existing postgres user. The next step is to set the password for this user

postgres=# ALTER USER postgres PASSWORD 'secret';
ALTER ROLE

Stop Backstage, and go to the root directory of your freshly installed Backstage App. Use the following commands to start the PostgreSQL client installation

root@devmaster:~/chicco# yarn --cwd packages/backend add pg

Add Postgres configuration in app-config.yaml

root@devmaster:~/chicco# vim app-config.yaml
client: pg
connection:
      host: ${POSTGRES_HOST}
      port: ${POSTGRES_PORT}
      user: "postgres"
      password: "secret"

Restart the Dev portal

root@devmaster:~/chicco# yarn dev
yarn run v1.22.21
$ concurrently "yarn start" "yarn start-backend"
$ yarn workspace app start
$ yarn workspace backend start
$ backstage-cli package start
$ backstage-cli package start
[0] Loaded config from app-config.yaml
[0] <i> [webpack-dev-server] Project is running at:
[0] <i> [webpack-dev-server] On Your Network (IPv4): http://192.168.0.117:3000/
[0] <i> [webpack-dev-server] Content not from webpack is served from '/root/chicco/packages/app/public' directory
[0] <i> [webpack-dev-server] 404s will fallback to '/index.html'
[0] <i> [webpack-dev-middleware] wait until bundle finished: /
[1] Loaded config from app-config.yaml
[1] 2024-02-07T13:31:32.561Z backstage info Found 2 new secrets in config that will be redacted
[1] 2024-02-07T13:31:32.574Z backstage info Created UrlReader predicateMux{readers=azure{host=dev.azure.com,authed=false},bitbucketCloud{host=bitbucket.org,authed=false},github{host=github.com,authed=false},gitlab{host=gitlab.com,authed=false},awsS3{host=amazonaws.com,authed=false},fetch{}
[1] 2024-02-07T13:31:32.781Z catalog info Performing database migration type=plugin
[1] 2024-02-07T13:31:33.834Z auth info Configuring "database" as KeyStore provider type=plugin
[1] 2024-02-07T13:31:33.998Z techdocs info Creating Local publisher for TechDocs type=plugin

Setting up authentication with github
Edit app-config.yaml and update the github PAT Token.

'''
root@devmaster:~/chicco# vim app-config.yaml
'''

Restart the Dev portal

root@devmaster:~/chicco# yarn dev
yarn run v1.22.21
$ concurrently "yarn start" "yarn start-backend"
$ yarn workspace app start
$ yarn workspace backend start
$ backstage-cli package start
$ backstage-cli package start
[0] Loaded config from app-config.yaml
[0] <i> [webpack-dev-server] Project is running at:
[0] <i> [webpack-dev-server] On Your Network (IPv4): http://192.168.0.117:9999/
[0] <i> [webpack-dev-server] Content not from webpack is served from '/root/chicco/packages/app/public' directory
[0] <i> [webpack-dev-server] 404s will fallback to '/index.html'
[0] <i> [webpack-dev-middleware] wait until bundle finished: /

In Part-2 We will use Backstage Dev portal to Deploy Resource in Azure using Terraform.

Happy Reading...............