Sm0ke

Posted on Aug 28, 2023 • Originally published at docs.app-generator.dev

Rocket, a free Django Generator - Insights for Developers

#django #generator

Hello Coders!

This article presents a few challenges I faced during the coding phase for Rocket Generator, a free service that generates Django starters (MIT License) using a simple & intuitive UI/UX. What's in the box and also a few words about the concept:

✅ Service can be used without an account
- ❔ Coding Dilemma: how to provide persistence without an account?
✅ In-House Generator
- ❔ Coding Dilemma: Why not using Cookiecutter or Copier
✅ Full Codebase Control
- ❔ Coding Dilemma: How to inject code in different parts of the project .. and have something that compiles
✅ Deployment
- ❔ Coding Dilemma: How to make the projects deployable from the beginning
✅ Integrated API
- ❔ Coding Dilemma: Users need an API at some point. How to offer this out-of-the-box
✅ Extended User Model
- ❔ Coding Dilemma: AppSeed users asked for this many times in support
✅ Slim-fit Codebase
- ❔ Coding Dilemma: How to provide small codebases
✅ Documentation CORE
- ❔ Coding Dilemma: What to use? Free or Paid services?

With all the above in mind, the project was started ~ January this year with a few SPECS regarding the new features and the entire knowledge used to code AppSeed in mind.

NOTE: AppSeed is the first generator I've built that uses a 1st generation pattern with some different solutions compared to the Rocker Generator.

AppSeed in cold numbers:

service can be used without account (we need the same for the new product)
17k+ generated products in ~= 18months
Sources Saved on GitHub using three rotating accounts
- This was needed to hack the GitHub API limiters

With this summary, let's get back to the real work.

✅ Persistence without an account

Based on our experience with AppSeed, we saw that users are comfortable using services where the sign-in is optional and we wanted to offer the same on Rocket Generator

In order to provide minimal persistence, the generated projects are saved in the database using a numeric representation of the IP that accesses the service. Here is the Python code for this:

import netaddr

def ipToINT(aIP):
    return int(netaddr.IPAddress(aIP))

# request comes from the Framework
def get_client_ip(request):
    x_forwarded_for = request.META.get('HTTP_X_FORWARDED_FOR')
    if x_forwarded_for:
        ip = x_forwarded_for.split(',')[-1].strip()
    else:
        ip = request.META.get('REMOTE_ADDR')

    # Convert to INT    
    return ipToINT( ip )

✅ In-House App Generator

This decision was probably the most important one. Using an existing tool with proven traction helps in the 1st phase of the project when you have basically nothing.

Here are the aspects that forced me to build something from scratch:

The lack of support that usually comes with the open-source projects
If the future evolution hits the wall in terms of features, you are forced to rewrite the entire codebase
Cookiecutter and Copier are basically string replaces and managing a contextual update on the codebase was impossible

By taking into account the above limitations I decided to write my own codebase parser that knows the following:

update Django Configuration that is quite non-linear (simple variables, lists of variables, dicts of lists ..) etc
Update the project routing
Update dependencies
Update ENV files
Edit/inject code using different files: Python, TXT, Yaml .. etc

The product is not open-source (I'm still thinking of making it public) but this parser was the show-stopper for the projects for about 3mo. Around April.2023 the parser & codebase injector were stable and usable in production. This means the generated codebase was compilable and deployable.

✅ Slim-Fit Codebase

The AppSeed Generator provides all the features inside the starter and this makes the codebase FAT, sometimes with 500 files or more (up to 1k).
From the user perspective, this might not be a problem but during the GitHub upload, the process sometimes gets canceled due to the overuse of GitHub API.

Rocket Generator uses a different pattern where all the features are libraries: UI Kits, and API Generator.
The process simply executes the following steps:

Collects the user input
Injects the configuration for the UI
Generates the models using the definitions
Activate the API for selected models

With this new pattern, the most complex starter shouldn't have more than 100 files because the actual build happens on the user side during compilation.

✅ UI (External Libraries)

All the UI Kits integrated with the generator can be used by anyone outside the service. Here are the links:

👉 Django Admin Soft - dashboard design
👉 Tabler - dashboard design
👉 Berry Bootstrap 5 - dashboard design
👉 Corporate UI - dashboard design
👉 Material KIT - UI Kit Design
👉 Pixel Bootstrap 5 - UI Kit Design

When the user selects one kit/dashboard from the above list, the generator executes the same steps on the sample codebase as the ones listed on the README.

For instance, if the user selects Soft Dashboard, here are the steps executed by the generator:

Add django-admin-soft-dashboard as a dependency in requirements.txt
Update configuration->INSTALLED_APPS section:
- Add admin_soft.apps.AdminSoftDashboardConfig
Update project routing
- Add path('', include('admin_soft.urls')),

The above steps are executed by a custom parser that knows how to safely update the codebase, apply changes and keep the project compilable once the changes are operated.

✅ API Generator

For each defined model, the user has the option to activate an API node with full CRUD access:

GET requests are public (get All items, get item by ID)
Mutating Requests (CREATE, UPDATE, DELETE) requires authentication

This feature is provided via Django API Generator, an open-source library built on top of DRF.

Regarding the generator processing, here are the steps performed for models integration:

Add django-api-generator as a dependency in requirements.txt
Update configuration->INSTALLED_APPS section:

INSTALLED_APPS = [
    'django_api_gen',            # Django API GENERATOR  # <-- NEW
    'rest_framework',            # Include DRF           # <-- NEW 
    'rest_framework.authtoken',  # Include DRF Auth      # <-- NEW   
]

Create new sections in the project settings

API_GENERATOR = {
    'MODEL_API_PATH'  : "MODEL_DEFINITION_PATH",
}

REST_FRAMEWORK = {
    'DEFAULT_AUTHENTICATION_CLASSES': [
        'rest_framework.authentication.SessionAuthentication',
        'rest_framework.authentication.TokenAuthentication',
    ],
}

The most challenging part is to have a controlled injection for each model because the user can activate the API for a single model, all models, or none.

✅ Extended User Model

This feature is something that most of the Django Apps require at some point.
To cover this point, the generator empowers the user to define a custom user model using a few defaults:

name
phone
bio

Note: The API generator option is not activated in the UI.

✅ Models Editor

The users can edit and design their database via a simple UI. This is unsupported on AppSeed.

✅ Deployment `Ready`

All starters generated with Docker support can be deployed via DeployPRO service using a simple flow:

Authenticate/register into DeployPRO
Connect your preferred cloud provider
- AWS, Azure or DigitalOcean
Create a new VPS Server
- this operation takes ~5min
Deploy the GitHub Repositor (needs ownership)

In less than 10 minutes, if the project has an usable Dockerfile, the project goes LIVE with an active Ci/CD flow.

✅ Documentation CORE

Since I decided to leave my corporate job in 2019 and code my own products, the DOCS part was always a dilemma.

I even wrote an undercover survey here on Dev to collect some suggestions but without much luck.

After a long path from MkDocs, GitBook, and Docsify I finally landed on Docusaurus which seems to cover (for now) what my users requested in terms of documentation.