TL;DR
AI-assisted coding tools are on the rise. In this article, we take a deep dive into two tools that use similar techniques, but are intended for different outcomes.
Smol AIâs âSmol-Developerâ gained a lot of notoriety very quickly by being one of the first such tools on the scene. It is a simple set of python scripts that allow a user to build prototype apps using natural language in an iterative approach.
Waspâs âGPT Web App Generatorâ is more of a newcomer and focuses on building more complex full-stack React + NodeJS web app prototypes through a simple prompt and fancy UI.
When comparing the two, Smol-Developerâs strength is its versatility. If you want to spend time tinkering and tweaking, you can do a lot to your own prompting, and even the code, in order to get decent results on a broad range of apps.
On the other hand, Wasp AI shines by being specific. Because itâs only built for generating full-stack React/NodeJS/Prisma/Tailwind codebases, it does the tweaking and advanced prompting for you, and thus it performs much better in generating higher quality content with less effort for a specific use case.
Will either of these tools completely replace Junior Developers in their current form? Of course not. But they do allow for rapid prototyping and testing of novel ideas.
Read on to learn more about how they work, which tool is right for the job at hand, and how you can use them in your current workflow.
Intro
The age of AI-assisted coding tools is fully upon us. GitHubâs Copilot might be the go-to professional solution, but since its release numerous open-source solutions have popped up.
Most of these newer solutions tend towards functioning as an AI Agent, going beyond just suggesting the next logical pieces of code within your current file, they aim to create simple prototypes of entire apps. Some are focused more on scaffolding entire app prototypes from an initial prompt, while others function as interactive assistants, helping you modify and improve existing codebases.
Either way, theyâre often being described as âAI Junior Developersâ, because they can take a product requirement (i.e. âpromptâ) and build a pretty good â but far from perfect â first iteration, saving developers a lot of time.
This article is going to focus on two tools that aim to build somewhat complex working prototypes from a single prompt: Smol AI and Wasp AI. Weâll test them out by running the same prompts through each and seeing what we get.
By the end of it, youâll have a pretty good understanding of how they work, their advantages and disadvantages, and what kind of tasks theyâre best suited for.
Before We Begin
Wasp = } is the only open-source, completely serverful fullstack React/Node framework with a built-in compiler and AI-assisted features that lets you build your app super quickly.
Weâre working hard to help you build performant web apps as easily as possible â including creating content like this, which is released weekly!
We would be super grateful if you could help us out by starring our repo on GitHub: https://www.github.com/wasp-lang/wasp đ
âŠeven Ron would star Wasp on GitHub đ€©
âïž Thanks For Your Support đ
The Tools
Smol-Developer
Smol AI (described as a platform for âmodel distillation and AI developer agentsâ) actually has a few open-source tools on offer, but Smol-Developer is the one weâll be taking a look at. It was initially released by Swyx on May 11th and already has over 10k GitHub stars!
It aims to be a generalist, prompt-based coding assistant run from the command line. The developerâs job becomes a process of iterative prompting, testing, and re-prompting in order to get the optimal output. It is not limited to any language or type of app it can create, although simple apps tend to work best.
Check out this tweet thread below to get a better understanding:
Running from the command line, Smol AI is essentially a chain of calls to the OpenAI chat completions (i.e. âChatGptâ) endpoint via a python script that:
- takes an initial user-generated prompt
- creates a plan based on internal prompts* for executing the app with:
- the structure of the entire app
- each file and its exported variables to be generated
- function names
- generates file paths based on the plan
- loops through file paths and generates code for each file based on plan and prompt
The generated output can then be evaluated by the developer and the prompt can be iterated on to account for any errors or bugs found during runtime.
Smol-Developer quickly gained notoriety by being one of the first of such tools on the scene, in addition to Swyxâs prominence within it. So if youâre curious to see whatâs being built with it, just check out some of the numerous YouTube videos on it.
One of my personal favorites is AI Jasonâs exposĂ© and commentary. He gives a concise explanation, shows you some great tips on how to use Smol-Developer effectively, and as a Product Designer/Manager he gives an interesting perspective on its benefits:
- Smol-Developer GitHub Repo: https://github.com/smol-ai/developer/
- *Curious to see what the internal system prompt looks like? Click Here
Waspâs GPT Web App Generator
In contrast to Smol-Developer, Waspâs AI tool, GPT Web App Generator, is currently an open-source web app (yes, itâs a web app that makes web apps). Since itâs release on the 12th of July, there have been over 6,500 apps generated with over 300 apps being generated each day!
Hereâs a quick 1 minute video showcasing how GPT Web App Generator works:
So to give a bit of background, Wasp is actually a full-stack web app framework built around a compiler and config file. Using this approach, Wasp simplifies the web app creation process by handling boilerplate code for you, taking the core app logic written by the developer and connecting the entire stack together from frontend to backend, and database management.
It currently works with React, NodeJS, Tanstack-Query, and Prisma, taking care of features like Auth, Routing, Cron Jobs, Fullstack Typesafety, and Caching. This allows developers to focus more on the fun stuff, like the appâs features, instead of spending time on boring configurations.
Because Wasp uses a compiler and config file to generate the app from, this makes it surprisingly well suited for guiding LLMs like ChatGPT towards creating more complex apps with it, as it essentially a plan or set of instructions for how to build the app!
Take this simple example of how youâd tell Wasp that you want username and password
authentication in your app:
// main.wasp file
app RecipeApp {
title: "My Recipes",
wasp: { version: "^0.11.0" },
auth: {
methods: { usernameAndPassword: {} },
onAuthFailedRedirectTo: "/login",
userEntity: User
}
}
entity User {=psl // Data models are defined using Prisma Schema Language.
id Int @id @default(autoincrement())
username String @unique
password String
recipes Recipe[]
psl=}
Waspâs config file is like an app outline that the compiler understands and can then use to connect and glue the app together, taking care of the boilerplate for you.
By leveraging the powers of Wasp, GPT Web App Generator works by:
- taking a simple user-generated prompt via the UI
- giving GPT a descriptive example of a Wasp app and config file via internal prompts*
- creating a plan that meets these requirements
- generating the code for each part of the app according to the plan
- checking each file for expected errors/hallucinations and fixing them
In the end, the user can download the codebase as a zipped file and run it locally. Simpler apps, such as TodoApp or MyPlants tend to work straight out of the box, while more complex apps need a bit of finessing to get working.
- Try out the GPT Web App Generator at: https://magic-app-generator.wasp-lang.dev/ or via the command line via Wasp's experimental release
- Wasp AI / Generator GitHub: https://github.com/wasp-lang/wasp/tree/wasp-ai/waspc/src/Wasp/AI
- *Curious to see what the internal system prompt looks like? Click Here
Comparison Test
Prompt 1: PONG Game
To get a sense for how each coding agent performed, I tried out two different prompts on both Smol-Developer and Waspâs GPT Web App Generator with only slight modifications to the prompts to fit the requirements of each tool.
The first prompt was the default prompt that comes hardcoded into Smol-Developerâs [main.py](http://main.py)
script:
a simple JavaScript/HTML/CSS/Canvas app that is a one player game of PONG.
The left paddle is controlled by the player, following where the mouse goes.
The right paddle is controlled by a simple AI algorithm, which slowly moves the paddle toward the ball at every frame, with some probability of error.
Make the canvas a 400 x 400 black square and center it in the app.
Make the paddles 100px long, yellow and the ball small and red.
Make sure to render the paddles and name them so they can controlled in javascript.
Implement the collision detection and scoring as well.
Every time the ball bounces off a paddle, the ball should move faster.
Note âčïž
For Waspâs GPT Web App Generator, I replaced the first line with âa simple one player game of PONGâ since Wasp will automatically generate a full-stack React/NodeJS app.
Both were able to create a functional PONG game out-of-the box, but only on the second try. The first try created decent PONG starters, but both had buggy game logic (e.g. computer opponent failed to hit ball, or ball would spin off into oblivion). I didnât change the prompts at all, but just simply ran them a second time each â and that did the trick!
For both of the generated apps, the game logic was very simple. Scores werenât recorded, and once a game ended, youâd have to refresh the page to start a new one.
Although, while Smol-Developer only created the game logic, GPT Web App Generator created the game logic as well as the logic for authentication, creating games, and updating a gameâs score, saving it all to the database (though the scoring functions werenât being utilized initially).
To be fair, this isnât really a surprise though as these features are baked into the design of Wasp and the Generator.
On the other hand, to get these same features for Smol-Developer, weâd have to elaborate on our prompt, giving it explicit instructions to implement them, and iterate on it a number of times before landing on an acceptable prototype.
This is what I attempted to test out with the second prompt.
Prompt 2: Blog App
This time, for the second app test, I used a default prompt featured on the GPT Web App Generator homepage for creating a Blog app:
A blogging platform with posts and post comments.
User owns posts and comments and they are saved in the database.
Everybody can see all posts, but only the owner can edit or delete them. Everybody can see all the comments.
App has four pages:
1. "Home" page lists all posts (their titles and authors) and is accessible by anybody.
If you click on a post, you are taken to the "View post" page.
It also has a 'New post' button, that only logged in users can see, and that takes you to the "New post" page.
2. "New post" page is accessible only by the logged in users.
It has a form for creating a new post (title, content).
3. "Edit post" page is accessible only by the post owner.
It has a form for editing the post with the id specified in the url.
4. "View post" page is accessible by anybody and it shows the details of the post with the id specified in the url: its title, author, content and comments.
It also has a form for creating a new comment, that is accessible only by the logged in users.
NoteđĄ
For the Smol-Developer prompt, I added the lines: âThe app consists of a React client and a NodeJS server. Posts are saved in an sqlite database using Prisma ORM.â
As this was a suggested prompt on the GPT Web App Generator page, letâs start with the Wasp app result first.
After downloading the generated codebase and running the app, I ran into an error Failed to resolve import "./ext-src/ViewPost.jsx" from "src/router.jsx". Does the file exist?
One quick look at the main.wasp
file revealed that the Generator gave the wrong path to the ViewPost
page, although it did get all the other Page paths correct (highlighted in yellow above).
Once that path was corrected, a working app popped up at localhost:3000. Nice!
The video above was my first time trying out the app, and as you can see, most of the functionality is there and working correctly â Authentication and Authorization, and basic CRUD operations. Pretty amazing!
There were still a couple of errors that prevented the app from being fully functional out-of-the-box, but they were easy to fix:
- Blog posts on the homepage did not have a link in order to redirect to the their specific post page â fixable by just wrapping them in
<Link to={
/post/${post.id}}>
- The client was passing the
postId
as a String instead of anInt
to thegetPost
endpoint â fixable by wrapping the argument inparseInt(postId)
to convert strings to integers
And with those simple fixes we got a fully functioning, full-stack blog app with authentication, database, and simple tailwind css styling! The best part was that all this took about ~5 minutes from start to finish. Sweet :)
Note đ§âđ»
The Generator saves all the apps it creates along with a sharable link, so if you want to check out the original generated Blog app code (before fixes) from above, click here: https://magic-app-generator.wasp-lang.dev/result/a3a76887-952b-4774-a773-42209c4bffa8
The Smol-Developer result was also very impressive, with a solid ExpressJS server and a lot of React client pages, but there were too many complicated errors that prevented me from getting the app started, including but not limited to:
- No build tools or configuration files
- The server was importing database models that didnât exist
- The server was importing but not utilizing Prisma as the ORM to communicate with the DB
- Client had Auth logic, but was not utilizing it to protect pages/routes
Because there were too many fundamental issues with the app, I went ahead and added some more lines to the bottom of the prompt:
âŠ
Scaffold the app to be able to use Vite as the client's build tool. Include a package.json file >with the dependencies and scripts for running the client and server.
This second attempt produced some of the changes I was looking for, like package.json files and Vite config files to bootstrap the React app, but it still failed to include:
- An index.html file
- Package.json files with the correct dependencies being imported from within the client and server
- A
prisma.schema
file - A css file (although it did include
classNames
in the jsx code)
On the other hand, the server code, albeit much sparser this time, did at least import and use Prisma correctly.
So I went ahead for a third attempt and modified and added the following lines to the bottom of the prompt:
âŠ
Scaffold the app to be able to use Vite as the client's build tool.
Make sure to include the following:
- package.json files for both the server and client. Make sure that these files include the >dependencies being imported in the respective apps.
- an index.html file in the client's public folder, so that Vite can build the app.
- a
prisma.schema
file with the models and their fields. Make sure these are the same models >being used app-wide.- a css file with styles that match the
className
s used in the app.
With these additions to the prompt, the third iteration of the app did in fact include them! Well, most of them, but unfortunately not all of them. Now I was getting the css and package.json files, but no vite config file was created this time, even though the instructions for using âVite as the clientâs build toolâ produced one previously.
Besides that, no auth logic was implemented, imports were out place or missing, and an index.jsx
file was also nowhere to be found, so I decided to stop there.
Iâm sure I could have iterated on the prompt enough times until I got closer to a working app, but at ~$0.80-$1.20 a generation, I didnât feel like racking up more of an OpenAI bill.
Note đž
Price per generation is another big difference between the Smol AI and Wasp AI. Because more work is being done by Waspâs compiler and less by GPT, each app costs about ~$0.10-$0.20 to generate (although Wasp covers the cost and allows you to use it for free), whereas to generate complex full-stack apps with Smol-Developer can cost upwards of ~$10.00!
Plus, there are plenty of YouTubers whoâve created videos about the process of using Smol-Developer and it seems they all come to similar conclusions: you need to create a very detailed and explicit prompt in order to get a working prototype (In fact, in AI Jasonâs Smol-AI video above, he mentioned that he got the best results out of the box when prompting Smol-Developer to write everything to one file only â of course this limits you to generating simple apps only that are not so easy to continue from manually).
Thoughts & Further Considerations
At their core, SmolAI and WaspAI function quite similarly, by first prompting the LLM to create a plan for the appâs architecture, and then to execute on that plan, file by file.
But because Smol-Developer aims to be able to generate a wider range of apps, the expectation is on the Developer (or âPrompt Engineerâ) to create a highly detailed, explicit prompt, which is more akin to a Product Requirement Doc that a Product Designer would write. This can take a few iterations to get right and pushes Smol-Developer in the direction of âNatural Language Programmingâ tool.
On the other hand, Waspâs GPT Web App Generator has a lot of prompting and programming going on behind the scenes, abstracted away from the user and hidden within the Generatorâs code and Waspâs compiler. Wasp comes with a lot of knowledge baked in and already has a good idea of what it wants to build, which means the user has less to think about it. This means that weâre more likely to get a working complex prototype from a short, simple prompt, but we have less flexibility in the kinds of apps weâre able to create â we always get a full-stack web app.
In general, Wasp is like a junior developer specialized in web dev and has a lot of experience with a specific stack, while Smol AI is a junior developer thatâs a generalist who is more versatile, but has less specific knowledge and experience with web dev đ
Smol AI | Wasp AI | |
---|---|---|
đ§âđ»Â Types of Apps | Varied | Full-stack Web Apps |
đŻÂ Programming Languages | All Types | JavaScript/TypeScript |
đ Complexity of Generated App | Simple to Medium | Medium to Complex |
đ°Â Price per Generation â via OpenAIâs API | $0.80 to $10.00 | $0.10 to $0.20Â |
đłÂ Payment Method | bring your own API key | free â paid for by Wasp |
đ Debugging | Yes, if youâre willing to tinker | Built-in, but limited |
đŁÂ Type of Prompt Needed | Complex and detailed, 1 or more pages (e.g. an entire Product Requirement Doc) | Simple, 1-3 paragraphs |
đ Intended User | Engineers, Product Designers wanting to generate a broad range of simple prototypes | Web Devs, Product Designers that want a feature rich full-stack web app prototype |
Other big differences lie within:
- Error Correction upon Code Creation
- Smol AI initially had a debugging script, but this has temporarily deprecated due to the fact that it expects the entire codebase when debugging, and current 32k and 100k token context windows are only available in private beta for GPT4 and Anthropic at the moment.
- Wasp AI has some error correction baked into its process, as the structure of a Wasp app is more defined and the range of errors are more predictable.
- Price per app generation via OpenAIâs chat completion endpoints
- Smol AI can cost anywhere from ~$0.80 to $10.00 depending on the complexity of the app.
- Wasp AI costs ~$0.10 to $0.20 per app, when using the default mix of GPT 4 and GPT 3.5 turbo, but Wasp covers the bill here. If you choose to run it just with GPT4, then the cost is 10x at $1.00 to $2.00 per generation and you have to provide your own API key.
- User Interface
- Smol Developer works through the command line and has minimal logging and process feedback
- Wasp AI currently uses a clean web app UI with more logging and feedback, as well as through the command line without a UI (you have to download the experimental Wasp release to do so at this time).
Overall, both solutions produce amazing results, allowing solo developers or teams iterate on ideas and generate prototypes faster than before. But they still have a lot of room for improvement.
For example, what these tools lack the most at the moment is in interactive debugging and incremental generation. It would be great if they could allow the user to generate additional code and fix problems in the codebase on the fly, rather than having to go back, rewrite the prompt, and regenerate an entire new codebase.
Iâm not aware of the Smol AI roadmap, but seeing that itâs received a grant from Vercelâs AI accelerator program, Iâm sure we will be seeing development on it continue and the tool improve (let me know in the comments if you do have some insight here).
On the other hand, as Iâm a member of the Wasp team, I can confidently say that Wasp will soon be adding the initial generation process and interactive debugging into Waspâs command line interface!
So I definitely think itâs early days and that these tools will continue to progress â and continue to produce more impressive results đ
Which Tool Should You Use?
Obviously, there can be no clear winner here as the answer to question of which tool you should use as your next âAI Junior Developerâ depends largely on your goals.
Are you looking for a tool that can generate a broad range of simple apps? And are you interested in learning more about building AI-assisted coding tools and natural language programming and donât mind tweaking and tinkering for a while? Well then, Smol-Developer is what youâre looking for!
Do you want to generate a working full-stack React/Node app prototype with all the bells and whistles as quickly and easily as possible? Head straight for Waspâs GPT Web App Generator!
Help Me Help You đ
If you havenât yet, please star us on GitHub, especially if you found this useful! If you do, it helps support us in creating more content like this. And if you donât⊠well, we will deal with it, I guess.
âïž Thanks For Your Support đ
In general, as Jason âAI Jasonâ Zhou said:
âIâm really excited about [AI-assisted coding tools] because if I want to user-test a certain product idea I can ask it to build a prototype very, very quickly, and test with real usersâ
Jason makes a great point here, that these tools donât really have the capacity to replace Junior Developers entirely in their current capacity (although they will surely improve in the future), but they do improve the speed and ease with which we can try out novel ideas!
I personally believe that in the near future we will see more domain-specific AI-assisted tools like Waspâs GPT Web App Generator because of the performance gains they bring to the end user. Code agents that are focused on a niche can produce better results out of the box due to the embedded knowledge. In the future, I think we can expect a lot of agents that are each tailored towards fulfilling a specific task.
But donât just take my word for it. Go ahead try out Smol-Developer and the GPT Web App Generator for yourself and let me know what you think in the comments!
Top comments (15)
It seems like purpose-built AI developers are always going to be more successful than general ones: shared across every web app, or server application, or any other kind of app, there will always be a lot of common knowledge that the agent needs. Putting that work onto the tool developer will make using it easier and allow for ever-increasing quality, as the tool developer iterates on the prompts.
GPT App Generator is a specific tool -> generating web apps from a description.
Smol AI is trying to be too general -> generating any app from a description. But these descriptions have to be so specific, at some point you have to wonder when you're just trying to program with English instead of code. And this makes sense: we've seen from failures in the past (like COBOL) how trying to make a general-purpose program description language to obviate the need for programmers just results in a bad programming language that still requires programmers.
this is very well put. it will be interesting to see how this holds up as LLMs improve over time.
wow, well said :)
Of course Wasp AI, is that even a question?
Smol step for a human, a big step for humanity :D
đ
Great comparison, to the point!
Thanks, Marin!
Great comparison! I think Wasp really has a great future in the AI-assisted web dev space đ
Thanks!
Also missed Sweep AI, an AI junior dev that's fully asynchronous, so you can offload your workflow to Sweep and work on something else. You can a GitHub issue for Sweep to address it by making a pull request. sweep.dev
Whatâs been your experience with it? Itâs very expensive! Smol and Wasp AI are much cheaper
Full disclosure: I'm one of the developers of Sweep. Our experience using it internally has been amazing, allowing us to ship way faster than before for small menial tasks.
We charge more because Sweep does more. Sweep's a fully autonomous AI junior dev with more scope rather than a convenient tool in your IDE or the like.
wow, this is a super detailed comparison! I especially like the table where you can quickly see the differences between the two.
Thanks!