Pythagora_v1 release reflection – what we learned, current state, next steps

Today, we’re launching Pythagora v1, so we wanted to reflect on our progress in the past year since we released the first version of GPT Pilot (the open source brain of Pythagora) and in the past six months since we released Pythagora beta.

In this post, you’ll read about how we got carried away with hype, thinking AI will be able to build apps much easier than it really can, how flashy demos deceived everyone as well as us, how we switched our focus from the technology to the DX (Developer experience), what can you build with Pythagora, why are we keeping the core technology open source, and what are our next steps.

History of Pythagora

When we started building Pythagora, our North Star was clear: enable developers to create production-ready applications using natural language. I’ve been a developer for the past 15 years, and my dream is to be able to build applications without having to deal with debugging. So, when I realized how good LLMs can code (you can read about my first experience with coding with ChatGPT in this blog post), I immediately started looking into how we can automate this process with my cofounder, Leon.

After a couple of months of experimentation, we created GPT Pilot. This was in September 2023. The community reacted well, so we continued working on it and soon started fundraising. At this time, we needed to create a roadmap for what we plan to build next year. We ended up building most of the roadmap, but the capabilities we set GPT Pilot to have were missed by a huge margin.

We predicted that, in 6 months (around May 2024), GPT Pilot will be able to build a 30k lines of code app in 2 days!

Learnings from Beta to v1

Not only that it wasn’t able to build a 30k LOC app in any amount of time, but it was so hard to use that we needed to do a reality check.

We realized that a bunch of other projects started to come up (most notably Devin) that promised the world while all they did was create a nice, flashy demo.

In the early days, we packed Pythagora with features like automated tests and autonomous code reviews, aiming for maximum autonomy. But reality hit hard. The technology just wasn’t there yet. So, we decided to take a step back.

We removed many components of Pythagora (like automated tests and self code review), refocused our efforts on user experience, and started investigating how people can use Pythagora to build any app they will end up actually using. So, we began introducing more and more human-in-the-loop interactions.

LLMs are still a technology in its infancy, and we must treat it as such. Many people think we can just say two sentences, like “Build me a clone of Reddit”, to AI and let it build the entire app with the infrastructure. Many decisions need to be made about how the app should work. This is exactly what we’re focused on. We want Pythagora to manage the codebase and ask humans whenever they need to make a decision.

Current state of Pythagora

With today’s release, we believe we’re breaking the new boundaries for us and the rest of the world. With Pythagora, people can build apps with up to 5000 lines of code ONLY by writing in natural language. You can see apps built with Pythagora here. All code in all these apps was written exclusively by Pythagora, and humans wrote no lines of code.

In reality, you can build even bigger apps with even 10k LOC. However, above the 5k LOC, the experience gets trickier, but we’re continuously working to push that boundary. Nevertheless, 5k LOC is quite a solid app size, which makes it truly useful. This is our north star – we want Pythagora to build apps that actually end up in production and not in some random folder on a computer.

That is why we added autonomous deployment so you can deploy the created app in one click online and share it with others.

We believe Pythagora is currently one of the most capable tools for building applications through plain English instructions, minimizing the need to write code manually. But hey, if you know of a tool that is able to build apps with this complexity without requiring humans to write a single line of code, we’d genuinely love to hear about it. Drop a comment or ping me directly at zvonimir@pythagora.ai

That said, we truly believe in that common phrase that AI will not replace humans, but humans using AI will. Pythagora is not a fully autonomous AI developer that wanders off into the wilderness of software development and returns with a fully built app – it is a tool that helps people build apps stellar fast without writing and managing the codebase. You just explain what you want to build and talk to Pythagora, as it constantly asks you questions to understand if it’s going in the right direction.

Managing expectations

First off, let’s temper expectations around zero-shot app creation. While it’s tempting to imagine an AI that builds an entire app from a few sentences, it’s not just about the technological hurdles. Even if the tech gets there—and it likely will in the next few years—is that what people really want?

Imagine working with a freelancer developer today to whom you can give a project and wait for them to come back with working software. Now, imagine telling that human developer your app idea in three sentences, and they disappear for a month, returning with a fully built app based on their interpretations. Chances are, it won’t align with your vision. Development is an iterative process filled with decisions that need to be reviewed, discussed, and tweaked.

We believe that humans will always be the decision-makers and creators. Our mission with Pythagora is to offload the grunt work to AI while keeping you in the driver’s seat. Think of humans as a control center operator—a concept echoed by Milo Medin from Benchmark Capital on a 20VC podcast—where AI handles the heavy lifting, but the human guides the direction.

In practical terms, this means creating a seamless user experience where AI does the coding autonomously but reaches out whenever a decision needs to be made. We want to avoid constant context-switching, where you must check in with the AI every few minutes or wait hours for it to return with updates. Rather, we want to enable a constant experience like with any other digital tool where you work with Pythagora for hours or days but get weeks or months of work done.

Looking Forward: The Future of Pythagora

We are very much focused on improving the agentic system, open sourced GPT Pilot, so it can solve more problems and build larger apps faster. However, we are even more focused on how to engage the human, the developer behind the keyboard whose goal is to make their ideas a reality. We are rigorously focused on improving the Developer Experience and making Pythagora a new extension of human creativity.

With that, we will be more focused on building the knowledge base and learning materials that will help everyone become a better builder of software applications. You can watch a 15 minute crash course on Pythagora here:

We are very passionate about building useful software tools to make our and other people’s lives easier, and we want others to be able to do that as well, without having to spend 15 years learning the ins and outs of how to write code.

I almost never had to spend effort on memory allocation while it was a requirement for every developer just a decade before I became an engineer. I hope that happens to writing and managing the countless lines of code in evergrowing codebases.

Conclusion

Launching Pythagora v1 is both a celebration and a stepping stone. We’re proud of how far we’ve come, but we’re even more excited about where we’re headed. The landscape of AI in code generation is evolving rapidly, and while there are challenges, these challenges make our team get up in the morning and march forward to solve them.

We’re committed to bridging the gap between AI capabilities and developer needs, focusing on creating tools that enhance your workflow rather than dictate it. As we continue to develop Pythagora, your feedback is invaluable. Let’s shape the future of coding together. You can sign up for early access to Pythagora v1 here.

Thanks for taking the time to read this, and I wish you happy coding…I mean, building 🙂

PS If you know of any other tools pushing the boundaries in AI code generation, share them! The more we collaborate, the better tools we’ll all have.