Recent AI models enhance software development experience and boost productivity. Our team integrates generative AI tools such as Github Copilot and Codium into their everyday coding tasks. Additionally, we've developed a custom UI for ChatGPT and made it accessible across the company. It's proven to be a valuable time-saving tool for both developers and designers.
We're also exploring vision models and beginning to utilize them in web front-end development. One concept is to employ a vision model to generate a functional web app from images. This method differs from Copilot or Codium, which provide micro-level assistance, such as writing a function, refactoring a class, or adding tests.
Other tools in the market strive for pixel-perfect pages, but they often produce unusable code due to incorrect page layout plans, inline CSS, an overload of HTML tags, or hard-coded data in the layout. This makes their output essentially useless for developers.
Our tool, on the other hand, scaffolds the entire app and offers a comprehensive app generation process. Simply input images and receive a fully functioning NextJS app in return!
How is this approach different?
The AI agent applies a range of prompting techniques to mirror a developer's thought process. Initially, it generalizes the layout from several images, before subsequently devising unique content for each page.
The produced code is reusable.
This tool is capable of generalizing layouts from various images, thus enabling the creation of shareable components. Additionally, it separates data or app state from the layout, facilitating seamless integration with APIs for real data.
Figure 1: Input UI image with sidebar on the right-hand side and a list of transactions
Figure 2: The tool is able to detect reusable components, such as sidebar and topbar, and put them in shared folder /app/_components
Figure 3: The tool is able to separate transactions data from the layout code
How much development time can it save?
The tool performs well on certain apps, creating pages that mirror the UI images. In some instances, it can create input forms with a 70% match, requiring only minor CSS adjustments to align with the UI design.
We utilized the tool for a client project, and it reduced development time by approximately 20%.
We are enhancing our prompt engineering to tackle challenges with certain layout types. We're also considering a feedback loop, where the apps created by the model are fed back into it for self-improvement.
Figure 4a: UI design of contact form
Figure 4b: Generated page of contact form
Figure 5: Generated ReactJS & TailwindCSS code for the contact form
Can I try this tool?
Indeed, a Proof of Concept (PoC) is available for you to try out. Refer to the instructional video for guidance on how to use the app.
This is currently a PoC, and due to the unpredictable nature of generative A, its effectiveness may vary depending on your application. We appreciate any feedback and encourage you to email us at teamcodest@codelink.io.
Here are some examples of applications generated entirely by the tool without any manual intervention.
Demo link
codest.ai
Top comments (1)
Wow!! I have been playing with Azure's Vision model a bit, but this post really blew my mind! Very cool! Thanks for sharing this.