First, I want to say thank you to the team at Vercel for putting in the leg work and creating Vercel AI SDK! Which allows you to create AI related projects using React and Svelte.Over the weekend, I decided to create a beautiful but minimalistic Linkedin Bio generator.
Introduction:
Read Vercel blog post introducing the AI SDK. After looking through the blog post, feel free to browse around the templates devs have started for us. Once you have found a template, feel free to look at the demo and review the GitHub repo. If you don't understand anything, check online or use your favorite chatbot!
I decided to go with the twitter bio generator since I had found out about the Vercel AI SDK on twitter. Since I was on LinkedIn earlier that day, I decided to create a LinkedIn summary generator or bio generator that anyone with a social network can use. I will be explaining, the features of next.js , and general walkthrough for building LLM powered apps!
Overview Of Vercel:
Vercel is a powerful platform to build AI-powered applications, with popular open-source and cloud LLMs, including support for streaming generative responses. It provides a seamless deployment experience for Next.JS applications, making it an ideal choice for our Linkedin Bio website. With Vercel, you can take advantage of features like automatic deployments, built-in CDN, server less functions, and more. It offers scalability, performance, and ease of use, allowing you to focus on building your website without worrying about infrastructure management.
Setting up the Environment:
To get started, we need to set up our environment. Open your IDE or terminal to run locally.Check the resources section for the Blog post by the Vercel team and the GitHub Repo!
The Frontend:
The Next.js frontend consists of a few elements:
A text box for users to copy their current bio or write a few sentences about themselves.
A dropdown where they can select the tone of the bio they want to be generated.
A submit Botton for generating their bio, which when clicked calls an API route that uses OpenAI's GPT-3 model and returns two generated bios!
Two containers to display the generated bio after we get them back from the API route.
The Backend:
I am still relatively new to next.js, one of its advantages lies in its ability to seamlessly develop both frontend and backend of your web app.By leveraging Next.ja we can build full stack apps!
To see exactly how to build out your backend, please refer to the Official Vercel blog since they created Next.js!
Limitations of the serverless function approach
You can build your web app with a server less approach, but there are some limitations that make edge a better fit for this kind of application:
If we're building an app where we want to await longer responses, such as generating entire blog posts, responses will likely take over 10 seconds which can lead to serverless timeout issues on Vercel's hobby tier. BUT the PRO TIER has a 60-second timeout which is usually enough for GPT-3 applications.
Waiting several seconds before seeing any data isn't a good user experience. Ideally, we want to incrementally show the users data as it's generated - just like OpenAIs ChatGPT.
The responses may take even longer due serverless lambda functions.
Vercel Edge Functions with streaming are here to help! Depending on which Node.js libraries you're working with it may not be edge compatible.For this case, they will work great. Lets dive into what edge functions are and how we use them to make faster generations and better user experiences!
Edge Functions VS. Serverless Functions
Edge functions are basically a more lightweight version of serverless functions. They have a smaller limit on code size, memory, and doesn't support ALL node.js libraries. Here's why we still want to use them:
Because Edge functions use a smaller edge runtime and run very close to users on the edge, they're also fast. They have virtually no cold starts and are significantly faster than serverless functions.
They allows for a great users experience, especially when paired with streaming. Streaming a response breaks it down into small chunks and progressively sends them to the client, as opposed to waiting for the entire response before sending it.
Edge functions have 30 seconds timeout and even longer when streaming, which exceeds the timeout limit for serverless functions on Vercel's Hobby plan.Using these can allow you to get past timeout issues when using AI APIs that take longer to respond. As an added benefit, Edge functions are also cheaper to run!
https://youtu.be/9Q9_CQxFUKY
Video By Vercel Team!
Now that we understand the benefits and cost-effectiveness of using edge functions, let's refactor our existing code to use them. Now you can start with your backend's API route!
Check the GitHub repo from the Vercel team or myself. Open the generate.ts file and you can see we define the edge function as an "edge" function. You will also see an added extra variable to the payload, to make sure OpenAI streams in chunks back to the client.
After the payload is defined there is a stream variable. There is a helper function, OPENAISTREAM, that enables us to incrementally stream responses to the client as we get data from OpenAI.
If you open the the OPENAIStream.ts file. You can see we sent a POST request to OPENAI with the payload. We create a stream to continuously parse the data received from OpenAI, all while waiting for the [DONE] token to be sent since this signifies its completion.When that happens the stream its closed.
Back in the frontend
We define a reader using the native we API, getReader(), and progressively add data to our generatedBio state as it's streamed in.
Conclusion
Thank you again to the Vercel team for creating the Vercel SDK , templates and blogs to help devs learn and ship products. If you want to create your own version or my Bioninja or the original TwitterBio from the Vercel team check out the resources below!
Keep learning , building and trying new things! Don't get trapped and live with the results of other people thinking!
Resources:
Bio Ninja - DEMO
Github: Bioninja
Top comments (0)