This is a submission for the Nylas Challenge: AI Expedition & Galaxy Brain

What I Built and Why

MailTales is a web app designed to breathe life into your email inbox. With the help of Nylas & AI, MailTales offers unique and creative ways to interact with your emails, transforming them into bite-sized stories, engaging summaries, and even providing insights through sentiment analysis. You can try it https://mailtales.amitwani.dev

Demo

MailTales: You can go to https://mailtales.amitwani.dev

Features

Chat Over Your Inbox: With MailTales, you can chat with your email inbox!

AI-Powered Email Summaries: MailTales provides concise, AI-generated summaries of your emails

Listen to AI-Generated Stories: MailTales can convert the content of your emails into engaging audio stories.

Sentiment Analysis and Categorization: MailTales' sentiment analysis feature helps you understand the tone of your messages. The app also categorizes emails based on their content.

Code

Tech Stack

Full Stack: NextJS
Backend: NodeJS
Deployment: Vercel, Fly.io
Database: PostgreSQL
Queues: Upstash Kafka
Email: Nylas
AI: Gemini

Architecture

Nylas: Nylas is used for authentication of user and fetching their emails from inbox
Gemini: Gemini is used for generating AI responses as well as generating embeddings. Model used for generating responses is gemini-1.5-flash and for embeddings it is text-embedding-004
Postgres: Postgres is used to store user and emails data. @vercel/postgres is used for this.
Upstash Kafka: Upstash Kafka is used for the job of generating email embeddings. Generating email embeddings is done by a NodeJS Backend by consuming messages from Upstash Kafka topic.

Your Journey

Nylas

I have used Nylas to get email inbox and its contents. I have used the Nylas NodeJS SDK for this. Logging in the user was very easy using the SDK. The grantId received is then saved to the DB for further requests. Overall it was really interesting experience in building on top of Nylas features. There are many more features which are still left to explore!

Get Emails

const queryParams: ListMessagesQueryParams = {
    in: ["INBOX"],
    limit: limit,
  }

  if (pageToken && pageToken.length > 0) {
    queryParams.pageToken = pageToken;
  }

  const messages = await nylas.messages.list({
    identifier: grantId!,
    queryParams,
  });

Search Emails

const queryParams: ListMessagesQueryParams = {
    in: ["INBOX"],
    limit: limit,
    searchQueryNative: "query"
  }

  if (pageToken && pageToken.length > 0) {
    queryParams.pageToken = pageToken;
  }

  const messages = await nylas.messages.list({
    identifier: grantId!,
    queryParams,
  });

Upstash Kafka

Used Upstash Kafka's REST API to produce message.

 const response = await fetch(`${process.env.UPSTASH_KAFKA_REST_URL}/produce`, {
      method: 'POST',
      headers: {
        Authorization: `Basic ${process.env.UPSTASH_KAFKA_REST_AUTH}`
      },
      body: JSON.stringify({ topic: "generate-email-embeddings", value: user.id })
    })

Used kafkajs in the backend to consume the messages

const kafka = new Kafka({
  brokers: [`${process.env.UPSTASH_KAFKA_BROKERS}`],
  ssl: true,
  sasl: {
    mechanism: 'scram-sha-256',
    username: `${process.env.UPSTASH_KAFKA_USERNAME}`,
    password: `${process.env.UPSTASH_KAFKA_PASSWORD}`
  },
  logLevel: logLevel.ERROR,
});

const consumer = kafka.consumer({ groupId: 'consumer_group_1' });

const run = async () => {
  await consumer.connect();
  await consumer.subscribe({ topic: 'generate-email-embeddings', fromBeginning: true });

  await consumer.run({
    eachMessage: async ({ topic, partition, message }) => {
      console.log({
        partition,
        offset: message.offset,
        value: message.value?.toString(),
      });

      const userId = Number(message.value?.toString());

      console.log("Generating email embeddings for user", userId);

      await generateEmailEmbeddings(userId);
    },
  });
};

Gemini & Vercel AI SDK

Used Vercel AI SDK to interact with Gemini models.

Get Streaming Response

export async function getGoogleResponse(body: any, system: string = "", tools: any = null) {
  const result = await streamText({
    model: google('gemini-1.5-flash'),
    system,
    messages: body,
    temperature: 0.0001,
    tools,
  });

  return result;
}

Generate Embeddings

export async function generateEmbeddings(emailStrings: string[]) {
  const { embeddings } = await embedMany({
    model: google.embedding('text-embedding-004'),
    values: emailStrings,
  });

  return embeddings;
}

Also, this was the first time of using NextJS and building a proper RAG agent from the scratch using Vercel AI SDK, so overall a great learning experience.

Future Scope

• More deeper insights into mailbox
• Integration of AI in alot of parts of the app
• Crafting beautiful professional replies.

Conclusion

It was a great experience in building this app using Nylas & Gemini over NextJS. I have learnt alot in the last week while building this project. Thanks to Dev.To as well for providing such platforms for building the projects.