Building an AI Debate Arena: Leveraging Multiple APIs for Enhanced Conversational AI

As an AI enthusiast and developer, I recently embarked on an exciting journey to create an AI Debate Arena. This project not only challenged my technical skills but also provided valuable insights into working with multiple AI APIs. In this post, I'll share my experiences, the challenges I faced, and the lessons I learned while building this unique platform.

Project Overview:

The AI Debate Arena is a web application where users can watch and interact with AI-powered debates on current topics. The unique twist is that one AI (using Mistral) argues for a position, while the other (using GPT-2) argues against it. Users learn about complex issues from multiple perspectives.

Tech Stack

• Next.js for the frontend and API routes

• Tailwind CSS for styling

• Hugging Face Inference API for GPT-2

• Mistral AI API for the Mistral model

Setting Up Multiple APIs

One of the most interesting aspects of this project was working with multiple AI APIs.

Here's how I set them up:

Hugging Face Inference API (GPT-2):

import { NextResponse } from 'next/server';
import { HfInference } from '@huggingface/inference';

const hf = new HfInference(process.env.HUGGINGFACE_API_KEY);

export async function POST(req: Request) {
  try {
    const { prompt } = await req.json();

    if (!prompt) {
      return NextResponse.json({ error: 'Prompt is required' }, { status: 400 });
    }

    const response = await hf.textGeneration({
      model: 'openai-community/gpt2',
      inputs: `Provide a concise and logical argument for the following debate topic: ${prompt}\n\nArgument:`,
      parameters: {
        max_new_tokens: 100,
        temperature: 0.7,
        top_p: 0.95,
        repetition_penalty: 1.2,
        stop: ["\n", "Argument:"],
      },
    });

    const cleanedResponse = response.generated_text.trim().replace(/^Argument:\s*/, '');

    return NextResponse.json({ response: cleanedResponse });
  } catch (error: any) {
    console.error('Error in Hugging Face API:', error);
    return NextResponse.json({ error: 'Internal Server Error', details: error.message }, { status: 500 });
  }
}

Mistral AI API:

import { NextResponse } from 'next/server';
import MistralClient from '@mistralai/mistralai';

const mistral = new MistralClient(process.env.MISTRAL_API_KEY);

export async function POST(req: Request) {
  try {
    const { prompt } = await req.json();

    if (!prompt) {
      return NextResponse.json({ error: 'Prompt is required' }, { status: 400 });
    }

    const chatResponse = await mistral.chat({
      model: "mistral-tiny",
      messages: [
        { role: "system", content: "You are participating in a debate. Provide concise and logical arguments." },
        { role: "user", content: prompt }
      ],
      max_tokens: 150,
      temperature: 0.7,
    });

    return NextResponse.json({ response: chatResponse.choices[0].message.content });
  } catch (error: any) {
    console.error('Error in Mistral AI API:', error);
    return NextResponse.json({ error: error.message || 'Internal Server Error' }, { status: 500 });
  }
}

Challenges and Solutions

1. API Response Formatting: Each API returned responses in slightly different formats. To standardize this, I created wrapper functions for each API call that cleaned and formatted the responses consistently.

2. Rate Limiting: I encountered rate limiting issues, especially with the Hugging Face API. To mitigate this, I implemented a simple queue system to manage API calls and added error handling to gracefully deal with rate limit errors.

3. Context Management: Ensuring that each AI model maintained context throughout the debate was challenging. I solved this by passing previous arguments as context in subsequent API calls.

Performance Comparison

Interestingly, I found that the performance of these APIs varied depending on the task:

• Mistral AI: Excelled in providing more nuanced and context-aware responses. It was particularly good at maintaining a consistent argument throughout the debate.

• GPT-2 (via Hugging Face): While less contextually aware, it often provided more creative and unexpected arguments, which added an interesting dynamic to the debates.

Guardrailing and Fine-tuning

I experimented with guardrailing to ensure the AI responses remained on-topic and appropriate. This was relatively straightforward with both APIs, involving prompt engineering and response filtering.
Fine-tuning was more challenging, especially with the Hugging Face API. However, I found that careful prompt design often negated the need for extensive fine-tuning.

Unique API Features

• Hugging Face: The variety of models available was a significant advantage. It allowed me to experiment with different GPT-2 variants easily.

• Mistral AI: Offered better out-of-the-box performance for maintaining context in longer conversations.

Lessons Learned

1. API Diversity is Valuable: Using multiple APIs provided robustness and allowed for more dynamic interactions.

2. Prompt Engineering is Crucial: Well-crafted prompts significantly improved the quality of AI responses across all APIs.

3. Error Handling is Key: Robust error handling for API calls is essential for a smooth user experience.

Conclusion

Building the AI Debate Arena was an enlightening experience. It showcased the strengths and weaknesses of different AI APIs and highlighted the importance of thoughtful integration in creating engaging AI-powered applications.

The project is open-source and available on GitHub.
I welcome contributions and feedback from the community!