aiSangeet: Generate Unique Music with Images from Prompts!

Introduction

Hey everyone! I’m excited to introduce my latest project, aiSangeet, a creative web app that merges the power of AI with music and visuals. If you're someone who enjoys experimenting with sounds and art, this tool is designed just for you!

What is aiSangeet?

aiSangeet is an AI-powered web app that allows users to generate unique pieces of music along with stunning visuals based on prompts you provide. Whether you're looking to create ambient background music or something more experimental, aiSangeet helps bring your ideas to life in just a few clicks.

How Does It Work?

Simply enter a prompt—anything from a mood to a specific theme—and let the AI do the rest! aiSangeet uses a blend of advanced machine learning models to generate a custom music track along with an image that represents the mood or theme of your input. It’s a simple yet powerful tool for musicians, visual artists, or anyone curious about the intersection of AI, music, and art.

Sample Code (React + Hugging Face)

Here’s an example of how you can use Hugging Face models on the backend and React on the frontend to generate both music and images:

React Frontend (Music and Image Prompt Form)

import React, { useState } from 'react';

function App() {
  const [prompt, setPrompt] = useState('');
  const [imageUrl, setImageUrl] = useState('');
  const [musicUrl, setMusicUrl] = useState('');

  const handleGenerate = async () => {
    // Call the backend to generate both image and music
    const response = await fetch('/api/generate', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({ prompt }),
    });
    const data = await response.json();
    setImageUrl(data.image);
    setMusicUrl(data.music);
  };

  return (
    <div>
      <h1>aiSangeet</h1>
      <input
        type="text"
        placeholder="Enter a prompt"
        value={prompt}
        onChange={(e) => setPrompt(e.target.value)}
      />
      <button onClick={handleGenerate}>Generate</button>

      {imageUrl && <img src={imageUrl} alt="Generated" />}
      {musicUrl && <audio controls>
        <source src={musicUrl} type="audio/mpeg" />
      </audio>}
    </div>
  );
}

export default App;

This React code sets up a simple UI to collect the prompt and then triggers the generation process.

Backend (Node.js + Hugging Face)

For the backend, you can use Node.js to interact with Hugging Face models to generate both music and images.

const express = require('express');
const axios = require('axios');
const FormData = require('form-data');

const app = express();
app.use(express.json());

app.post('/api/generate', async (req, res) => {
  const { prompt } = req.body;

  try {
    // Call Hugging Face API for image generation
    const imageResponse = await axios.post(
      'https://api-inference.huggingface.co/models/CompVis/stable-diffusion-v1-4',
      { inputs: prompt },
      {
        headers: {
          Authorization: `Bearer YOUR_HUGGINGFACE_API_KEY`,
        },
      }
    );
    const image = imageResponse.data[0].url;

    // Call Hugging Face API for music generation
    const musicResponse = await axios.post(
      'https://api-inference.huggingface.co/models/YOUR_MUSIC_MODEL',
      { inputs: prompt },
      {
        headers: {
          Authorization: `Bearer YOUR_HUGGINGFACE_API_KEY`,
        },
      }
    );
    const music = musicResponse.data.music_url; // Assume response contains music URL

    res.json({ image, music });
  } catch (error) {
    console.error(error);
    res.status(500).send('Error generating music or image');
  }
});

app.listen(5000, () => {
  console.log('Server is running on port 5000');
});

In this backend code:

• We use Hugging Face models for both image and music generation.
• The Stable Diffusion model is used for generating the image, and you can swap in a music generation model available on Hugging Face for the audio part.
• The React app fetches this data and displays it to the user.

About the Creator

aiSangeet was created by Vishal Tiwari, a passionate developer currently pursuing B.Tech in Computer Science (AI & ML). Vishal is also the creator of Engineer Astra, a platform helping students explore top colleges across India. You can check out more of Vishal's work on Engineer Astra as well!

Try aiSangeet Today

You can try out aiSangeet for yourself here! I’d love to hear your feedback, so feel free to leave a comment or reach out if you have any suggestions or ideas for future updates.