DEV Community

Cover image for Chat with your PDF using Pinata,OpenAI and Streamlit

Chat with your PDF using Pinata,OpenAI and Streamlit

Jagroop Singh on October 10, 2024

In this tutorial, we’ll build a simple chat interface that allows users to upload a PDF, retrieve its content using OpenAI’s API, and display the r...
Collapse
 
john12 profile image
john

@jagroop2001 ,
Wow, I didn't know we could create such an AI project so easily! Integrating Pinata, OpenAI, and Streamlit opens up so many possibilities for building interactive applications.
I would try with image to text generation. Can you suggest is it feasible ?

Collapse
 
jagroop2001 profile image
Jagroop Singh

@john12 ,
Creating an image-to-text generation application using OpenAI's models is definitely possible! The advanced features of models like GPT-4 and DALL-E make this a realistic and exciting project.

Collapse
 
john12 profile image
john

@jagroop2001 ,Thank you so much! 🙌✨ I'm really excited about this project! 😊

Thread Thread
 
jagroop2001 profile image
Jagroop Singh

@john12 , let me know if you need any help in your project.

Thread Thread
 
john12 profile image
john

@jagroop2001 , sure thanks

Collapse
 
anh_ta_ec592abca466578198 profile image
Anh Ta

Great tutorial @jagroop2001 !

Question: How do I restrict the PDFs to files that I have already preloaded? Basically, disallow users from uploading their own documents and only allowing chat with my PDFs.

Collapse
 
jagroop2001 profile image
Jagroop Singh • Edited

@anh_ta_ec592abca466578198 ,
To restrict users to only chat with preloaded PDFs, you can disable the file upload functionality and instead provide a list or dropdown menu of the available PDFs ( Yes you can manually upload files on @pinata server
directly)
. When a user selects a PDF from the list, you can load that specific document for processing. This ensures they can only access and interact with the PDFs you have preloaded, preventing them from uploading any of their own files.

Collapse
 
anh_ta_ec592abca466578198 profile image
Anh Ta

Thank you! I'll try it.

Thread Thread
 
jagroop2001 profile image
Jagroop Singh
Collapse
 
martinbaun profile image
Martin Baun

I highly recommend you take a look at LangChain :)

Collapse
 
jagroop2001 profile image
Jagroop Singh

Sure @martinbaun , any resources ??

Collapse
 
jagroop2001 profile image
Jagroop Singh

@martinbaun , are you pointing to built this project using Langchain using RAG's ?

Collapse
 
alonsoir profile image
@alonso_isidoro

This example is not going to prevent hallucinations, nor are they going to be indexed. In my opinion, having the files in IPFS is a good idea to keep the files in a secondary location before loading them into something like FAISS or some other vector database with a suitable index applied. PDF files need to be processed, they may contain images and tables and these need to be indexed as well.

Collapse
 
jagroop2001 profile image
Jagroop Singh • Edited

@alonsoir ,
I think combining IPFS for storage and a solid indexing approach with vector databases will create a more reliable system for handling diverse content types within PDFs.

Collapse
 
hraifi profile image
sewiko • Edited

I didn't get any mail from pinata due to this I am not able to continue with this. @jagroop2001

Image description

Collapse
 
jagroop2001 profile image
Jagroop Singh

Please reach out to @pinata team through this email team@pinata.cloud or try with different email account. @hraifi

Collapse
 
hraifi profile image
sewiko

@jagroop2001 thanks, worked for my different email account.

Image description

Thread Thread
 
jagroop2001 profile image
Jagroop Singh

Oh great !!

Collapse
 
works profile image
Web

@jagroop2001
Whoa! Fantastic Project using OpenAI and Pinata. I've tried this and it works well.
Your API keys aren't functioning, by the way. I attempted to utilize this

Collapse
 
jagroop2001 profile image
Jagroop Singh

@works ,
yes because I have shown this for demo purpose after that I delete the @pinata keys and regenerated new ones.

Collapse
 
works profile image
Web

@jagroop2001 , got it.
Can you guide me that how I would build a platform like that where code file uploaded and OpenAI generate code review of it and also provide optimized code correction.

Is this possible with OPEN AI

Thread Thread
 
jagroop2001 profile image
Jagroop Singh

@works ,
Yes, it's possible to build a platform that allows code file uploads, with OpenAI generating code reviews and offering optimised corrections. You can achieve this by integrating OpenAI's API or Gemini API ( which is free) or Open Source Model for code analysis and Pinata for secure file storage, all within a React-based front-end.

I'm already working on this exact problem statement and plan to publish the project within a few days, using Pinata, OpenAI, React, and other technologies.

Thread Thread
 
works profile image
Web

Wow , I will be waiting for this as this would really help me to learn with your code refence. @jagroop2001

Collapse
 
paxnw profile image
caga

Sounds interesting @jagroop2001 ,
Why Pinata when we can store that in any 3rd party bucket or even in backend public folder ?

Collapse
 
jagroop2001 profile image
Jagroop Singh

@paxnw ,
@pinata is great because it leverages IPFS, giving files a decentralized home that's secure, accessible.
Unlike a typical backend folder or cloud storage, IPFS ensures that files are immutable and distributed, reducing dependency on any single server.
This can boost performance, especially for apps that need reliable file access across multiple locations. Plus, Pinata’s API makes integration and file management a breeze!

Collapse
 
paxnw profile image
caga

sound's conveniencing !! @jagroop2001

Thread Thread
 
paxnw profile image
caga

@jagroop2001 , while running got the error :

OpenAI's Response: Error:

You tried to access openai.Completion, but this is no longer supported in openai>=1.0.0 - see the README at https://github.com/openai/openai-python for the API.

You can run openai migrate to automatically upgrade your codebase to use the 1.0.0 interface.

Alternatively, you can pin your installation to the old version, e.g. pip install openai==0.28

A detailed migration guide is available here: https://github.com/openai/openai-python/discussions/742
Enter fullscreen mode Exit fullscreen mode

Do you know how to resolve this ? I tried to find online but doesn't work .

Thread Thread
 
jagroop2001 profile image
Jagroop Singh

@paxnw ,Yes, I have already experienced it. Fortunately, I know how to fix it:

If you want to upgrade your codebase to be compatible with the new version, you can run:

openai migrate
Enter fullscreen mode Exit fullscreen mode

OR
If you prefer to keep using the old version until you're ready to migrate, you can pin your installation:
pip install openai==0.28

Thread Thread
 
paxnw profile image
caga

@jagroop2001 , this one worked for me :

openai migrate
Enter fullscreen mode Exit fullscreen mode
Collapse
 
zh2332926 profile image
flydog259

pinata is not the best choice.

Collapse
 
jagroop2001 profile image
Jagroop Singh

What would you recommend @zh2332926

Collapse
 
zh2332926 profile image
flydog259

Arweave

Collapse
 
devnenyasha profile image
Melody Mbewe

Wow, I didn't know we could do such a project. Thats interesting

Collapse
 
jagroop2001 profile image
Jagroop Singh

Thanks, @devnenyasha ! There’s so much more to come with projects like these.

Collapse
 
iabdsam profile image
Abdul Samad

good idea

Collapse
 
jagroop2001 profile image
Jagroop Singh

Thanks @iabdsam

Collapse
 
josectoscano profile image
Jose Carlos Toscano

This is actually a great idea! Awesome implementation guys

Collapse
 
jagroop2001 profile image
Jagroop Singh

Thanks @josectoscano ,any other suggestion or feedback about this ?

Collapse
 
__4ldyfwkymhmd profile image
محمد يزيد الصبري

That's cool, well!

Collapse
 
jagroop2001 profile image
Jagroop Singh
Collapse
 
mine4me01 profile image
The Real Abraham 'Black MacGyver' Wilcox

totally rad!

Collapse
 
jagroop2001 profile image
Jagroop Singh

Thanks @mine4me01 !!

Collapse
 
femi_akinyemi profile image
Femi Akinyemi

But how do we test it outside the local Machine?

Collapse
 
jagroop2001 profile image
Jagroop Singh

@femi_akinyemi , one can either deploy it on streamlit or any other third party server like aws , google cloud , real cloud etc.

Collapse
 
dark_coder_vibes profile image
Dark Coder

there is a project called repochat that is for the same purpose. still great read!

Collapse
 
jagroop2001 profile image
Jagroop Singh

@dark_coder_vibes ,
Thank you for the heads-up! I appreciate the mention of repochat—it’s always interesting to see different approaches to similar goals. Glad you enjoyed the read!

Collapse
 
glimpse profile image
Glimpse

Cool!

Collapse
 
jagroop2001 profile image
Jagroop Singh

Thanks @glimpse ,There’s so much more to come with projects like these.

Collapse
 
detzam profile image
webstuff

did you use trial or u had to pus card info?

Collapse
 
jagroop2001 profile image
Jagroop Singh

@pinata offers a range of affordable plans, including a free tier, making it accessible for various users who need reliable file storage and management solutions.

Image description

Collapse
 
hraifi profile image
sewiko

Wow , such a great project for learning . I definitely work on this. @jagroop2001

Collapse
 
jagroop2001 profile image
Jagroop Singh

Thanks @hraifi , I am happy that this would help you.

Collapse
 
hraifi profile image
sewiko

@jagroop2001 , I am really passionate about this project and eager to contribute to its development. I believe working on it will help me deepen my understanding and skills, and I would love to extend its functionality further. I would like to propose this as my final year project for university, as it aligns perfectly with my academic goals. I'm confident that my contribution will add value, and I am excited to collaborate on this.

Thread Thread
 
jagroop2001 profile image
Jagroop Singh

Go for it @hraifi ,
make sure to create credentials of pinata account and try with videos and images as well with Open AI

Thread Thread
 
hraifi profile image
sewiko

Sure, OpenAI and Pinata are paid I think.

Thread Thread
 
jagroop2001 profile image
Jagroop Singh

Yes OpenAI is paid but @pinata is at least free for our personal projects where real user's are equals to none or very few.
I think you should use Gemini API key and update my code accordingly, It will work perfectly fine.

Thread Thread
 
hraifi profile image
sewiko

Thanks, just suggestion: You can add this project under Hacktoberfest so that more developer's will work on this. @jagroop2001

Thread Thread
 
jagroop2001 profile image
Jagroop Singh

@hraifi , great idea !!