DEV Community

Christian Heilmann
Christian Heilmann

Posted on

PrivateGPT - Running "ChatGPT" offline on local documents

PrivateGPT is a python script to interrogate local files using GPT4ALL, an open source large language model. It is pretty straight forward to set up:

  1. Clone the repo
  2. Download the LLM - about 10GB - and place it in a new folder called models.
  3. Place the documents you want to interrogate into the source_documents folder - by default, there's a text of the last US state of the union in there.
  4. Run the ingest.py script - this can a long time. On this MacBook M1 it was 2 minutes though
  5. Run the privateGPT.py script and you get a prompt in your Terminal:

I asked it what the US president has to say about the situation in the Ukraine and it gave me a synopsis and where in the document the information is from.

Screenshot of the terminal with a question providing an analysis of the situation in the Ukraine and the answer

Neat!

Top comments (13)

Collapse
 
alexmgp7 profile image
Alexander

can I use this to do my own chatgpt on a webpage?

Collapse
 
codepo8 profile image
Christian Heilmann

You could but there are easier implementations like this one: github.com/mlc-ai/web-llm

Collapse
 
alexmgp7 profile image
Alexander

I know is probably easy for most people, but I enter in the repository and I didnt understand, if you can help me more, I will be very grateful: c and, much text in English confuses me

Collapse
 
thumbone profile image
Bernd Wechner

I can't see why not. What would prevent that? It's a python script. Nothing to do you running out server side in response to requests and delivering responses based on our containing the results.

Collapse
 
alexmgp7 profile image
Alexander

I want to develop an AI chatbot in my university on a web that can help each student that need help of any information or protocol of the university.

Thread Thread
 
onlycoder000 profile image
Only Coder

use python fastapi

Collapse
 
alexmgp7 profile image
Alexander

I'll try, I'm a bit of a newbie to this

Thread Thread
 
onlycoder000 profile image
Only Coder

Same just learning new things .

Collapse
 
nelsonfigueroa profile image
Nelson Figueroa

This is pretty fun to play around with, thanks for sharing!

Collapse
 
ashusharmatech profile image
Ashutosh Sharma

This is very interesting, I will give it a try and post the results.

Collapse
 
notyourmomtoday profile image
Erica

Can you make it search your Google Drive?

Collapse
 
andypiper profile image
Andy Piper

This code relies on local files in one of a set of understood formats (text, markdown, etc). You could extend it to pull down your Google Drive content, or sync it to a local directory first.

Collapse
 
aryansingh3 profile image
ARYAN SINGH

ive read somewhere that it only works on CPU becasue of llama cpp , cant we run it in gpu linux environment

Some comments have been hidden by the post's author - find out more