TL;DR
Learn how to use Polyglot Notebooks and Kernel Memory together to enhance your day-to-day tasks with the power of AI.
- TL;DR
- Motivation
- Example - βGenerate a summary of a blogβ
-
Anatomy of the Setup (aka
notebook-agent
) - Conclusion
- References
Source code: https://github.com/NikiforovAll/notebook-agent
Motivation
Polyglot Notebooks is a powerful tool for developers. They allow you to combine code, text, and visualizations in a single document. This makes it easy to explore data, experiment with different tasks, and share your results with others.
π‘ Notebooks are a great way to bring programmability to your work, allowing you to use code to automate repetitive tasks, analyze data, and generate reports. AI capabilities are a great addition to that.
π‘ The more you invest in notebooks, the easier it becomes for you to be productive. Your experiments are stored over time, constituting a knowledge base that you can refer to at any moment.
π― For this blog post, my goal is to show you how to work with LLMs in a notebook environment. It would be great to be able to use external sources as a context for my prompts to support my day-to-day activities.
Example - βGenerate a summary of a blogβ
Letβs say you have a blog post that you want to summarize. You can use the LLM to generate a summary of the blog post. Hereβs how you can do it:
- Crawl the blog post and extract the text from the HTML content.
- Use the LLM to generate a summary of the blog post.
It turns out that the Kernel Memory takes care of it for you. All the heavy lifting is done by the Kernel Memory
. You just need to provide the context and the prompt.
Here is the code that shows you how to do it:
var docId = await memory.ImportWebPageAsync("https://nikiforovall.github.io/tags.html");
var answer = await memory.AskAsync("What the nikiforovall blog is about?", filter: MemoryFilters.ByDocument(docId));
Demo
Here is the demo:
Output:
The nikiforovall blog, titled "N+1 Blog," focuses on topics related to programming and IT. It features a variety of posts that cover different aspects of software development, particularly within the .NET ecosystem. The blog includes discussions on modern API development, asynchronous programming, microservices, cloud computing, and tools like Keycloak for identity management. Additionally, it explores design patterns, coding stories, and practical coding techniques, making it a resource for developers looking to enhance their skills and knowledge in these areas.
π€ But, how do I get an instance of memory
you may ask? Letβs see how to do it using Testcontainers
Anatomy of the Setup (aka notebook-agent
)
I have created a project called notebook-agent
. Basically, it is a starting point that has all the necessary bits to get you started.
See: https://github.com/NikiforovAll/notebook-agent
Here is the structure of the project:
.
βββ .devcontainer
β βββ devcontainer.json
βββ .gitignore
βββ .vscode
β βββ extensions.json
βββ README.md
βββ src
βββ OpenAiOptions.cs - Options for OpenAI
βββ ServiceDefaults.cs
βββ appsettings.json - Configuration file
βββ playground.ipynb - entry point (example)
βββ ask-release-notes.ipynb - (example)
βββ setup-infrastructure.ipynb - PostgreSQL, pgvector
βββ setup-kernel.ipynb - Compose Kernel Memory
We have a few notebooks that will help you get started. Letβs start from the infrastructure bit: setup-infrastructure.ipynb
// setup-infrastructure.ipynb
using Testcontainers.PostgreSql;
using Npgsql;
var db = new PostgreSqlBuilder()
.WithImage("pgvector/pgvector:pg16")
.WithPortBinding(5432, 5432)
.WithDatabase("memory-db")
.WithUsername("postgres")
.WithPassword("postgres")
.WithReuse(true)
.WithVolumeMount("memory-db-volume", "/var/lib/postgresql/data")
.Build();
await db.StartAsync();
var connectionString = db.GetConnectionString();
In the code above, we are using Testcontainers
to start a PostgreSQL instance with the pgvector
extension. The pgvector
extension is used to store the embeddings of the documents. The embeddings are used to search for the most relevant documents based on the context.
Once the infrastructure is ready, letβs see how to configure the Kernel Memory
: setup-kernel.ipynb
// setup-kernel.ipynb
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Configuration;
using Microsoft.KernelMemory;
using Microsoft.KernelMemory.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using Microsoft.SemanticKernel.Connectors.OpenAI;
HostApplicationBuilder builder = Host.CreateApplicationBuilder();
var textGenerationOpetions = builder.Configuration // configured in appsettings.json
.GetSection("TextGeneration")
.Get<OpenAiOptions>();
var textEmbeddingOptions = builder.Configuration // configured in appsettings.json
.GetSection("TextEmbeddingGeneration")
.Get<OpenAiOptions>();
builder.Services.AddKernelMemory<MemoryServerless>(memoryBuilder =>
{
memoryBuilder
.WithPostgresMemoryDb(new()
{
ConnectionString = builder.Configuration.GetConnectionString("memory-db")! // connection string to the PostgreSQL
}
)
.WithSemanticKernelTextGenerationService( // text completion
new AzureOpenAIChatCompletionService(
deploymentName: textGenerationOpetions.Deployment,
endpoint: textGenerationOpetions.Endpoint,
apiKey: textGenerationOpetions.ApiKey
),
new SemanticKernelConfig()
)
.WithSemanticKernelTextEmbeddingGenerationService( // text embedding
new AzureOpenAITextEmbeddingGenerationService(
deploymentName: textEmbeddingOptions.Deployment,
endpoint: textEmbeddingOptions.Endpoint,
apiKey: textEmbeddingOptions.ApiKey
),
new SemanticKernelConfig()
);
});
IHost host = builder.Build();
IKernelMemory memory = host.Services.GetRequiredService<IKernelMemory>();
IServiceProvider services = host.Services;
π Now, we have an the instance Kernel Memory named - memory
. Just run the import command from a notebook: #!import ./setup-kernel.ipynb
Another Example - βAsk .NET 9 Release Notesβ
For example, assume we want to be able to chat with LLM about the latest release of .NET 9. Here is how to do it:
- Create a new notebook:
ask-release-notes.ipynb
- Import the
setup-kernel.ipynb
notebook - Index of the release notes
- Ask the question
1οΈβ£ Index release notes:
var tags = new TagCollection() { ["release"] = [".NET 9","RC1"] }; // tagging provides additional information, improves search results
var librariesReleaseNotes = "https://raw.githubusercontent.com/dotnet/core/main/release-notes/9.0/preview/rc1/libraries.md";
var aspnetCoreReleaseNotes = "https://raw.githubusercontent.com/dotnet/core/main/release-notes/9.0/preview/rc1/libraries.md";
Task[] tasks = [
memory.ImportWebPageAsync(librariesReleaseNotes, tags: tags),
memory.ImportWebPageAsync(aspnetCoreReleaseNotes, tags: tags),
];
await Task.WhenAll(tasks);
2οΈβ£ Ask the question:
var answer = await memory.AskAsync("What are the latest additions to .NET 9 release?", minRelevance: 0.80);
answer.Result.DisplayAs("text/markdown");
Here is the output:
Conclusion
In this post, we have seen how to use Polyglot Notebooks and Kernel Memory together to empower your day-to-day tasks with the power of AI. We have seen how to set up the infrastructure using Testcontainers
and how to configure the Kernel Memory
to use the LLM.
π I hope you found it helpful. If you have any questions, please feel free to reach out. If youβd like to support my work, a star on GitHub would be greatly appreciated! π
Top comments (0)