ChatCraft Adventures #8

This week in ChatCraft

Release 1.4

After a short hiatus, ChatCraft release 1.4 has been released and is available here:
This week has been slow for me. I've been stuck on my Pull Requests described in my previous blog post, and I haven't been able to contribute PR reviews or find new issues. I'm hoping to have these resolved quickly before they go stale, so I can devote more time to larger issues and features.

Pull Requests

Prompt Updating

In this Pull Request, I've updated the ChatCraft system prompt to return inline math in KaTeX, like it currently does with math markdown in its own block:

- When responding with math, inline or otherwise, I ALWAYS use KaTeX syntax in math Markdown by enclosing EVERY mathematical expression, equation, variable, and formula with double-dollar signs \`($$)\`, for example: $$O(n\\log n)$$, $$1024 * 1024 = 1048576$$, $$1024^2$$, $$X$$

This updated is meant to solve the issue of ChatCraft returning inline math expressions in LaTeX Math mode syntax:
(like \(2^{20}\) instead of $$2^{20}$$).

With the new prompt, the GPT-3.5-turbo LLM seems to prefer returning inline math markdown as KaTeX, as seen in this conversation.

However, before this can be merged, we're worried that my updated prompt might instruct the LLM to use KaTeX syntax when returning code, which we don't want. We want the prompt to instruct the LLM to use KaTeX only when returning math markdown, and not to mix KaTeX syntax when returning code. I've tried to mitigate this by including the clause "When responding with math..." to the rule, but whether an LLM will properly follow these instructions can be hard to test. I'm hoping to receive feedback from my instructor or even the open source community.

Until working on this issue I never really considered using ChatCraft or ChatGPT for math, but I see the utility now.
I found that the updated prompt can address in-line Math Markdown, but it can fail when asking ChatCraft to describe scientific formulas, like the gas laws or the Michaelis-Menten formula.

In these conversations, ChatCraft fails to return individual variables in KaTeX syntax:

I originally had a second rule that fixes this issue:

- I NEVER wrap mathematical expressions, equations, variables, or formulas in round brackets, \`(\`, \`)\`. For example, instead of responding with (V_{max}), I would respond with $$V_{max}$$

...but I removed it because I was worried about introducing new ways for the LLM to give improper responses.
I'll need to clarify with the community whether this rule should be added back.

Dynamic Imports

This Pull Request involves dynamically importing a few large dependencies to reduce bundle size:

• elkjs (for Mermaid)
• cytoscape (for Mermaid)
• katex (for Math rendering)

In an older PR, I tried dynamically importing the modules that use these dependencies:

• mermaid
• rehype-katex
• remark-math

... but I found that dynamically importing rehype-katex and remark-math crashes my preview app due to a workbox error:

In the new PR, I was able to dynamically import mermaid, but the problem is mermaid is almost always needed, so dynamically importing it doesn't help much. elkjs and cytoscape are never directly called in the source code (at least according to git grep), and are only invoked through mermaid (as dependencies). I'm not sure if there's another way to dynamically import only these dependencies of mermaid.

I'll need some help dynamically importing rehype-katex and remark-math.

Plans for Release 1.5

In release 1.5, I hope to have these PRs revised and merged.
I'm actually thinking about postponing these PRs further so I can focus on a new feature I'd like to add, adding files to a ChatCraft conversation.