Robot Notes: What are Trainable Parameters in AI Language Models?

Imagine you're teaching a robot to recognize and draw animals. Each time you show it a picture, it tries to remember and learn something new about animals. In this case, "trainable parameters" are like notes the robot takes to remember and get better at recognizing animals.

Now, how many notes does it take? That depends on how complex your robot is. If it's a simple one, it might need just a few notes. But a super complex robot, like those used for understanding and talking in human language, needs a lot of notes (millions or even billions) to remember all the details.

In large language robots (LLMs, like GPT-3), these notes help them understand and use language just like humans. The more notes it has, the better it gets at talking and answering questions in a way that makes sense.