Imagine a 2D map of the United States. Now, imagine I have a function that takes a place name and provides the lat/long of that place on the map. So, I pass "Miami, FL" and "Ft. Lauderdale, FL" and I get coordinates whose distances are closer than "New York City" and "London". And using this function I could implement functionality on top of it -- I could give an estimate of driving time for closer locations (for example) -- it would obviously break down for some places, but it would be good enough for some applications.
The goals of models like GPTx are to define a coordinate system (many many dimensions), such that text that maps to similar concepts are closer together. And the relative distance maps to relative closeness and direction and combinations seem to also map as they would on a physical map. It seems to preserve "stylistic" attributes -- closeness in style have close coordinates.
Once you have a function that can produce a coordinate like this, you can build applications on top of it. Like the driving time application above, sometimes it works well, and sometimes it breaks down.
Imagine a 2D map of the United States. Now, imagine I have a function that takes a place name and provides the lat/long of that place on the map. So, I pass "Miami, FL" and "Ft. Lauderdale, FL" and I get coordinates whose distances are closer than "New York City" and "London". And using this function I could implement functionality on top of it -- I could give an estimate of driving time for closer locations (for example) -- it would obviously break down for some places, but it would be good enough for some applications.
The goals of models like GPTx are to define a coordinate system (many many dimensions), such that text that maps to similar concepts are closer together. And the relative distance maps to relative closeness and direction and combinations seem to also map as they would on a physical map. It seems to preserve "stylistic" attributes -- closeness in style have close coordinates.
Once you have a function that can produce a coordinate like this, you can build applications on top of it. Like the driving time application above, sometimes it works well, and sometimes it breaks down.
You have never met a 5 year old have you?