at least if it was “vectors in a high-dimensional space” it would be like. at least a little bit accurate to the internals of llm’s. (still an entirely irrelevant implementation detail that adds noise to the conversation, but accurate.)
That’s literally what a neural network is… It’s a graph, a bunch of nodes and the edges that connect them, and weights on those edges. It’s not three dimensional, it’s n-dimensional, where n is based on the size of the network
We can click on one of these points (“iceland”) to see its nearest neighbors in the high-dimensional space (mostly other countries!) as well as other points that belong to the same cluster (Cluster 18 is this red cluster).
for people without a technical background: this is gibberish
at least if it was “vectors in a high-dimensional space” it would be like. at least a little bit accurate to the internals of llm’s. (still an entirely irrelevant implementation detail that adds noise to the conversation, but accurate.)
for people with a technical background, it’s 120d8 psychic damage
That’s literally what a neural network is… It’s a graph, a bunch of nodes and the edges that connect them, and weights on those edges. It’s not three dimensional, it’s n-dimensional, where n is based on the size of the network
Here’s a tool that visualizes it