As a noob, these are questions I want answers for.
- A lot of knowledge is hierarchical. Why aren’t embeddings designed around this fact?
- The closest we seem to get is hyperbolic embeddings (1). But was this not the default? (2)
- To read: https://gwern.net/tree-embedding
- The most popular way to query embeddings seems really dumb. Why go through all that trouble, and then try to find chunks that are similar to the question?!
- Why isn’t there more emphasis on retrieval? At least in 2025, we saw Grok 3 (which pulls from Twitter) and various flavors of Deep Research (which pulls from the web)…