qproj wikiEmbeddings and cosine similarity
An embedding model turns a piece of text into a vector, a list of a few hundred numbers. It is trained so that texts with similar meanings produce vectors pointing in similar directions. “The eye has a purpose” and “every organ has a function” end up close together. “The river is cold” ends up somewhere else.
Comparing texts
The usual measure of closeness is cosine similarity, the cosine of the angle between two vectors. A value of 1 means they point the same way, and values near 0 mean the texts are unrelated. Each text is embedded once. After that, comparing two texts is just a dot product, so searching thousands of candidates is cheap.
Models that embed each text separately, as here, are called bi-encoders. They are fast but somewhat coarse. A cross-encoder reads both texts together and is usually more accurate, at a higher cost.
Small models
Sentence-embedding models such as MiniLM, BGE-small and E5-small are 20–130 MB and run comfortably on a CPU. That makes them practical inside a game that cannot rely on a strong GPU. They can be exported to ONNX and run from Rust.
In Quell
Embeddings are planned for three jobs:
- Interpreting notes: comparing what a player writes with descriptions of each concept, to work out which concepts a note is about. This is tier 1 of Reading notes.
- Retrieving examples: choosing the solved examples most similar to a new passage, for few-shot prompting.
- Checking terms: testing whether two phrasings in an argument mean the same thing.
External links
- Sentence Transformers — a widely used library and model collection for sentence embeddings.
- Cosine similarity — the measure itself.
What links here: Cross-encoders, In-context learning, ONNX and ort