Topic: Running models

Part of NLP and argumentation.

This topic contains 2 articles.

• ONNX and ort — A portable format for trained models, and the Rust library that lets the game run them without Python.
• Quantization and llama.cpp — Storing a model’s weights in fewer bits so that it fits in memory and runs on a CPU.