The oracle pipeline

design · draft · NLP and argumentation · edited 2026-11-06

Family B on the arms page. Source: .docs/designs/pta-atip-algorithm.typ.

The procedure is mostly rules. Every judgement a rule can’t make is an oracle: a function that answers one small question with a probability for each answer. Each oracle has arms (a rule baseline, a small model, a larger one), compared on gold one oracle at a time. It’s reliable and easy to debug, but each specialised model costs compute and work.

flowchart TD
  A["text"] --> S1["1 · segment into locutions"]
  S1 --> S2["2 · relate: TA, YA, RA/CA/MA"]
  S2 --> S3["3 · reconstruct canonical statements"]
  S3 --> S4["4 · analyse statements"]
  S4 --> S5["5 · type: form + substance (rules), lever"]
  S5 --> S6["6 · tag concepts"]
  S6 --> R["review in the xAIF editor"]
  click S1 href "#s1"
  click S2 href "#s2"
  click S3 href "#s3"
  click S4 href "#s4"
  click S5 href "#s5"
  click S6 href "#s6"

The order follows how a human annotator works. Each step’s arms:

step	decides	arms
1 · segment	which spans are argumentative units (L-nodes)	parser + discourse markers; GLiNER2 spans; small LLM; oAMF segmenters
2 · relate	which unit answers which; argue, disagree or restate; reported speech; unstated premises	marker rules; NLI over pairs; LLM proposal checked by NLI
3 · reconstruct	each I-node’s wording, which decides the form. The hardest step.	O2 same term? (lemma + coreference, embeddings, cross-encoder); O9 rewrite proposer (generative); O10 rewrite check (NLI both ways)
4 · analyse	subject/predicate (O1), F/V/P (O3), s/p/u (O4), second-order + A/U (O5), perspective (O6)	rules (modals, quantifiers, reporting verbs) first; then GLiNER2 or small classifiers
5 · type	form and substance by rule (`deriveCode`); γ’s i/d (O7); the lever (O8)	cue words; NLI over each candidate lever written as a sentence
6 · concepts	what each argument is about	reading-notes lab tiers: authored ranges, embeddings, GLiNER2

Two rules from the judgement-call log:

If step 5 lands on novel, send the argument back to step 3. novel usually means a missed premise, a wrong statement type or a wrong form.
“A claim about a kind is universal.” “F if you can check it by looking, V if it applies a standard.”

Runtimes

Lab models run through ONNX Runtime, so the game can run the same model through ort in Rust. Generative arms run through llama.cpp.

What links here: Arms and decisions, Phases, Spike: natural language → xAIF, Quell debate: plan map, Argument maps, Cross-encoders, Dependency parsing, Natural language inference, ONNX and ort, Stage-wise evaluation, oAMF