Contents · 3 sections
Part of What Is Jev? Six Hands-On Experiments With TypeSafe's System One Model. Every number here comes from a saved run you can download at the end of the post.
This replicates one of TypeSafe's own cookbooks (they use GitHub's terms of service) on a document of our choosing. The idea is almost too simple: a Choice question can have up to 255 options, so give it every line of a document as the options and ask which line answers the question. No index, no embeddings, no retrieval step.
Setup
We used the Apache License 2.0, which has 169 non-empty lines. Each call carries the question and the whole numbered document in the state, a Choice over the 169 line ids with the line text as each option's description, a Noul asking whether the document answers the question at all, and a Score for how completely the winning line answers on its own.
state: { question, document: "L2: Apache License\nL3: Version 2.0, January 2004\n…" }
questions:
line: Choice L2 · L3 · … · L202 (169 options, each described by its text)
answered: Noul "Does the document contain an answer at all?"
completeness: Score ["not at all", "partially", "fully"]
→ ~7,700 tokens per question · $0.0003Results
| Question | Best line | P(line) | Answered? |
|---|---|---|---|
| Do I have to include a copy of the license when I redistribute? | L96 "Derivative Works a copy of this License; and" | 0.96 | 0.99 |
| What happens to my patent license if I sue someone over patents? | L87 "granted to You under this License for that Work shall terminate" | 0.70 | 0.94 |
| Is there any warranty? | L147 "WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND" | 0.48 | 0.98 |
| Can I use this in a commercial product? | L78 "use, offer to sell, sell, import, and otherwise transfer the Work" | 0.47 | 0.95 |
| What is the capital of France? | L2 "Apache License" | 0.35 | 0.01 |
The clear questions got a clear line. The fuzzier ones ("is there a warranty?", "can I use this commercially?") got the right *passage* with the probability spread across its lines, which is exactly what the 0.47 means: the answer lives in several lines, and the model is telling you so instead of pretending one line has it all. The last row is the one we like most. A Choice has to choose, so it picked the title, but the Noul said the document doesn't answer the question with 99% certainty. That separate "is there an answer at all?" question is what makes the search trustworthy.
What we'd do with this
For a document under a few hundred lines, this is a complete search feature in one API call: no pipeline to maintain, and the results explain themselves. Longer documents can be searched in windows of 250 lines. In our own studio, the same shape solves "find the asset they mean": a person describes a picture in a sentence, the candidate files are the options, the model picks, and a Noul says whether anything actually matches. That's the next thing we're wiring.
