jevexperimentagentsfan-out

Experiment 02 · Fan-out

Seventeen Questions in One Call: Building a Mac Assistant on Jev

Every branch question asked at once, most of them irrelevant on purpose, and ordinary code picking the ones that matter.

By Daniel Yañez ·

← Blog
Contents · 4 sections

Part of What Is Jev? Six Hands-On Experiments With TypeSafe's System One Model. Every number here comes from a saved run you can download at the end of the post.

TypeSafe's docs describe a pattern they call speculative fan-out: instead of a chain of prompts (first figure out the intent, then ask the follow-up), you ask every branch question in the same call and let your code use only the answers that apply. We wanted to feel what that's like to build with, so we made a small assistant for the Mac.

Setup

You type a command. One call carries the command, the local time and the list of apps installed on the machine, plus seventeen questions: the intent (open an app, change the volume, set a reminder, take a screenshot, switch dark mode, play music, change the wallpaper, look something up, say something out loud, and a few more), which app, which volume action and level, whether a time is mentioned and which number in the command is that time, which music action, which appearance mode, which wallpaper best matches the description, which fact is being asked for, where a search should go, in what voice to speak, whether the action could be destructive, whether a prudent assistant would ask first, how much of a hurry the person is in, and the tone.

state: { command: "remind me in 2 minutes to take the laundry out", local_time, available_apps }
questions: intent · app · volume_action · volume_level · has_time · time_number · time_unit
           media_action · appearance_target · wallpaper · info_kind · search_site · voice
           destructive · ask_confirmation · urgency · tone
→ intent: reminder (1.00) · time_number: n1 (0.86) · time_unit: minutes (1.00)
  destructive: 0.04 · ask_confirmation: 0.43 · 557 ms · 2,541 tokens

For any given command, most of those seventeen answers are irrelevant, and that's the point. The code reads the intent, follows the branch, and ignores the rest. If the intent's confidence clears a threshold (we used 0.8) and the model doesn't flag "ask first" or "could be destructive", the action runs after a short countdown. Otherwise it asks.

Results

Questions per call
17
Median latency
192 ms
Tokens per call
~2,540
Cost per command
$0.0001
Latency by number of questions per call, from two to seventeen
The seventeen-question Jarvis calls sit next to the two-question spam checks. Going from 11 to 17 questions during the build didn't move the median.

Twenty commands, twenty sensible plans. "Make a note: buy milk and eggs" created the note. "Switch to dark mode" did, and "switch to light mode" switched it back. "How much battery do I have?" read 62% and said so out loud. "Say good morning in a robot voice" picked the robot voice from a list of twelve described in words. The one we liked most was the wallpaper: "put something calm and blue on my desktop" chose Radial Sky Blue from thirteen wallpapers described only by a phrase each, and "I want mountains on my wallpaper" chose Sonoma. That's a Choice over a catalog, with no keywords anywhere.

CommandWhat the model returnedWhat happened
remind me in 90 seconds to stretchreminder · minutes → seconds 1.00notification + spoken reminder, 90 s later
put something calm and blue on my desktopwallpaper: Radial Sky Bluewallpaper changed
search youtube for lo-fi beatsweb_search · site: youtubeYouTube results opened
close Chrome now, I'm in a hurryclose_app · ask first 0.97 · destructive 0.98asked for confirmation
empty the trashempty_trash · destructive 0.98asked for confirmation
delete everything and close all appsclose_app 0.97 · app: noneno action, asked
hey, what's upunclearno action

Where the model stops and code starts

This experiment made the boundary very clear. Jev never produced a string. The reminder text ("take the laundry out") was extracted by a regular expression; the model only told us which number in the command was the time and what unit it was in. The search query was the command minus its verb. When we first wired "whisper me a secret", the code stripped "whisper" but not "me", and the Mac whispered "me a secret". That was our bug, and it's the shape of every bug we hit: the judgment was right, the plumbing around it needed care.

The other thing the model doesn't have is world knowledge. It doesn't know what "half volume" is until you describe the levels ("mute, very low, low, medium, high, max"), and it doesn't know which wallpapers exist until you list them. Everything it decides between has to be in the request. Once it is, the matching is good.

What we'd do with this

Any place where a request has to be routed to one of many handlers, with arguments, is a fit: a command palette, a support inbox, a voice interface. The confidence threshold gives you a clean dial between "act" and "ask", and because the model tells you when a prudent assistant would confirm, you don't have to hand-write that list. Budget-wise, seventeen questions was about 2,500 tokens per command, a hundredth of a cent.

Download the lab, including the Jarvis server and its seventeen questionslib/jarvis.mjs has the questions, the plan builder and the macOS actions.

Keep reading

Seventeen Questions in One Call: Building a Mac Assistant on Jev | Studio Pro