Select context while keeping required facts

Select chunks without editing the transcript

The lab proposes keep or omit_from_candidate for each chunk. The original transcript stays intact. This makes it possible to compare the shorter selection with the full conversation and recover anything missing.

Run the local example

Download the ZIP above, unzip it and open a terminal in that folder. You need Python 3.10 or later; there are no packages to install. dry-run writes request payloads. demo reads the supplied, hand-written responses and runs the application logic. Use a new output folder for each run.

python jev_lab.py --mode dry-run --out requests
python jev_lab.py --mode demo --out demo-run

Mark the facts you must keep

The goal and an unresolved commitment have protected flags. The example deliberately gives them low model values: your application should keep them anyway. Apply the same rule to system instructions and other required context. Reassemble selected chunks in conversation order.

Include the task in each question

A chunk can matter for one task and be irrelevant to another. Send the current task and the exact chunk together, retaining a stable ID. If you truncate input before the request, inspect what was actually sent; a model cannot judge text that was removed.

Check which of the five chunks survive

The supplied answers keep the goal, order ID, threshold-boundary example and commitment. Only the weather chunk is omitted. The protected goal and commitment remain despite low values. Inspect results.json to confirm which chunks were kept by a rule.

keep = sample.get('protected', False) or answer['noul'] >= cutoff
# Never edit the original transcript in this step.
# Reconstruct selected chunks in original order.

Replay a completed task with both contexts

Pick a conversation whose outcome you know. Run the downstream task once with full context and once with the selection, keeping model settings fixed. Check missing facts, wrong actions and completion before comparing token counts. A smaller prompt that loses an order ID is not an improvement.

Try your own chunks

Use the samples.json structure for your data, including protected flags. Set TYPESAFE_API_KEY and replace 0.8 below with a cutoff chosen on validation conversations. The starter requires --calibrated for this live Noul task; that flag does not perform calibration. Only --high affects this keep/omit policy.

python jev_lab.py --mode live --samples own-chunks.json \
  --high 0.8 --calibrated --out live-context
# Replace 0.8 with a threshold selected on validation transcripts.

Keep full context when selection fails

A timeout, missing chunk or invalid answer must not silently drop information. Fall back to the original context or an explicit review path. Retain both the original and selected versions while evaluating the change.