Questions we’re in the middle of.
An open notebook. What we're testing, what's unresolved, and what we still don't know. Everything here is in progress.
- Exploring
- 4
- Building
- 2
- Testing
- 1
7 entries
- BuildingInfrastructureJun 2026
Attributing cost through nested agent runs
When an agent calls a tool that calls another model, spend belongs to a tree rather than a flat list of API calls. Working through how scope should nest, what happens when a run is abandoned partway, and how attribution survives a single event loop interleaving many runs.
- Observability
- Agents
- SDK Design
- contextvars
- TestingEvaluationMay 2026
How much of RAG quality is decided at chunk time
Most weak answers in a retrieval system trace back to retrieval, not generation. Comparing fixed-size, overlapping and structure-aware splitting on the same corpus and question set, to see how much of the gap a better prompt can actually recover. Early impression is: much less than expected.
- RAG
- Embeddings
- Evaluation
- ChromaDB
- ExploringAI SystemsApr 2026
Can an extraction model be trusted to say it's unsure?
Human-in-the-loop systems route on confidence, so a miscalibrated confidence signal quietly breaks the entire safety argument. Looking at whether self-reported certainty, output-distribution signals or agreement across samples give a threshold worth routing on.
- Calibration
- Extraction
- Human-in-the-Loop
- Reliability
- ExploringAI SystemsMar 2026
Working inside a small model's context budget
Running inference locally removes per-query cost and keeps data in place, but replaces those with a hard constraint: much less context and much less tolerance for a loose prompt. Investigating how far careful retrieval and context packing close the gap against a hosted frontier model.
- Local Inference
- Ollama
- Context Design
- Gemma
- ExploringMachine LearningFeb 2026
Separating seasonal swing from real vegetation growth
NDVI rises every growing season whether or not anything was planted, and the seasonal amplitude is large enough to swamp a multi-year trend. Testing decomposition approaches on irregularly sampled series, since cloud cover means observations arrive whenever they arrive.
- Time Series
- Geospatial
- NDVI
- Signal Processing
- ExploringNotesJan 2026
A working taxonomy of LLM system failure modes
Collecting the ways these systems actually break in practice — retrieval that returns plausible but wrong passages, extraction that is confidently malformed, silent truncation, cost that scales with a retry loop nobody noticed. Notes toward a checklist worth running before shipping.
- Reliability
- Evaluation
- Systems Design
- BuildingInfrastructureNov 2025
When expensive work should leave the request
Notes on the boundary between work that belongs in a request and work that belongs in a queue — and on the interface consequence, since asynchronous work means the user is looking at a job rather than a result, and pending state has to be represented honestly.
- Queues
- BullMQ
- System Design
- UX
Entries without a link are working notes — there’s no writeup to read yet. When one exists, the entry links to it and its status changes to Published.
Have a difficult problem worth building?
We’re interested in ambitious software, AI systems, research collaborations and real problems where careful engineering actually changes the outcome.