Skip to content
Lab

Questions we’re in the middle of.

An open notebook. What we're testing, what's unresolved, and what we still don't know. Everything here is in progress.

Exploring
4
Building
2
Testing
1
Category

7 entries

  • BuildingInfrastructureJun 2026

    Attributing cost through nested agent runs

    When an agent calls a tool that calls another model, spend belongs to a tree rather than a flat list of API calls. Working through how scope should nest, what happens when a run is abandoned partway, and how attribution survives a single event loop interleaving many runs.

    • Observability
    • Agents
    • SDK Design
    • contextvars
  • TestingEvaluationMay 2026

    How much of RAG quality is decided at chunk time

    Most weak answers in a retrieval system trace back to retrieval, not generation. Comparing fixed-size, overlapping and structure-aware splitting on the same corpus and question set, to see how much of the gap a better prompt can actually recover. Early impression is: much less than expected.

    • RAG
    • Embeddings
    • Evaluation
    • ChromaDB
  • ExploringAI SystemsApr 2026

    Can an extraction model be trusted to say it's unsure?

    Human-in-the-loop systems route on confidence, so a miscalibrated confidence signal quietly breaks the entire safety argument. Looking at whether self-reported certainty, output-distribution signals or agreement across samples give a threshold worth routing on.

    • Calibration
    • Extraction
    • Human-in-the-Loop
    • Reliability
  • ExploringAI SystemsMar 2026

    Working inside a small model's context budget

    Running inference locally removes per-query cost and keeps data in place, but replaces those with a hard constraint: much less context and much less tolerance for a loose prompt. Investigating how far careful retrieval and context packing close the gap against a hosted frontier model.

    • Local Inference
    • Ollama
    • Context Design
    • Gemma
  • ExploringMachine LearningFeb 2026

    Separating seasonal swing from real vegetation growth

    NDVI rises every growing season whether or not anything was planted, and the seasonal amplitude is large enough to swamp a multi-year trend. Testing decomposition approaches on irregularly sampled series, since cloud cover means observations arrive whenever they arrive.

    • Time Series
    • Geospatial
    • NDVI
    • Signal Processing
  • ExploringNotesJan 2026

    A working taxonomy of LLM system failure modes

    Collecting the ways these systems actually break in practice — retrieval that returns plausible but wrong passages, extraction that is confidently malformed, silent truncation, cost that scales with a retry loop nobody noticed. Notes toward a checklist worth running before shipping.

    • Reliability
    • Evaluation
    • Systems Design
  • BuildingInfrastructureNov 2025

    When expensive work should leave the request

    Notes on the boundary between work that belongs in a request and work that belongs in a queue — and on the interface consequence, since asynchronous work means the user is looking at a job rather than a result, and pending state has to be represented honestly.

    • Queues
    • BullMQ
    • System Design
    • UX

Entries without a link are working notes — there’s no writeup to read yet. When one exists, the entry links to it and its status changes to Published.