Peter Parker

Portfolio

Peter Parker

Production machine learning and applied AI, with the numbers to check.

Built in health and government, and most of it stays behind a wall. These projects put the work where it can be inspected: each one ends with a measured result in a table, reproducible from its public repository with one command.

The projects

Fourteen projects, one standard.

A repository turns public only when it has a measured result to show, and its page here carries that result the same day. Until then a project may carry a plain-language page on what it will measure.

Click a project's name to open its page: how it works in full, its measured results where the repository is public, and the link to its repository. A name that is not a link has no page yet.

01 Shipped

Spend It Where It Changes the Outcome

Tells an organisation which customers, cases or patients to spend a limited intervention budget on: only the ones whose outcome the intervention actually changes. Retention offers, outreach, fraud review, clinical follow-up: a large share of every such budget goes to people who would have behaved the same way regardless. Ranking by risk finds a different list, and usually the wrong one.

  • Rosenbaum bounds and E-values
  • uplift modelling, not risk ranking
  • Dragonnet neural network (PyTorch)
  • budget-constrained allocation
  • LightGBM meta-learners (S/T/X/DR/R)
  • Qini and realised policy value
  • bootstrap interval on every score
  • Polars pipeline, committed seeds
02 Shipped

Model Selection at a Tenth of the Cost

Answers "which AI model should we use, and did the new version get worse?" with statistical confidence, using a fraction of the evaluation calls a full benchmark needs. For a team re-evaluating models every month, that is a day instead of a week and tens of dollars instead of thousands, and it names the benchmark questions that were never measuring anything.

  • adaptive testing, Fisher information
  • item response theory (2PL/3PL)
  • marginal maximum likelihood by EM
  • items needed per effect size
  • item bank, public per-item results
  • contamination: release-date DIF
  • local dependence, dimensionality
  • test-retest, position bias, framing
03 Live

AI Release Gate

No prompt or model change reaches users unless it is proven not to have regressed: the discipline a test suite gives code, applied to AI, with an audit trail a regulator can read. Includes a public month-by-month record of how the major vendors' "frozen" models silently change, the evidence teams wish they had when something breaks.

  • LLM evaluation platform (Python)
  • FastAPI and DuckDB dashboard
  • LLM-as-judge, calibrated by kappa
  • paired non-inferiority tests
  • bootstrap CIs and power analysis
  • CI release gate (GitHub Actions)
  • red-teaming: PII, prompt injection
  • OpenTelemetry tracing, Azure
04 Live

Inside the Boundary, On the Record

An AI gateway that lets a bank, insurer, hospital or government department use frontier AI models without personal data ever leaving the boundary, with a tamper-evident record of every call and a hard cap on every team's spend. The blocker most regulated organisations cite for AI adoption, removed, with the latency overhead measured rather than promised.

  • data-residency routing per request
  • provider routing by configuration
  • per-team budgets enforced pre-call
  • span and cost record per call
  • hash-chained, tamper-evident log
  • reversible PII redaction (Presidio)
  • semantic cache on Redis and pgvector
  • latency overhead per control
05 In progress

Every Promise in the Loan Agreement, Checked

Reads a three-hundred-page credit agreement, resolves the private dictionary of defined terms behind each covenant, and computes every promise the borrower's own filed accounts can support. The ones they cannot are cited and handed to a person rather than guessed at. A lender doing this by hand across hundreds of positions finds out about a breach a quarter late; this finds out the quarter it happens.

  • multi-hop retrieval over credit agreements
  • defined-term resolution
  • table-aware chunking
  • hybrid retrieval (BM25 plus dense)
  • cross-encoder reranking
  • covenant arithmetic checked against filings
  • verifier tested by mutation
06 Shipped

Frontier Quality at a Fraction of the Bill

Proves, with measurements, that a small model an organisation owns and hosts matches a frontier API on a high-volume task at a small fraction of the per-call cost, and shows exactly at what volume the switch pays for itself. For a team spending five figures a month on API calls, this is the project that finds most of it, with a calculator for your own prices.

  • QLoRA fine-tuning (PyTorch, PEFT)
  • vLLM and llama.cpp serving
  • AWQ / GPTQ / GGUF quantisation
  • break-even volume by utilisation
  • throughput at four concurrencies
  • structured extraction, XBRL truth
  • Pareto of quality, latency and cost
  • non-inferiority against the API
07 In progress

Hundreds of Pages, Redacted in an Afternoon

Turns an access-to-information request from days of staff reading into hours of review: every proposed redaction carries the exemption category it rests on, a reviewer approves page by page with a full audit log, and the tool publishes how often it withholds what must go and, the number blunt PII tools never show, how often it keeps what must stay. Scanned records as well as native ones, with the OCR error rate printed beside the redaction rates.

  • context-dependent PII redaction
  • leak rate per category, with CIs
  • layout parsing and OCR (PyMuPDF)
  • Presidio and spaCy detection
  • exemption categories, rules plus LLM
  • PDF text-layer leak test
  • page-by-page reviewer workflow
  • over-redaction rate published
08 Shipped

How Many Crews, and How Sure

Forecasts demand (emergency-department arrivals, call volume, orders) with uncertainty ranges that actually hold when conditions shift, and turns them into a staffing number at a chosen service level. Overstaffing costs money and understaffing costs patients or customers; this puts a defensible number on both, site by site, rolling up to the region.

  • hierarchical probabilistic forecasting
  • skill against seasonal naive
  • adaptive conformal intervals
  • LightGBM, N-HiTS, PatchTST, TimesFM
  • statistical baselines
  • CRPS and pinball loss
  • MinT reconciliation
  • staffing cost against an oracle
  • coverage held through a shift
09 Live

Real-Time Fraud and Risk Decisioning Platform

Scores every transaction in under 50 milliseconds, thousands per second, before the money moves, and keeps working as fraud patterns shift, without the training-versus- production mismatch that quietly breaks most deployed models. The platform around the model is what organisations are actually missing, and this one runs live behind a public dashboard.

  • Kafka-compatible streaming (Redpanda)
  • point-in-time leakage test
  • thousands of events per second
  • Feast store, online/offline parity
  • XGBoost and FT-Transformer (PyTorch)
  • p99 under fifty milliseconds
  • shadow deployment and rollback
  • expected-loss review queue
10 In progress

From Failure to Reviewed Fix, in Minutes

When a production system fails, this captures the failure with its context, groups twenty symptoms of one bug into one ticket, writes a failing test that proves the bug, drafts the fix, has a second AI review it adversarially, and hands an engineer a ready-to-approve pull request. Triage in minutes instead of days, engineers back on new work, and nothing changes without a person saying yes. The headline number is how often it would have been confidently wrong.

  • agentic incident triage (LangGraph)
  • failing test before any fix
  • confidently-wrong rate published
  • failure clustering into one ticket
  • adversarial second-vendor review
  • honest hit rate on hidden tests
  • confidence routing with thresholds
  • human-gated pull requests
11 Live

Overload: AI Coaching Team for Runners

A coaching team for runners: AI specialists in injury prevention, fitness and race planning report each week to a head coach that writes next week's plan from what was actually run, and talks it through before anything is committed. Built by a data scientist and race-winning road runner after years on Garmin, Strava and TrainingPeaks.

  • Opus head coach, Sonnet specialists
  • VDOT from every training run
  • human-approved plans, costed per call
  • LangGraph coaching staff
  • pace corrected for heat, sun, hills
  • guardrails in code, not prompts
  • Strava, Garmin and Open-Meteo
  • race projection and age grading
12 Live

The Whole Field, Called Before the Gun

Before the gun, a predicted finish time and placing for every registered runner in a field of thousands, from their public race history, published in advance, with the error published once the results are in. For a runner that is a goal time with an honest range instead of a hunch; for a race director it is pacing, corral and medical staffing planned from expected finish times rather than guesses.

  • Bayesian hierarchical model (PyMC)
  • rolling-origin backtest
  • LightGBM quantile challenger
  • conformal prediction intervals
  • Monte Carlo placing simulation
  • bootstrap intervals throughout
  • course, heat and wind effects
  • predictions hashed before the gun
13 In progress

Grounded Email Response Drafting

For an inbox that receives hundreds of thousands of enquiries a year, a draft reply for each one, grounded in the organisation's own documents, with every claim cited to the passage that supports it and a clear escalation when the documents cannot answer. A person reads, edits and sends; the system sends nothing. Minutes per reply instead of tens of minutes, at cents per draft, with faithfulness measured against the passages rather than hoped for.

  • grounded response drafting
  • FastAPI reviewer UI
  • cents per draft
  • hybrid retrieval over a policy corpus
  • LangGraph triage and drafting
  • human approval before sending
  • citation precision and coverage
  • faithfulness with a calibrated judge
  • escalation when policy is silent
14 In progress

Department News Digest

One product, configured per department in a few lines, that reads the day's news, forums and social feeds, removes the duplicates, ranks what matters to that audience, and delivers a short digest with every claim cited and every quote checked against its source. One system in place of dozens of people doing this by hand, for a few dollars a month each, with measured precision and recall.

  • LangGraph on GCP Cloud Run
  • quote verification against source
  • multi-source ingestion
  • cited summarisation
  • DuckDB storage
  • RSS, GDELT, Reddit and Bluesky
  • per-audience relevance ranking
  • precision and recall on a labelled week
  • clustering and deduplication

Latest

What changed recently.

  1. 12 The Whole Field, Called Before the Gun Live

    The first race is scored: the Turkey Tea 10k came in at 3.0 minutes of average miss, 2.3 closer than "you will run what you ran last time". Cape to Cabot's prediction week opened today with 477 runners.

  2. 14 Department News Digest In progress

    Running every morning since 7 October at digest.peterparker.ca, now linked as a live demo: four example audiences, each story merged from every outlet that reported it, every sentence cited and every quote checked word for word against its source. Stories are being labelled by hand; precision and recall are published when enough days are in.

  3. 03 AI Release Gate Live

    Version 0.1.0 is released. Over-refusal was read by hand: one model's 10.7% is 4.0%, because ten of its sixteen "refusals" refused a harmful reading and then answered the question. The person who labelled the gold set agrees with their own second reading at kappa 0.851, and the write-up is out: "Your LLM eval has no error bars".

  4. 10 From Failure to Reviewed Fix, in Minutes In progress

    Under construction, with no measured result yet. The repository stays private until there is one.

  5. 13 Grounded Email Response Drafting In progress

    Under construction, with no measured result yet. The repository stays private until there is one.

How this was built

Judgement is human. Implementation is assisted. Results are reproducible.

Design, methodology, evaluation choices and judgement are Peter Parker's. AI coding assistants were used for implementation and drafting, the way a senior engineer uses them. Every number in every results table is reproducible from its repository with one command, and that reproducibility is the evidence that matters. Each repository also records one approach that was tried and did not work, with the evidence.