The problem
The central constraint: language models are allowed to classify passages against a closed nine-mechanism vocabulary and verify quotes verbatim, but every numerical score comes from deterministic additive rules over curated modifier cells. This keeps the output reproducible and lets each number be traced to its source passage.
The system
What I built
- Scored risks along velocity, impact and confidence, with provenance recorded at the level of the individual modifier cell.
- Built a deterministic additive engine over a set of curated modifier cells so scores are reproducible run to run.
- Enforced an architecture where LLMs classify against a closed vocabulary and verify verbatim quotes, and never emit a score.
- Automated ingestion from multiple sources with source-level permissions and a four-gate admission test.
- Added an audited LLM-as-a-judge second opinion that annotates but cannot override the deterministic result.
Stack
PythonFastAPISQLAlchemyAlembicPostgreSQLSupabaseHS256 JWTAPSchedulerhttpxPlaywrightpdfplumberPyPDFBeautifulSoupGoogle GeminiGroqOpenRouterTavilyPytestReactViteTailwind CSSshadcn/ui
Skills applied
- Deterministic scoring model design
- LLM boundary enforcement
- Data provenance and auditability
- Web scraping and document parsing
- Scheduled ingestion workers
- Automated testing
- Technical methodology documentation