Command Palette

Search for a command to run...

AI Change Impact Studio

live

AI-assisted change management platform: ADKAR-scored impact assessments, RAG over uploaded documents with independently verified citations, and a labelled evaluation suite that measures retrieval, citation checking and document comparison.

Readiness heat map
Technologies & Frameworks
PythonFastAPIReactGeminiRAGPydanticEvaluationSQLAlchemy

Given a described business process change, the studio generates a Prosci ADKAR impact assessment, a Red/Amber/Green readiness heat map, a RACI matrix, a RAID log, a communication plan and a training matrix, exportable to Word and PowerPoint. Retrieval over uploaded process documents grounds the answers, and every citation is checked against the cited document before it is labelled verified.

The latest update adds an evaluation suite: 12 documents, 42 labelled questions and 74 labelled citations, with the held-out test split run once under an adoption rule fixed in advance. TF-IDF retrieval was compared with BM25, stemming and Gemini embeddings, and no alternative was reliably better, so retrieval was left unchanged.

The evaluation also exposed four defects in citation verification, all now fixed and covered by regression tests. One was a tolerant matcher that accepted quotes with altered numbers, such as "$75" cited as "$85"; numbers now have to match exactly. On the labelled cases the current verifier scores precision and recall of 1.00, up from a recall of 0.60.

For document comparison, passing the full documents to the model on small projects raised the recall of known changes from 0.52 to 0.93 across 15 live runs per setting, at the cost of roughly 2.3 seconds of extra latency.

ADKAR impact assessment
ADKAR impact assessment
Communication plan
Communication plan
Document comparison with citation verification
Document comparison with citation verification