Methodology
How AILOS measures AI leadership capability.
The 7-minute diagnostic is a structured instrument. It measures five vectors in three passes — what you say, what you've actually done, and what you'd do. This page documents what it measures, how it scores, the research it draws from, and the current state of validation.
For the short version, download the one-page methodology PDF.
01 — The instrument
A 20-item, 5-vector instrument in three parts.
Self-report alone measures confidence as much as capability. Parts 2 and 3 exist to correct for that: one asks what actually happened, the other asks what you would do.
Part 1
Self-report
10 items · 2 per vector
5-point agreement scale with anchored verbal labels. Produces the stated score.
Part 2
Evidence
5 items · 1 per vector
Concrete multiple-choice questions about the last 90 days. No typing — pick the closest true answer.
Part 3
Scenario judgment
5 items · 1 per vector
Realistic situations with four plausible responses. The choice reveals the managerial reflex.
- Length~7 minutes median completion time across all three parts. Parts 2 and 3 are skippable; skipping returns a self-reported-only result.
- OutputPer-vector score (1–5), an aggregate maturity score, a maturity stage (Exploring → Adopting → Integrating → Compounding), a 12-week roadmap, and a first decision modeler prescribed against each of your two weakest vectors.
- ScoringStated score = mean of the two Part 1 items. Evidenced score = blend of that vector's Part 2 and Part 3 items. The reported score is the stated score adjusted toward the evidenced score, clamped to a maximum shift of ±0.5 points so a single item can't dominate.
- Confidence gapWhere stated exceeds evidenced by more than 1.0 point, the result flags a confidence gap on that vector. This is surfaced as information for development planning, not as a penalty — it is usually the most actionable line on the report.
- Cut-pointsStage thresholds are fixed, not relative to a cohort, so an individual's stage is stable as the sample grows.
- ProgressAnswers are saved in the browser as you go; the assessment can be left and resumed without losing responses.
02 — Construct definitions
The five vectors.
Literacy
Knowing what it can and can't doWhat it is. How well the leader — and the people doing the work — understand AI's real capabilities, limits, and failure modes well enough to catch a wrong answer.
What it isn't. Not tool knowledge or prompt fluency.
Example item: “I can explain to my team, in plain words, where AI helps us and where it tends to get things wrong.”
Workflow Integration
In the work, not beside itWhat it is. Whether AI is embedded in how the team actually runs its repeat work — with named integration points — versus sitting on the side as an experiment a few curious people run.
What it isn't. Not pilots, sandboxes, or licence counts.
Example item: “In the last three months, I've personally looked at our repeat work and decided where AI should take some of it.”
Trust & Verification
Neither rubber-stamping nor dismissingWhat it is. How rigorously output gets checked before it carries weight — an explicit review gate, and a willingness to name it publicly when AI was wrong.
What it isn't. Not blanket suspicion, and not blind acceptance.
Example item: “Nothing AI-written goes to a client until a real person on my team has read it carefully.”
Psychological Safety
Can people admit AI helped?What it is. Whether people can say “I used AI for this” — or “I'm struggling with it” — without losing perceived contribution or quietly working around the leader.
What it isn't. Not general team morale.
Example item: “People on my team feel comfortable saying 'I used AI for this' without it lowering their perceived contribution.”
Compounding
Wins that get reusedWhat it is. Whether prompts, patterns, and lessons get captured and reused across the team — so capability accumulates instead of resetting with each person.
What it isn't. Not a tool library nobody opens.
Example item: “When something works with AI, it gets written down somewhere the rest of the team will actually find it.”
03 — Research basis
Where the vectors come from.
The vectors are derived from published research on AI in management, knowledge work, and organizational adoption. The instrument synthesizes these frames; it is not an endorsement by, or a publication of, the institutions named.
- Oxford — Future of Work program — Frames AI as a complement-vs-substitute decision at task level, not job level.
- MIT Sloan / MIT CSAIL — AI & Business strategy — Distinction between AI-augmented decision making and AI-automated execution.
- Stanford HAI — Human-Centered AI — Responsible deployment patterns and human-in-the-loop review.
- Harvard Business School — Generative AI & knowledge work — Effect of AI on individual vs team performance and the “jagged frontier.”
- World Economic Forum — Future of Jobs — Skill-shift forecasts that shape the team-adoption vector.
04 — Validation status
Honest current state.
"Grounded in research" is a basis claim. Validation is a separate, ongoing process. We publish status, not marketing.
Content validity
Items reviewed against the published research base above and pre-tested with practitioner readers.
Status: complete
Internal consistency (Cronbach's α)
Per-vector reliability measured on the design-partner sample. Target α ≥ 0.70 per vector.
Status: pilot in progress — figures published when N is sufficient
Test–retest stability
Re-take cadence built into the product (quarterly). Stability target: r ≥ 0.75 over 30 days for leaders without intervention.
Status: pilot in progress
Construct validity
Confirmatory factor analysis on the 5-vector structure once enrolled-cohort N is sufficient.
Status: scheduled
Updated values are published on this page as design-partner data crosses each threshold. We do not publish reliability figures derived from an insufficient sample.
05 — Limitations & responsible use
What this instrument is not for.
- • It remains a leader-reported instrument. Parts 2 and 3 ground the self-rating in recalled behavior and stated judgment, which reduces — but does not eliminate — the gap between perception and observed performance. It is not a 360 and does not collect input from the leader's team.
- • If Parts 2 and 3 are skipped, the result is self-reported only and is labelled as such. Confidence-gap flags are unavailable in that case.
- • It is a leading indicator of AI leadership capability — useful for development planning, cohort design, and 12-week roadmaps. It is not appropriate as a performance-review or hiring instrument.
- • Scores are stable across cohorts (fixed cut-points), so an individual's score is not affected by who else takes the diagnostic.
- • Re-take cadence is quarterly. Re-taking more often will not meaningfully reflect capability change.
- • Data handling, retention, and team-rollout privacy are documented on the Trust page.