The framework in motion
Seven phases, one responsible thread.
FRAME maps onto the seven phases of an evaluation. Each phase has a way of deciding whether AI is appropriate, protocols for using it well, quality checks, and documentation requirements — with governance running throughout.
phase one
Design
Establishing foundations for responsible AI use
Decisions made here shape whether AI will strengthen or undermine the whole evaluation. Before choosing any tool, assess AI appropriateness across five dimensions.
What FRAME asks here
- Contextual sensitivity: is AI culturally appropriate and politically safe for these communities and topics?
- Data characteristics: is the data compatible with the tools (quality, volume, structure)?
- Stakeholder expectations: do commissioners and participants welcome or distrust AI involvement?
- Capacity assessment: does the team have the literacy, time, and resources to use AI responsibly?
- Risk-benefit analysis: do the benefits justify the risks and costs?
phase two
Structuring & Planning
Integrating AI into evaluation methodology
Position AI explicitly within the methodology rather than bolting it on. Theorise how AI use contributes to the evaluation’s theory of change.
What FRAME asks here
- Specify, for each AI application: the task, the data, the human validation role, performance criteria, and the fallback if AI proves inadequate.
- Set quality-assurance protocols: compare AI and human coding on samples, run calibration checks, keep audit trails, and review for bias.
- Design the workflow for “hybrid intelligence”: preserve human authority while leveraging AI through clear task allocation.
phase three
Data Collection
AI-assisted collection with participant protections
AI in data collection raises consent and transparency requirements beyond standard evaluation ethics.
What FRAME asks here
- Enhance informed consent: participants should know when AI processes their responses and may request human-only processing where feasible.
- When AI interacts directly with participants, they should know they are engaging with AI.
- Apply data-quality protocols: input validation, monitoring during collection, and ground-truthing AI interpretations against field observation.
phase four
Analysis
Maintaining human–AI collaboration with validation
AI-assisted analysis needs specific safeguards built around structured prompting and multi-stage validation.
What FRAME asks here
- Develop and document structured prompts to keep analysis consistent and reproducible.
- Validate iteratively: compare AI and human coding on samples, spot-check during analysis, and review AI themes against source data.
- Manage hallucination: verify every claim against source, cross-check systematically, and prompt AI to express uncertainty where it can.
phase five
Judgment
Preserving human evaluative authority
This is where FRAME is most emphatic: evaluative judgment must remain human. This is a fundamental feature of evaluation, not a temporary limit awaiting better algorithms.
What FRAME asks here
- Keep with humans: weighing evidence against values, contextual interpretation, ethical deliberation, professional accountability, and democratic legitimacy.
- Let AI support judgment by synthesising large evidence bases, flagging inconsistencies or gaps, and offering alternative interpretations for human consideration.
- Determining whether a pattern is evaluatively significant always requires human judgment about values, context, and consequences.
phase six
Reporting
Transparent communication with appropriate disclosure
Counter “shadow AI” with explicit disclosure through three mechanisms.
What FRAME asks here
- Process disclosure: describe how AI was used, what it did, and how its outputs were validated.
- Limitation acknowledgment: note the potential for hallucination, bias, and context-dependency.
- Verification statement: confirm that AI-assisted content has been checked by human reviewers who take responsibility for accuracy.
phase seven
Utilization
Supporting uptake while managing limitations
AI can extend the reach of findings through innovative dissemination. FRAME encourages this within three guardrails.
What FRAME asks here
- Accuracy preservation: simplification for new formats must not become distortion.
- Audience-appropriate disclosure: adapt how AI involvement is disclosed to the audience's needs and literacy.
- Feedback integration: give audiences a way to flag concerns about AI-generated content, with a process to investigate and correct.
Running throughout
Governance holds every phase together
Across all seven phases, cross-cutting governance keeps AI use monitored, accountable, and responsive — not just set up once and forgotten.
Ready to apply FRAME to your next evaluation?
Download the FRAME checklist, or enrol in the AI for MEL course to put the framework into practice.