Working Checklists for the NIST AI RMF: A Practical Tips Series: MAP

MAP

Part 2 of 4 — GOVERN, MAP, MEASURE, MANAGE

If GOVERN is the structure that’s supposed to make everything else real, MAP is where most of that structure gets tested for the first time because MAP happens before deployment, when it’s still cheap to be wrong. The failure mode here isn’t skipping MAP. Almost everyone does something they’d call “risk assessment” before shipping an AI system. The failure mode is doing it shallow: a checkbox form filled out by the team that built the system, reviewed by nobody who’d catch what they missed, filed, and never revisited once the system meets the real world.

MAP exists to answer one question honestly before you commit resources: do you understand what this system does, who it affects, and what could go wrong? Here’s the checklist, organized by NIST’s five MAP categories.

MAP 1: Context Is Established and Understood

• The intended purpose of the AI system is written down in specific terms. This is not “improves efficiency” but the actual decision or output it produces
• Legal, regulatory, and normative context specific to this use case has been identified. This is not your general AI policy, but what applies to this system in this jurisdiction and sector
• The review team includes people from outside the building team who understands where this system will be used
• Organizational risk tolerance for this specific use case has been stated by someone with the authority to state it
• Requirements were gathered from people who will use or be affected by the system, not just from the internal sponsor who wants it built

Theater vs. real:Theater is a use case form where “risk tolerance” is a dropdown nobody thought about. Real is a documented decision that this system’s risk level required a specific sign-off, and that sign-off happened before build, not after.

MAP 2: The System Is Categorized Honestly

• The system’s proximity to high-stakes decisions has been explicitly assessed, not assumed away because “it’s just a recommendation”
• The system’s known limitations are documented somewhere a downstream user would see them, not buried in a model card nobody reads
• If this system is high-risk under any applicable framework (EU AI Act Annex III, sector-specific rules, your own internal tiering), that classification has been made explicitly and isn’t just going to get discovered during an audit

Theater vs. real:Theater is categorizing every internal tool as “low risk” by default because the form makes low risk the easiest box to check. Real is a categorization that occasionally surprises the team that built it.

MAP 3: Benefits, Costs, and Scope Are Actually Compared

• Expected benefits are documented in terms specific enough to later verify
• Costs are documented too, including the ones that aren’t financial
• The scope of intended use is written down narrowly enough that “scope creep” is a detectable event, not a vague feeling six months later
• There’s a defined process for evaluating whether the people operating or overseeing this system are proficient enough to do so
• Human oversight is designed, not assumed. If a human is supposed to be “in the loop,” their actual authority, workload, and incentive to actually intervene have all been considered

Theater vs. real:Theater is a “human in the loop” checkbox where the human reviews 400 outputs an hour and has no realistic ability to catch anything. Real is oversight designed around what a human can do in the time and attention they’ll have.

MAP 4: Third-Party Risk Is Mapped, Not Assumed Away

• Risks and benefits introduced by third-party models, data, or components are explicitly mapped for this use case and not covered by a generic vendor questionnaire filled out once at procurement
• Internal controls exist for the specific third-party dependency this system has

Theater vs. real:Theater is “we reviewed the vendor’s SOC 2” as the entirety of third-party risk mapping for an AI-specific dependency. Real is knowing exactly what breaks in your system if the vendor’s model output distribution shifts.

MAP 5: Impacts to People Are Actually Mapped

• Likely impacts to individuals, groups, and communities affected by this system are identified before deployment, not discovered through a complaint
• The mapping prioritizes the impacts that are both probable and severe
• Someone outside the immediate build team has asked, specifically, “who is most likely to be harmed by this if it fails, and how would we know?”

Theater vs. real:Theater is an impact assessment template with a “potential harms” field left blank because nobody wanted to slow down the launch. Real is a launch date that moved because the impact mapping surfaced something real.

The test that cuts across all five

MAP fails quietly. Nothing breaks the day you skip it. The cost shows up later, disguised as an incident, a regulatory inquiry, or a use case that has to be unwound in production instead of redesigned on paper. The organizations that get MAP right aren’t the ones with the most thorough forms. They’re the ones where the form changed at least one decision before launch.

Next in this series: MEASURE — the checklist for proving your AI system does what you claim, instead of asserting it.