A practical third-party security review methodology

Third-party review sits in an awkward place. The request is usually urgent, the evidence belongs to someone else, and the people who understand the business need are rarely the same people who understand the technical risk. A methodology gives those groups a shared route to a decision. The governance and risk-management outcomes in the NIST Cybersecurity Framework 2.0 support that separation of evidence, risk, ownership, and action.

The eight-stage manual workflow

1. Intake. Capture requester, business owner, product, version or service, intended use, timeline, users, integrations, and decision authority.
2. Triage. Rate inherent risk using data sensitivity, privilege, exposure, code execution, criticality, user population, regulatory obligations, and switching difficulty. Select a proportionate review depth.
3. Choose the evidence path. Use source analysis for code, extensions, packages, repositories, and authorized archives. Use vendor evidence for operated SaaS and commercial services. Use both when both are available.
4. Collect and verify evidence. Confirm identity, version, source, scope, dates, integrity, and authority. Inventory missing or unavailable evidence rather than silently treating it as covered.
5. Analyze. For source, evaluate vulnerabilities, dependencies, authentication and authorization, data protection, injection, cryptography, errors, configuration, code quality, third-party components, provenance, artifact consistency, and licensing. For vendors, evaluate configured requirements using audit reports, testing, policies, architecture, questionnaires, contracts, and responses.
6. Resolve gaps. Ask targeted follow-up questions, document potential mitigating controls, and distinguish confirmed facts, reasonable inferences, open questions, and human risk acceptance.
7. Decide and implement. Approve, conditionally approve, hold pending evidence, or deny. Record configurations, restrictions, owners, deadlines, evidence citations, and residual risk.
8. Reassess. Trigger review by time, major version, ownership change, incident, architecture change, new data use, new privilege, or material control failure.

Evidence rules that keep the method honest

• Cite the page, response, file, or code location behind material conclusions.
• Record evidence dates and the system or version to which it applies.
• Use explicit states such as meets, partial, does not meet, pending, accepted, partial coverage, and unavailable.
• Do not convert missing evidence into a pass.
• State limitations, false-positive and false-negative risk, and where qualified human review is required.
• Keep source findings and vendor-control evidence distinct even when they appear in the same decision.

This evidence discipline draws from the assessment procedures in NIST SP 800-53A, the supplier due-diligence questions in NIST SP 1326, and the supply-chain practices in NIST SP 800-161 Rev. 1.

Treat traceability as a deliverable

A reviewer should be able to move in both directions through the record. Starting with a requirement, the reviewer should find the evidence, the assessment state, any gap, the resulting condition, and the owner. Starting with a source document or code finding, the reviewer should see which conclusion it affected. That chain is what makes a report reviewable. It also exposes situations where several conclusions rest on one vague marketing statement or where a severe finding never reached the recommendation.

Record the issuer, scope, period or version, and storage location for each evidence item. For source, preserve the analyzed artifact, commit or version, package manifest, available lockfile, and code references. For vendor evidence, preserve the product and service boundary, audit period, testing scope, document date, and relevant page citations. NIST SP 800-53A emphasizes evidence that is sufficient to support assessment findings. This does not mean collecting every document a supplier can produce. It means being able to explain why the selected evidence is relevant and where it stops.

A defensible shortcut is still documented

Small teams cannot perform a deep review for every low-impact tool. Establish tiers and exit criteria. A low-risk review may rely on a brief evidence set and managed deployment. A high-risk review may require source analysis, current audit evidence, penetration testing, contract changes, and executive risk acceptance. The methodology stays consistent even when depth changes.

A worked example: two requests, one method

The first request is a small source-readable formatter that converts plain text into a local document. It runs on managed laptops, has no network permission, receives no credentials, and can be removed without affecting a critical process. The second request is an identity vendor that will authenticate the entire workforce, receive employee attributes, integrate with the directory, and become a dependency for access to production systems.

The formatter still needs an intake record. Confirm the exact package or extension, publisher, version, source-artifact relationship, permissions, data handling, dependencies, known advisories, suspicious behavior, licensing, and deployment controls. The review may stop after a focused source and package assessment if the evidence supports the low-impact use. A managed allowlist, pinned version, restricted permissions, and reassessment on major version change may be proportionate conditions. NIST SP 800-161 Rev. 1 provides supply-chain practices that support examining provenance and supplier-related risk even when the component is small.

The identity vendor follows the same stages but requires deeper evidence. The reviewer maps authentication, privileged access, availability, incident response, logging, encryption, change management, subcontractors, recovery, and notification requirements to current evidence. A SOC 2 report can support controls within its scope and period. Penetration-test material can support specific technical testing. Contracts and product configuration establish obligations and the controls the customer must enable. Missing evidence remains pending, and contradictions become follow-up questions rather than being averaged away.

Business context changes the decision. The formatter has limited impact because it is local, replaceable, and restricted. The identity service is a concentration of privilege and availability risk. Even strong vendor evidence may lead to conditions such as phishing-resistant administrator authentication, tested break-glass access, exported logs, recovery exercises, contractual incident timing, and an exit plan. NIST IR 8286 Rev. 1 connects cybersecurity risk information with enterprise objectives and risk decisions, which is the reason the same technical observation can matter differently in two deployments.

The method is consistent because both requests preserve intake, triage, evidence, analysis, gaps, decision, ownership, and reassessment. The effort is proportionate because the evidence depth and approval authority match the potential impact. Tiering should reduce unnecessary work. It should never erase the decision record.

Review the review

Before approval, ask whether the evidence actually supports the recommendation, whether severe findings and deal breakers are visible, whether scope and limitations are clear, whether conditions are enforceable, and whether a skeptical reviewer could reconstruct the reasoning. A polished report is not a substitute for a sound chain of evidence.

Use workbooks as decision records

Choose the workbook that matches the evidence. For SaaS reviews, Download workbook. For browser extensions, Download workbook. For source-readable software, Download workbook. Keep the completed file with its cited evidence. The workbooks provide structure, but they do not replace qualified technical, risk, privacy, procurement, or legal review.

Close the loop on follow-up evidence

A follow-up question needs the same structure as a finding. State the unresolved requirement, cite the evidence that created the question, ask for a specific artifact or explanation, assign an owner, and set a due date. When the answer arrives, preserve it and record the reviewer's disposition. A response can close the gap, reduce it, contradict earlier evidence, or create a new question.

Do not silently replace the original status. If a requirement moved from pending to meets, the record should show what new evidence caused the change and who accepted it. If a vendor answer remains an unsupported assertion, label it that way. This history matters when a later reviewer asks why the software was approved or when the next assessment receives different evidence.

Finally, verify approval conditions after deployment. Configuration evidence, an executed contract amendment, a completed access review, or a tested recovery procedure should be attached to the decision record. If a deadline passes without evidence, route the exception to the named decision owner. An approval condition that nobody checks is only optimistic prose.

Sources and further reading

• NIST Cybersecurity Framework 2.0
• NIST SP 800-161 Rev. 1, Cybersecurity Supply Chain Risk Management Practices
• NIST SP 1326, Cybersecurity Supply Chain Risk Management: Due Diligence Assessment Quick-Start Guide
• NIST SP 800-53A Rev. 5, Assessing Security and Privacy Controls
• NIST SP 800-115, Technical Guide to Information Security Testing and Assessment
• NIST IR 8286 Rev. 1, Integrating Cybersecurity and Enterprise Risk Management

Managing reviews across teams

Can I Run That? can organize software analysis, vendor evidence, requirement mapping, citations, gaps, reassessment, and decision history across teams. Your organization still supplies business context, validates material findings, and owns the final decision.

The method should make disagreement useful

A good methodology does not force every reviewer to agree. It makes the evidence, assumptions, and risk tolerance visible enough to explain why they disagree. That is a much stronger control than a checkbox that always turns green.