How to automate Azure Policy evidence for cATO

1. Establish the control map and Policy coverage

Start with the cATO in Azure implementation workflow. For each selected requirement, list expected resources, applicable technical checks, supporting runtime/manual evidence, control owners, and review conditions. Use the Azure Policy regulatory initiative as an input to that map, not as a claim that every requirement is implemented.

Export the assignment and referenced definitions/initiative, including effective versions where available, parameters, assignment scope, notScopes, selectors, overrides, enforcement mode, and exemptions. In an initiative, retain the policyDefinitionReferenceId; the same policy definition can appear more than once with different parameters. Inventory inherited management-group assignments and resource exclusions so a subscription-only list does not hide the effective configuration.

Check resource type, alias, evaluation mode, and supported cloud/API behavior. A definition cannot observe every guest OS or data-plane condition through ARM. Use machine configuration or service-specific checks where appropriate, and mark unsupported or manually assessed coverage explicitly.

2. Test a narrow audit rule before broad enforcement

The following custom policy rule checks whether a Storage account’s ARM publicNetworkAccess setting is exactly Disabled. Save it as storage-public-network-rule.json when following the IaC guide. Wrap it in a custom policy definition using mode: Indexed. It is not an official regulatory initiative or a complete network-control test.

{
  "if": {
    "allOf": [
      {
        "field": "type",
        "equals": "Microsoft.Storage/storageAccounts"
      },
      {
        "field": "Microsoft.Storage/storageAccounts/publicNetworkAccess",
        "notEquals": "Disabled"
      }
    ]
  },
  "then": {
    "effect": "audit"
  }
}

The rule audits configuration; it does not disable public access or create a private endpoint. It also flags SecuredByPerimeter because this example’s requirement specifically calls for Disabled. If the approved architecture uses Network Security Perimeter or another access design, review a rule appropriate to that requirement instead of calling the alternative universally noncompliant.

In a disposable approved test scope, create or select fixtures for Disabled, Enabled, and any supported alternate/default state. Verify the expected policy results and the actual application network behavior. Do not disable public access on a production account before validating its private routes, DNS, and required service access.

3. Trigger evaluation and preserve the assignment configuration

After approved authentication, choose the correct cloud and subscription. This example selects Azure Government; use AzureCloud for an approved commercial deployment. Capture the assignment scope separately from the subscription being evaluated, since an assignment may be inherited from a management group.

set -euo pipefail
umask 077
: "${APPROVED_SUBSCRIPTION_ID:?Set the approved subscription}"
: "${APPROVED_ASSIGNMENT_NAME:?Set the assignment name}"
: "${APPROVED_ASSIGNMENT_SCOPE:?Set its actual ARM scope}"
: "${EVIDENCE_METADATA_DIR:?Use a new private metadata directory}"

az cloud set --name AzureUSGovernment
az account set --subscription "$APPROVED_SUBSCRIPTION_ID"
mkdir "$EVIDENCE_METADATA_DIR"
az policy assignment show \
  --name "$APPROVED_ASSIGNMENT_NAME" \
  --scope "$APPROVED_ASSIGNMENT_SCOPE" \
  --expand LatestDefinitionVersion,EffectiveDefinitionVersion \
  --output json > "$EVIDENCE_METADATA_DIR/assignment.json"

# Starts an evaluation; it does not deploy or remediate resources.
# Omitting --no-wait waits for the scan operation to complete.
az policy state trigger-scan --subscription "$APPROVED_SUBSCRIPTION_ID"

Record scan request/start/completion and collection times. Microsoft documents a daily standard evaluation plus create/update and on-demand triggers, with no universal completion-time guarantee. A completed scan does not establish support for every resource provider, freshness of every row, or completeness of your control coverage. Apply an explicit timeout and a freshness/coverage check before accepting results.

Use a scoped evidence-reading identity and appropriate permission for triggering evaluation. Deployment or remediation permissions are unnecessary for a read-only export. Verify that version expansion and APIs are supported in the selected cloud and CLI version; if a required field cannot be collected, report a collection gap instead of fabricating it.

4. Retrieve all result pages without discarding evidence

Policy Insights latest/queryResults uses a POST query and can return @odata.nextLink. Follow each continuation URL, save raw pages in a protected per-run directory, and reject a partial or failed retrieval. Avoid a dashboard’s top-N view or filtering only for compliant/noncompliant rows; those views can hide gaps and exemptions.

This Python example uses an already authenticated Azure CLI. Set APPROVED_SUBSCRIPTION_ID, the full APPROVED_ASSIGNMENT_ID, and a new EVIDENCE_RUN_DIR. It discovers the current cloud’s ARM endpoint and retrieves one assignment’s latest policy-state pages. It collects evidence only; it does not make an authorization or gate decision.

import json
import os
from pathlib import Path
import subprocess
from urllib.parse import urlencode, urlsplit
from uuid import UUID

def az(*args):
    result = subprocess.run(
        ["az", *args, "--output", "json", "--only-show-errors"],
        check=True, capture_output=True, text=True, timeout=180,
    )
    return json.loads(result.stdout)

subscription = str(UUID(os.environ["APPROVED_SUBSCRIPTION_ID"]))
assignment = os.environ["APPROVED_ASSIGNMENT_ID"]
cloud = az("cloud", "show")
arm = cloud["endpoints"]["resourceManager"].rstrip("/")
origin = urlsplit(arm)
query = urlencode({
    "api-version": "2019-10-01",
    "$filter": "PolicyAssignmentId eq '" + assignment.replace("'", "''") + "'",
})
url = (f"{arm}/subscriptions/{subscription}/providers/"
       f"Microsoft.PolicyInsights/policyStates/latest/queryResults?{query}")

os.umask(0o077)
output = Path(os.environ["EVIDENCE_RUN_DIR"])
output.mkdir(parents=True, exist_ok=False)
(output / "cloud.json").write_text(json.dumps(cloud, indent=2))
seen, count = set(), 0
while url:
    parsed = urlsplit(url)
    if (parsed.scheme != "https" or parsed.netloc != origin.netloc
            or url in seen):
        raise RuntimeError("Unexpected or repeated policy pagination URL")
    seen.add(url)
    page = az("rest", "--method", "post", "--url", url)
    if not isinstance(page, dict) or not isinstance(page.get("value"), list):
        raise RuntimeError("Policy response is missing its record array")
    (output / f"policy-page-{len(seen):04d}.json").write_text(
        json.dumps(page, indent=2)
    )
    count += len(page["value"])
    url = page.get("@odata.nextLink")
if count == 0:
    raise RuntimeError("No policy rows: verify scope, coverage, and evaluation")
print(f"Retrieved {count} records; freshness and coverage still need validation")

The collector deliberately fails on CLI errors, malformed responses, repeated continuation links, and zero result rows. Raw files may remain after a failure; mark the run failed and do not publish it as a completed evidence package. A production collector also needs bounded retry/backoff for transient API errors, a job timeout, metrics, assignment/definition/exemption exports, resource inventory, and a completion manifest written only after all required checks succeed.

For the Azure CLI, az policy state list --all requests historical states within the specified interval instead of latest states. It is not a promise of complete resource coverage or a substitute for correctly handling a raw REST response’s continuation link. Use an explicit time window and retain evaluation timestamps when building history.

5. Validate freshness, coverage, and result semantics

Join observed state records to the expected resource/check inventory, using resource ID, assignment ID, initiative reference ID, and policy definition identity/version where available. Normalize resource IDs for comparison while preserving the original values in evidence. Compare row evaluation timestamps with the control’s freshness requirement and relevant deployment time; the collector’s current timestamp cannot refresh an old evaluation.

Do not treat an assignment’s DoNotEnforce mode as missing evaluation: it can still produce compliance results while suppressing enforcement. Conversely, a Disabled effect does not demonstrate the intended check. Preserve both the rule effect and enforcement mode. A resource absent from the evaluated set needs an applicability or collection explanation, not an automatic pass.

6. Handle exemptions, attestations, and remediation separately

Azure Policy exemptions can count toward overall compliance even when the resource cannot be evaluated or has a waiver. Export their Waiver or Mitigated category, assignment/reference scope, metadata, and expiresOn where set. Map the exemption to the program’s approval record; creating an exemption does not itself establish risk acceptance. Alert on missing/expired review conditions.

Manual-policy attestations can attach metadata and evidence links to a stated compliance result. Preserve the attestor, scope, supporting artifact, and review/expiry information. An attestation is a claim with supporting evidence, not a newly executed technical test.

For Modify or deployIfNotExists, explicitly provision the assignment’s managed identity and least-privilege role assignments when using IaC. The portal’s automatic role behavior should not be assumed for your Terraform/Bicep workflow. Existing resources generally need a remediation task; record task status and failures, then reevaluate and test the resource. Keep operationally risky remediation behind the applicable change approval.

7. Package the evidence for review and repeat collection

Store the raw pages, normalized results, expected resource coverage, assignment/definition snapshots, exemptions, relevant attestations, scan metadata, collector version, cloud/subscription context, and reviewer disposition. Give the package a stable run ID and include artifact digests, but protect the storage and provenance as well: a hash alone does not authenticate who generated the artifact.

Run the workflow after relevant deployments and on the agreed operational schedule. Test a missing permission, stale row, empty result, expired waiver, and deliberate configuration change before relying on the collector. Continue with the Terraform/Bicep pipeline guide to link this observed evidence to a reviewed infrastructure change.

Sources and technical review

Public documentation reviewed October 13, 2026. Recheck the applicable program requirements and the selected Azure cloud, API, and tool versions before implementation.

• Microsoft mapping of DoD cATO activities 3.5.1 and 3.5.2
• Azure Policy overview and supported evaluation modes
• FedRAMP shared responsibilities and the limits of Policy compliance
• Policy compliance data, evaluation triggers, and scan completion
• Policy assignment structure, effective versions, and enforcement mode
• Azure CLI policy state commands, including --all
• Policy Insights subscription query and continuation responses
• Policy exemptions and their compliance treatment
• Manual-policy attestations and evidence links
• Remediation identities and role assignments

The examples illustrate an engineering workflow. No Azure deployment, provider authorization-package review, agency risk acceptance, or cATO approval was performed for this guide.