You can now type “audit my AWS account” into an AI assistant and have it actually inspect your cloud for security problems. No dashboards, no scripts, no console — just a sentence. The catch, discovered in a real test this week, is that a subtle bug made the assistant report a perfectly clean account while serious problems sat right there in the data. Here’s what happened, and how it got fixed in NSAuditor AI’s 0.16.4 release.

Plain-language cloud audits, running on your own machine

NSAuditor AI is a security scanner that checks cloud accounts (AWS, Azure, Google Cloud) and your network for misconfigurations, then maps each finding to six compliance standards — SOC 2, HIPAA, PCI DSS, ISO 27001, NIST CSF 2.0 and CIS Controls v8 — in a single pass.

What makes it feel modern is how you drive it. It runs as an MCP tool (Model Context Protocol, the plumbing that lets AI assistants use external tools) inside Claude Desktop and Claude Code. So instead of learning the tool, you talk to it: “audit my AWS account.” The assistant runs about 20 specialized auditors and reports back.

Crucially, this is local-only. The scan runs on your machine, against your credentials, and nothing about your cloud leaves your computer. For security teams nervous about handing account details to a cloud service, that “zero data exfiltration” design is the whole point.

The bug: audited, but reported “0 findings”

In a recent 0.16.x update, the team added the scan_cloud tool so you could trigger the audit conversationally. It worked — auditors ran, and a later update made them run in parallel so a full account scan fits inside Claude Desktop’s roughly 60-second time budget.

Then an operator installed the build, asked Claude Desktop to “audit my AWS account,” and got back a reassuring result: account audited, ~20 auditors ran, zero findings.

The account was not clean. It had 8 critical exposures, including:

• Two “shadow admin” users with unrestricted permissions (a wildcard * that grants everything)
• A public S3 storage bucket
• A firewall left wide open to the internet on SSH, Postgres and Redis ports
• A public Lambda function URL with no authentication
• An unprotected database table

Any one of these is the kind of thing attackers scan for constantly. The tool found all of them — and then told the user everything was fine.

Why it lied

The findings were never actually lost. They sat in the raw scan results the entire time. The failure was at the very last step: building the summary the user sees.

The tool was constructing that summary using an internal component designed for network port scans, not cloud compliance findings. That component didn’t understand what a cloud finding was, so it quietly discarded all of them. The audit succeeded; the report-writer threw the answers away.

This is the most dangerous kind of bug in a security tool. A false alarm wastes your time. A false “all clear” over a real hole can cost you a breach.

The fix, and an unusually thorough cleanup

The repair was direct: the tool now builds its summary straight from the scan results — severity counts plus the full list of critical and high-severity issues — and writes its report from that. The guarantee is now structural: a cloud that has findings can no longer report zero. The tool also stopped printing a misleading “host is up” line and now correctly scopes “audit my AWS account” to AWS only.

Before shipping, the change went through an adversarial review specifically hunting for missed problems. It found four more edge cases, all fixed the same day:

• Each finding now carries its resource name (which bucket, which user, which firewall rule), so a “critical” is actually actionable. The review caught that the S3 finding was losing its bucket name.
• A list-truncation that could hide a critical issue behind lower-priority ones — fixed by sorting critical-first before any cap.
• A messy raw-data fallback replaced with a clean description.
• Results that couldn’t be attributed to a specific cloud are now grouped and surfaced, not silently dropped.

Confirmed working

The fix was verified in production. Re-running “audit my AWS account” in Claude Desktop now returns all 20 auditors and surfaces 8 critical, 38 high, 41 medium, 46 low and 25 passing checks — each critical labeled with the exact resource at fault.

No new auditors were added, no compliance mappings changed, and the fix lives entirely in the free Community Edition engine; the Enterprise build is just a paired version bump. If you’re auditing a large account, the team suggests bumping one setting (CLOUD_PLUGIN_TIMEOUT_MS=45000) so the heaviest checks have room to finish.

The bigger lesson is a familiar one for anyone trusting AI-assisted tooling: a confident, clean-looking answer is not the same as a correct one. The scan was right all along — it just took a careful human review to make sure the AI actually told us so.

Learn more about NSAuditor AI Enterprise →