What a Network Security Auditor Actually Does (and What They Can’t)

A network security auditor collects evidence that an organisation’s network controls work the way the organisation says they work, and reports the distance between the two.

Every part of that sentence carries weight. Evidence, not opinion. The way the organisation says they work: an auditor tests the design an organisation committed to, not the design the auditor would have chosen. And reports, not fixes. The moment an auditor starts fixing things, they stop being able to audit them.

The title covers two things at once, which is why searching for it returns a mixture of job postings and dictionary definitions. It names a role a person holds, and it names a category of tooling that produces the raw material that person works from. This article is about the role. The tooling appears where it belongs, which is underneath.

Internal audit, external assessor, penetration tester

Three jobs get filed under the same heading and they answer different questions.

The internal auditor works inside the organisation and reports, in a well-run company, to the audit committee rather than to the CISO. Their subject is the organisation’s own control set: is the control designed to do what it claims, and did it operate that way over the period under review? Their output is a finding that management must formally respond to, with an owner and a date.

The external assessor is engaged from outside to reach a formal conclusion against a named standard, for a named scope and a named period. A SOC 2 service auditor is a CPA firm working under AICPA attestation standards. An ISO/IEC 27001 auditor works for a certification body accredited under ISO/IEC 17021-1. A PCI DSS Report on Compliance is produced by a Qualified Security Assessor, and the QSA qualification is held by an individual working at a QSA company, not by a freelancer. These people produce artifacts with commercial and sometimes legal weight.

The penetration tester answers a different question entirely: can this be broken, by someone with these starting conditions, in this window. That is exploitability, not control effectiveness. A clean pen test does not mean an access review is running monthly, and a failed access review does not mean anyone can get in. PCI DSS v4.0.1 keeps them separate on purpose: Requirement 11.3 covers vulnerability scanning, Requirement 11.4 covers penetration testing. Two requirements because they are two activities. And Requirement 11.4.1 puts the burden of validating segmentation and scope-reduction controls on the pen-test methodology, not on the scan.

Underneath all three sits the automated substrate: the scanners and cloud auditors that enumerate what is actually configured. That layer is not an auditor. It is where the auditor’s raw material comes from.

The independence rule is the job

An auditor must not audit systems they administer. This is not etiquette; it is the property that makes the output worth anything.

The standard formulation in internal audit is the three-lines model: operations own the risk, risk and compliance functions oversee it, internal audit provides independent assurance and reports to the board. ISO/IEC 27001:2022 Clause 9.2 makes internal audit mandatory and requires the audit process to be objective and impartial, which in practice rules out the people who designed and implemented the thing under audit. AICPA independence rules govern who may serve as a service auditor. Every one of these exists for the same reason: a person cannot credibly attest that their own work is correct.

The practical consequences show up fast:

• If you wrote the firewall rules, you cannot be the person who signs off that the rulebase is reviewed and justified.
• If you hold the break-glass credentials, someone else reviews their use.
• The auditor’s own access should be read-only and scoped, and it should be logged like anyone else’s. An auditor who needs write access to collect evidence is an auditor about to contaminate it.

That last point is also the reason serious teams care what permissions a scanning tool requires. A collection tool that asks for write access to production has just made every artifact it produces harder to defend.

What a network security auditor actually examines

The examination list is longer than “run a scan”, and the interesting questions are almost never about the existence of a control.

Firewall and security-group rulebases. Not “is there a firewall” but: is every rule attributable to a business justification, is it dated, does it have an owner, and has it been reviewed? PCI DSS v4.0.1 Requirement 1.2.5 says it plainly: all services, protocols and ports allowed are identified, approved, and have a defined business need. Requirement 1.2.1 requires the configuration standards for those rulesets to be defined, implemented and maintained. The finding is rarely “port 3389 is open to the world” — although that happens. It is usually “this rule has no owner and nobody can say why it exists.”

Segmentation. Whether a sensitive zone is genuinely isolated, or isolated on a diagram. In cloud estates this means route tables, peering connections, transit gateway attachments, and the security groups that reference each other. Segmentation that has never been tested is an assumption.

Access control. Joiner/mover/leaver in practice, not in policy. Multi-factor coverage, including the accounts that quietly bypass it. Privileged access, break-glass, and service accounts with keys nobody has rotated in three years. PCI DSS v4.0.1 Requirement 8.2.5 requires terminated users’ access to be revoked immediately, and Requirement 7.2.4 requires every user account and its privileges to be reviewed at least once every six months. Both are controls an auditor can test against an HR leaver list in an afternoon.

Patch and exposure state. The count of vulnerabilities is the least useful number in the report. The useful numbers are time-to-remediate by severity, and whether the remediation window in the policy matches the one in the data.

Logging. Whether logs exist, whether they are retained for the period the framework requires, whether anyone reads them, and whether clocks agree well enough across sources that a timeline can be reconstructed. CIS Controls v8 gives audit log management a control of its own, Control 8. HIPAA §164.312(b) requires audit controls over systems holding electronic protected health information.

Cloud IAM. The hardest one, and the one where scanning earns its place. What matters is not the permissions assigned but the permissions effective, after resource policies, permission boundaries, service control policies and role chaining are resolved. Cross-account trust with a wildcard principal, a role assumable by anyone in the account, a storage bucket exposed through a policy nobody read: these are not visible by inspection, they are visible by evaluation.

Encryption and key custody. At rest, in transit, and the question people forget: who can decrypt. A key policy that grants decrypt to a broad principal set undoes the encryption claim above it.

EXAMINE, INTERVIEW, TEST

Assessment frameworks share a three-method model for gathering evidence. NIST SP 800-53A and NIST SP 800-171A name the methods explicitly: EXAMINE, INTERVIEW, TEST. The AICPA’s procedures vocabulary divides the same ground differently, with inspection, inquiry, observation and reperformance among its terms, but the idea is identical: three ways to learn something, and they are not interchangeable.

EXAMINE means reading an artifact: a configuration export, a policy document, a rulebase, a log extract, an access list, a ticket.

INTERVIEW means asking the people who run the process. Does the review actually happen, or does the calendar invite happen? Who approves an exception, and what did they do the last time one was requested at 6pm on a Friday?

TEST means exercising the control. Attempt the login. Restore the backup and time it. Try the lateral movement the segmentation is supposed to prevent.

This model is the thing to learn first, because it explains exactly where automation fits.

A scanner supplies EXAMINE-method material, for a subset of assessment objectives, at a scale and repeatability no human can match. It supplies nothing for INTERVIEW. For TEST it is marginal: reading a configuration is not the same as exercising the control that configuration is supposed to implement.

That is why “the tool says we pass” is not a sentence anyone in this field says. A determination is made across all three methods, by a person who is accountable for it, for a named scope and a named period. Most requirements retain at least one objective about a defined procedure, an authorized list, or an identified set — and configuration state cannot evidence any of those. A tool that renders a pass/fail verdict on a control is claiming something it is structurally unable to know.

Where the substrate fits

The honest framing for tooling in this space is: it collects and normalises examine-method material, and it routes that material to the control it is relevant to. Nothing more, and that is already a lot of work.

NSAuditor AI, built by this magazine’s publisher, is one example of the category. The Community Edition is MIT-licensed and free, with 27 scanner plugins. The Enterprise Edition adds 29 more: 28 cloud auditors across AWS, Azure and GCP, plus one non-cloud plugin, for 56 in total. Findings route to eight frameworks: SOC 2 (TSC 2017), HIPAA §164.312, NIST CSF 2.0, PCI DSS v4.0.1, ISO/IEC 27001:2022, CIS Controls v8, GDPR Article 32 (security of processing), and NIST SP 800-171 Rev 2 as evidence substrate for CMMC Level 2 preparation. It runs on Windows, macOS and Linux, alongside the classic NSAuditor desktop suite for Windows.

Three design decisions in it are worth naming, because they generalise to any tool an auditor lets near production. Scans run locally rather than shipping estate data to a vendor. The cloud plugins issue read-only calls, with the mutating verbs blocked by a guard in the code rather than promised in the documentation. And timestamping through an RFC 3161 Time-Stamp Authority is opt-in with no default, because it is an outbound call and plenty of the estates that most want evidence integrity will not permit one.

What it does not do is tell you whether you pass. It says what it found.

What changed when infrastructure became API-addressable

Twenty years ago a network audit meant a rulebase export, a port scan, a walk through a comms room and a lot of sampling. Sampling was a necessity: reading everything was not physically possible, so the methodology was built around defensible selection.

A paginated API call now returns every security group rule in a region, and a loop over regions returns the account. The population is enumerable. Three things follow.

Sampling became a choice. If the whole population is readable, selecting 25 items needs a reason. Some assessors still sample because the standard’s reporting template expects it. But the argument has shifted from “we cannot read it all” to “we chose not to”.

The scarce skill moved from collection to adjudication. Anyone can produce four thousand findings. The job is deciding which forty matter, which are compensated for elsewhere, which are accepted risk with a signature behind them, and which are the tool misreading a resource type. An auditor who cannot argue down a false positive is going to burn every relationship they have.

Freshness and custody started to matter. A configuration export is true at the instant it was taken. A SOC 2 Type II period typically covers six to twelve months. So the question is no longer “is this correct” but “was this correct continuously, and what shows that”. Which leads directly to provenance: who produced this JSON, from which account, when, and can anyone tell whether it changed afterwards.

Certifications, and what each one signals

• CISA (ISACA). The audit-process credential. If you are hiring one person to run an audit function, this is the one that signals they understand audit, and not only security.
• CISSP (ISC2). Breadth across security domains. A common baseline in job postings. It is not an audit qualification and does not pretend to be.
• ISO/IEC 27001 Lead Auditor. An accredited training course plus exam. In practice this is what a certification body requires from the people it fields under ISO/IEC 17021-1.
• CCSP (ISC2). Cloud-specific, and increasingly the differentiator, because most of the estate under audit is now someone else’s data centre.
• CISM (management-facing), CIA (internal audit as a discipline, from the IIA), and PCI QSA (held by an individual at a QSA company) are the common adjacencies. OSCP belongs to the penetration testing track, which is the neighbouring job, not this one.

Certifications get the interview. What gets the offer is the ability to write a finding a business can act on: condition, criteria, cause, effect, recommendation, in language an owner can respond to without a translator.

If you are hiring one

Ask a candidate to write a finding in front of you from a real configuration snippet. Ask who they reported to in their last role, and listen for whether independence was structural or aspirational. Ask what they would do with four thousand scanner findings on day one — the good answer involves grouping by control and by owner before anything else, and the bad answer involves a spreadsheet sorted by severity.

The strongest signal is negative: a candidate who treats a scan result as a control conclusion has not done the job, whatever their certifications say.

If you are becoming one

The common routes in are sysadmin or cloud engineer to security engineer to audit, or IT audit and accounting to security audit. Both work. The first arrives knowing what the evidence means; the second arrives knowing what an assessment is.

Two pieces of advice that people rarely give. First, learn one framework end to end at control-id granularity, not “we do SOC 2” but what CC6.1 asks for and what evidence discharges it. Depth in one transfers; breadth across all of them does not. Second, practise writing. This is a writing job with a technical prerequisite, and the finding is the deliverable. An accurate observation nobody acts on has not improved anyone’s security.

The role exists because someone has to check, independently, whether the controls an organisation describes are the controls it runs. Tooling has made the collection part enormously cheaper. It has not made the checking part optional, and it has not made it automatic.

Further reading: a step-by-step network security audit checklist, with the auditor’s role at each step, is on nsauditor.com.