Many of the clients that contact us regarding penetration tests ask us to see if we can "break-in to their Internet systems." We tell them we can certainly help, but we first ask them to unpack what it is they really want to know. "Is our organization secure from Internet hackers?" is the typical reply. Fair enough and a great question to have answered. Quite understandably, they also have a limited budget and do not want to do anything that could potentially interrupt business services. In this entry, I want to expand everyone's thinking regarding what a pentest does or does not prove.
Budget and time will limit what can be targeted in a pentest. The tester and client organization will agree on a particular vector, which we will call "A", with the objective of identifying and possibly exploiting any weaknesses that could compromise the confidentiality, availability, or integrity of the system and the network to which it is attached. Let's assume vector "A" is the target organization's Internet edge systems, i.e. their Web, FTP, and SMTP servers. Based on their business constraints, the tester must also define the scope and rules of engagement, such as budget not to exceed $5K; attacks only performed after hours; no social engineering; no denial of service.
The test begins. After performing information gathering, port/service scanning, vulnerability scanning, and exploits, the tester may find the organization's devices on vector "A" are free from remotely exploitable weakness. Sure, there's some security weaknesses that must be fixed, but vector "A" is currently free from exploitable root/admin level holes. That's good news, but it comes with a very serious caveat:
Cyber thieves have no "rules of engagement." They also have no budget or time constraints. Yes, there's the personal risk of imprisonment, but the adversary's constraints are almost zero when compared with those of the professional penetration tester. A determined intruder will do what it takes to achieve their goal -- via phone, physical penetration, and denial of service, if needed. This means once the attacker (or his process) determines vector "A" is not the the best route, vectors "B", "C", "D", and so on will be tested until the goal is achieved. Given these issues, an honest executive summary for the pentest of vector "A" would read:
"Vector "A" is safe from a pentester under constraints x, y, & z. Your ability to withstand a real attacker that will not stop at Vector "A" may be determined at a future date, with or without notice. It may have occurred already, but you've not yet detected it."
Don't use a limited pentest to convince management, clients, or yourself that your "organization is secure." That is like having a dermatologist examine only your back and declare your body is free from suspicious and possibly pre-cancerous spots. Use pentests to explore specific assumptions such as "Can someone gain unauthorized access to our SSH server from the Internet?" then go a couple of steps further by evaluating detection: "What would it look like if someone breached our SSH server?" My main thrust here is that prevention will fail, i.e. a real attacker will likely use a technique that is not publicly known or not allowed in your scope, so augment your pentesting to include identifying what a breach looks like in your environment.
We use safe and trustworthy "agents" from Core IMPACT that can mimic the activities of an intruder. If we cannot penetrate your DMZ, we will have the sysadmin install an agent on a DMZ host and declare the host as "breached." We then begin attacking DMZ to DMZ, or DMZ to inside, just like a determined intruder would. Most, if not all of our clients have never seen the events that are generated by this approach. This gives them a non-hostile and valuable chance to see what a breached and probing system looks like. More importantly, it also gives them the opportunity to verify detection systems like IDS/IPS, AV, AntiSPAM, OS Logs, and SIM, are capturing our activities. This isn't a new concept -- it's tiger teams, red/blue teaming spun a little differently.
Zero day exploits will continue to arise. Patches will get missed. Your users will click a link they shouldn't. Simply put: prevention will fail. Using a pentest to support a "we're secure" assertion is dangerous, especially if the test does not evaluate your ability to detect a penetration. And, failing to embrace that a pentest alone only evaluates a constrained pentester's skill against a slice of your network, is a perilous method for determining whether your entire organization is safe from an uninhibited, highly motivated, and very skilled adversary.
