A CISO’s Guide to Pentesting

References

****************************

Hello, and welcome to another episode of CISO Tradecraft -- the podcast that provides you with the information, knowledge, and wisdom to be a more effective cyber security leader. My name is G. Mark Hardy, and today we're going to explore a number of things a CISO needs to know about pentesting. As always, please follow us on LinkedIn, and make sure you subscribe so you can always get the latest updates.

Now to get a good understanding of pentesting we are going over the basics every CISO needs to understand.

What is it
Where are good places to order it
What should I look for in a penetration testing provider
What does a penetration testing provider need to provide
What’s changing on this going forward

First of all, let's talk about what a pentest is NOT. It is not a simple vulnerability scan. That's something you can do yourself with any number of publicly available tools. However, performing a vulnerability scan, and then acting on remediating what you find, is an important prerequisite for a pentest. Why pay hundreds of dollars per hour for someone to point out what you can find yourself in your bunny slippers sipping a latte?

Now let’s start with providing a definition of a penetration test. According to Wikipedia a penetration test or pentest is an authorized simulated cyber-attack on a computer system performed to evaluate the security of a system. It’s really designed to show weaknesses in a system that can be exploited. Let’s think of things we want to test. It can be a website, an API, a mobile application, an endpoint, a firewall, etc. There’s really a lot of things you can test, but the thing to remember is you have to prioritize what has the highest likelihood or largest impact to cause the company harm. You need to focus on high likelihood and impact because professional penetration tests are not cheap. Usually, they will usually cost between $10,000-$30,000 but if you have a complex system, it’s not unheard of to go up to $100,000. As a CISO you need to be able to defend this expenditure of resources. So, you will usually define a clear standard that our company will perform penetration tests on customer facing applications, PCI applications, and Financially Significant Application or SOX applications once per year.

My friend John Strand, who founded Black Hills Information Security, pointed out in a recent webcast that sometimes you, the client, may not know what you mean by the term pentest. Sometimes clients want just a vulnerability scan, or sometimes an external scan of vulnerabilities to identify risk, or sometimes a compromise assessment where a tester has access to a workstation and tries to work laterally, or sometimes a red team where a tester acts like a threat actor and tries to bypass controls, or a collaborative effort involving both red teams and blue teams to document gaps and to help defenders do their job better. He goes on to state that your pentest objective should be to "provide evidence of the effectiveness of current defensive mechanisms and attack detection methodologies."

Please do not confuse a penetration test with a Red Team exercise. A red team exercise just wants to accomplish an objective like steal data from an application. A penetration test wants to enumerate vulnerabilities in a scoped target system so the developer can patch and remediate. It’s a subtle difference but consider that a red team only needs to find one vulnerability to declare success, whereas a penetration test keeps going to help identify potentially exploitable vulnerabilities. Now, is a pentest about finding ALL vulnerabilities? I would say no – there are vulnerabilities that might require a disproportionate amount of resources to exploit for little or no value – something with a CVSS score of 4.0 or the like. Those can often be left unpatched without consequence – the cost of remediating may exceed the value of the risk avoided. There really is a “good enough” standard of risk, and that is called “acceptable risk.” So, when scoping a pentest or reviewing results, make sure that any findings are both relevant and make economic sense to remediate.

Let’s take the example that you want to perform a web application pentest on your public website so you can fix the vulnerabilities before the bad actors find them. The first question you should consider is do you want an internal or an external penetration test. Well, the classic answer of "it depends" is appropriate. If this website is something of a service that you are selling to other companies, then chances are those companies are going to ask you for things like an ISO 27001 certification or SOC 2 Type 2 Report and both of those standards require, you guessed it, a penetration Test. In this case your company would be expected to document a pentest performed by an external provider. Now if your company has a website that is selling direct to a consumer, then chances are you don’t have the same level of requirement for an external pentest. So, you may be able to just perform an internal penetration test performed by your company’s employees.

I'd be remiss if I didn't mention the Center for Internet Security Critical Controls, formerly know as the SANS Top 20. The current version, eight, has 18 controls that are listed in order of importance, and they include pentesting. What is the priority of pentesting, you may ask? #18 of 18 -- dead last. Now, that doesn't mean pentests are not valuable, or not useful, or even not important. What it does mean is that pentests come at the end of building your security framework and implementing controls. Starting with a pentest makes no sense IMHO, although compliance-oriented organizations probably do this more often than they should. That approach makes the pen testers job one of filtering through noise -- there are probably a TON of vulnerabilities and weaknesses that should have been remediated in advance and could have been with very little effort. Think of a pentest as a final exam if you will. Otherwise, it's an expensive way to populate your security to-do list.

OK let’s say we want to have an external penetration test and we have the 10-30K on hand to pay an external vendor. Remember this, a penetration test is only as good as the conductor of the penetration test. Cyber is a very unregulated industry which means it can be tricky to know who is qualified. Compare this to the medical industry. If you go to a hospital, you will generally get referred to a Medical Doctor or Physician. This is usually someone who has a degree such as a MD or DO which proves their competency. They will also have a license from the state to practice medicine legally. Contrast this to the cyber security industry. There is no requirement for a degree to practice Cyber in the workforce. Also, there is no license issued by the state to practice cyber or develop software applications. Therefore, you need to look for relevant Cyber certifications to demonstrate competency to perform a Penetration Test. There’s a number of penetration testing certifications such as the Certified Ethical Hacker or CEH, Global Information Assurance Certification or GIAC GPEN or GWAP, and the Offensive Security Certified Professional or OSCP.

We strongly recommend anyone performing an actual penetration test have an OSCP. This certification is difficult to pass. A cyber professional must be able to perform an actual penetration test and produce a detailed report to get the actual certification. This is exactly what you want in a pentester, which is why we are big fans of this certification. This certification is a lot more complicated than remembering a bunch of textbook answers and filling in a multiple-choice test. Do yourself a favor and ask for individuals performing penetration tests at your company to possess this certification. It may mean your penetration tests cost more, but it’s a really good way to set a bar of qualified folks who can perform quality penetration tests to secure your company.

Now you have money, and you know you want to look for penetration tests from companies that have skilled cyber professionals with years of experience and an OSCP. What companies should you look at? Usually, we see three types of penetration testing companies. Companies that use their existing auditors to perform penetration tests – firms like KPMG, EY, PWC, or Deloitte (The Big 4 1/2). This is expensive but it’s easy to get them approved since most large companies already have contracts with at least one of these companies. The second type of company that we see are large penetration testing companies. Companies like Bishop Fox, Black Hills Information Security, NCC Group, and TrustedSec, focus largely on penetration testing and don't extend into other areas like financial auditing. They have at least 50+ penetration testers with experience from places like the CIA, NSA, and other large tech companies. Note they are often highly acclaimed so there is often a waitlist of a few months before you can get added as a new client. Finally, there are boutique shops that specialize in particular areas. For example, you might want to hire a company that specializes in testing mobile applications, Salesforce environments, embedded devices, or APIs. This is a more specialized skill and a bit harder to find so you have to find a relevant vendor. Remember if someone can pass the OSCP it means they know how to test and usually have a background in Web Application Penetration testing. Attacking a Web application means being an expert in using a tool like Burp Suite to look for OWASP Top 10 attacks like SQL injection or Cross Site Scripting. This is a very different set of skills from someone who can hack a Vehicle Controller Area Network (CAN) bus or John Deere Tractor that requires reverse engineering and C++ coding.

Once you pick your vendor and successfully negotiate a master license agreement be sure to check that you are continuing to get the talent you expect. It’s common for the first penetration test to have skilled testers but over time to have a vendor replace staff with cheaper labor who might not have the OSCP or same level of experience that you expect. Don’t let this happen to your company and review the labor and contract requirements in a recurring fashion.

Alright, let’s imagine you have a highly skilled vendor who meets these requirements. How should they perform a penetration test? Well, if you are looking for a quality penetration testing guide, we recommend following the one used by Google. Google, whose parent company is called Alphabet, has publicly shared their penetration testing guidelines and we have attached a link to it in our show notes. It’s a great read so please take a look. Now Google recommends that a good penetration test report should clearly follow an assessment methodology during the assessment. Usually, penetration testers will follow an industry recognized standard like the OWASP Web Security Testing Guide, the OWASP Mobile Security Testing Guide, the OWASP Firmware Security Testing Guide, the PCI DSS Penetration Testing Guide, The Penetration Testing Execution Standard, or the OSSTMM which stands for The Open Source Security Testing Methodology Manual. These assessment methodologies can be used to show that extensive evaluation was done, and a multitude of steps/attacks were carried out. They can also standardize the documentation of findings. Here you will want a list showing risk severity level, impact from a business/technical perspective, clear concise steps to reproduce the finding, screenshots showing evidence of the finding, and recommendations on how to resolve the finding. This will allow you to build a quality penetration test that you can reuse in an organization to improve your understanding of technical risks.

If I can get good penetration tests today, perhaps we should think about how penetration testing is changing in the future? The answer is automation. Now we have had automated vulnerability management tools for decades. But please don’t think that running a Dynamic Application Security Testing Tool or DAST such as Web Inspect is the same thing as performing a full penetration test. A penetration test usually takes about a month of work from a trained professional which is quite different from a 30-minute scan. As a cyber industry we are starting to see innovative Penetration Testing companies build out Continuous and Automated Penetration Testing tooling. Examples of this include Bishop Fox’s Cosmos, Pentera’s Automated Security Validation Platform, and Stage 2 Security Voodoo and Mage tooling. Each of these companies are producing some really interesting tools and we think they will be a strong complement to penetration tests performed by actual teams. This means that companies can perform more tests on more applications. The other major advantage with these tools is repeatability. Usually, a penetration test is a point in time assessment. For example, once a year you schedule a penetration test on your application. That means if a month later if you make changes, updates, or patches to your application then there can easily be new vulnerabilities introduced which were never assessed by your penetration test. So having a continuous solution to identify common vulnerabilities is important because you always want to find your vulnerabilities first before bad actors.

Here’s one final tip. Don’t rely on a single penetration testing company. Remember we discussed that a penetration testing company is only as good as the tester and the toolbox. So, try changing out the company who tests the same application each year. For example, perhaps you have contracts with Bishop Fox, Stage 2 Security, and Black Hill Information Security where each company performs a number of penetration tests for your company each year. You can alternate which company scans which application. Therefore, have Bishop Fox perform a pentest of your public website in 2022, then Stage 2 Security test it in 2023, then Black Hills test it in 2024. Every penetration tester looks for something different and they will bring different skills to the test. If you leverage this methodology of changing penetration testing vendors each cycle, then you will get more findings which allows you to remediate and lower risk. It allows you to know if a penetration testing vendor’s pricing is out of the norm. You can cancel or renegotiate one contract if a penetration testing vendor wants to double their prices. And watch the news -- even security companies have problems, and if a firm's best pentesters all leave to join a startup, that loss of talent may impact the quality of your report.

Thank you for listening to CISO Tradecraft, and we hope you have found this episode valuable in your security leadership journey. As always, we encourage you to follow us on LinkedIn, and help us out by letting your podcast provider know you value this show. This is your host, G. Mark Hardy, and until next time, stay safe.

#90 - A CISO’s Guide to Pentesting

CISO Tradecraft®

#90 - A CISO’s Guide to Pentesting