Background

There are still many gaps in the security industry when it comes to defining Vulnerability Assessment (VA), Penetration Testing, and Red Teaming. Interestingly, some people think a Pentest is the same as Red Teaming, or even worse, both are considered equal to a Vulnerability Assessment. Objectively, all three have very different functions.

RTO

Although technical definitions can vary in every organization depending on their business model, this article is written in English to help clear up the confusion based on my perspective and experience as a practitioner. Hopefully, this approach provides a more grounded picture of what actually happens in the field.

Introduction to Red Teaming

Fundamentally, Red Teaming is a simulation process to mimic real-world attack threats.

RTO

This operation is done by positioning ourselves as a threat actor or the “bad-guy” through realistic attack scenarios using Tactics, Techniques, and Procedures (TTPs) similar to the behavior of real-world Advanced Persistent Threats (APT).

ROT

For reference, TTPs often used in the field can be studied through Threat Intelligence reports or the MITRE ATT&CK® framework.

The Objectives of Red Teaming

The main goal of Red Teaming is to train the internal team’s capability (Blue Team) in detecting, responding to, and preventing advanced attack threats. This operation is not just limited to testing tools or the existing security stack, but also measuring the effectiveness of the overall security operation, covering People, Process, and Technology.

Additionally, Red Teaming provides deep insights for the organization regarding the real impact a threat actor can cause if they successfully execute a threat in our infrastructure. In other words, we try to map the risk from an actual attack perspective.

Correlation and Characteristics

To illustrate the relationship and differences between Vulnerability Assessment (VA), Penetration Testing, and Red Teaming, we will use the concept of an inverted pyramid.

This visualization helps us see how breadth (coverage) and depth (detail) are inversely proportional in each method:

Vulnerability Assessment (VA)

Tends to have a very wide coverage but is not very deep regarding finding vulnerabilities. The main focus is scanning, enumeration, and reporting with the goal of helping us identify the attack surface quickly. In terms of time, this assessment tends to be faster than other methods.

Penetration Testing (Pentest)

Using the previous Vulnerability Assessment results, a Pentester uses the VA information as initial data to move to a deeper level. Here, we perform exploitation to prove the existence of a gap, identify risks, and measure the real impact of a vulnerability. Unlike VA which only lists the gaps, Pentest focuses on how far those gaps can be exploited.

Red Teaming Operation

Some methods and techniques in Penetration Testing are indeed used in Red Teaming. Therefore, having a Pentesting skill set is a minimum requirement before running a Red Teaming operation.

In terms of focus and goals, the two are very different. Red Teaming focuses more on the overall and deep security operations covering People, Process, and Technology. The goal is to train the organization and internal team (Blue Team) in detecting, responding to, and preventing advanced attacks or we can say, “to make the Blue Team or process better.”

Assessment Type	Scope and Objectives
Vulnerabilty Assesment	Broad Scope // Prioritized list of vulnerabilities
Penetration Testing	Varied Scope // Measures risk of vulnerabilities
Red Teaming	Varied Scope // Measures impact at an operational level

From the attack execution side, Red Teaming is much deeper and stealthier (silent) compared to Penetration Testing. In a Pentest, we are often “noisy” because the goal is to find as many gaps as possible in a limited time. On the other hand, Red Teaming can include physical attacks (like hardware implants or lock picking), social engineering, and spear phishing. In terms of duration, this operation takes quite a long time even months to achieve a specific objective.

A Red Teamer will do whatever it takes to achieve their goal (objective-oriented). They can use small vulnerabilities and misconfigurations just as a foundation to perform lateral movement or pivoting from one application or network segment to another. The final goal can vary, from compromising critical servers and stealing sensitive data to simulating a specific adversary profile to measure the Blue Team’s effectiveness all done according to the agreed Rules of Engagement (RoE).

Summary Comparison Table

RTO

Pirates vs. Ninjas

The Pirates vs. Ninjas analogy is often used in the security community to distinguish between Penetration Testing and Red Teaming. This analogy is very accurate to describe the mindset and operations of both.

The fundamental difference between a Pirate and a Ninja in how they reach their goal.

Pirate	Ninja
Tough	Fast
Rough & Flashy	Quiet & Stealthy
Careless	More organized
Good in long-range combat	Good in long-range & close combat

Pentester (The Pirate)

Pirates tend to be ”noisy”, aggressive, and focus on looting as much as possible in a short time. They usually come to break down the defensive walls and take all the treasures (vulnerabilities) they find.

Red Teamer (The Ninja)

Ninjas usually move at night, hiding in the shadows with silent steps (stealth). They don’t try to open every locked door, their focus is only one which is to find one small gap that can lead them directly to the main target without their presence ever being realized.

Who’s Better?

In achieving a goal, they use different techniques and expertise. We cannot ask a Pirate to run a silent operation that requires months of patience. Likewise, we don’t want to use a Pentester to measure the effectiveness of a Blue Team’s ability to detect, respond to, and prevent advanced attacks. On the other hand, using a Red Team just to find and exploit basic vulnerabilities is a waste of budget and resources.

So, who is better? Both are very capable in their respective fields. The key is not to find who is best, but to put them on the right mission and target.

Red Teaming Readiness

So, what kind of company is suitable for Red Teaming, and when is the right time to start?

Red Teaming is recommended if the company already has a mature security maturity level. This means the organization is already confident with its defenses because it has routinely performed Vulnerability Assessment and Penetration Testing before.

If a company is already at an enterprise scale with a very wide scope, routinely performs assessments (Pentest or Bug Bounty), and already has an internal team (Blue Team, Threat Hunter) to detect threats, then Red Teaming is a great reference to test and train the team’s ability to detect and respond to advanced attacks that might be carried out by a threat actor without the internal team realizing it.

Closing Thoughts

Hopefully, this short writing can help clear up the existing confusion and provide a new perspective for friends in the industry. Don’t let us only start cleaning up after an incident has already happened. Stay sharp and keep hunting!

Stop Calling It a Red Teaming

Maland | Wednesday, May 20, 2020