The purposes and myths of security automation

The purpose of security automation is to discover all potential security defects before any software release by applying both open source security tools and automation testing frameworks. However, security automation doesn't mean to completely replace manual security testing. Security automation aims to reduce the amount of repeated manual testing and increase testing coverage in an efficient manner. Potential security flaws can exist anywhere, from the source code and third-party components to an insecure configuration or vulnerable infrastructure. As the release cycles of cloud services and software development are getting shorter, the development team needs not only functional testing but also automated security testing. If your team is planning on implementing security automation, it's suggested that you begin with the following resources:

Resource on common security issues	How it could help
OWASP (Open Web Application Security Project) Top 10 Application Security Risks	This lists general web application security risks, such as injection, authentication, XML external entity (XXE) attacks, and misconfiguration. The OWASP also suggests related testing tools and prevention controls for each security issue.
CWE Top 25 Software Errors	This lists the most common software coding errors, such as SQL injection, Cross-site request forgery (CSRF), and buffer overflow. It can be a good checklist for a code-security review.

Security testing can be a tedious and repetitive process. The functional testing may only need to ensure the functionality, but the security testing needs to cover various kinds of the testing scenarios, such as authentication, authorization, XXE, injection, deserialization, and more (see the OWASP resource mentioned in the previous table). Therefore, a certain level of automation would be helpful, considering security testing's repetitive nature, scope, and importance. Be reminded that our goal is not to automate 100% of security testing cases, but to focus on those testing cases that are manually executed and repeated, and so can be done more efficiently by automation. By automating those repeated security testing cases, penetration testers can take time to exploit in-depth weaknesses and logic flaws or review security requirements (all of which can't be done by automation).

When it comes to security automation, there are some challenges and some myths. A lack of proper security or automation knowledge leads to some misunderstanding of security automation. Here are some general suggestions and clarification. We will explore more through the different hands-on case studies in the coming chapters.

Myth 1 – doesn't security testing require highly experienced pentesters?

Our first myth is that security testing requires highly experienced penetration testers and automation testing can't find serious issues.

If we can guide the automation properly, serious security issues can be identified. On the other hand, automated security testing can also result in false-positive issues that need further manual verification. However, there are certain kinds of security testing scenarios that would be ideal for automation; some of those are listed here:

• Detecting the use of banned functions, risky APIs, or weak encryption algorithms. Automated systems can do a good job of scanning code for security issues if we properly define the patterns we are looking for.
• Weak RESTful API authentication and authorization behaviors, such as bypassing authorization vulnerabilities.
• Data input validation may require massive amounts of random testing data input. This kinds of data input testing technique is also called fuzz testing which the prepared data and payload are dynamically generated for the test subject in an attempt to make it crash.
• Repeated UI walk-through, sign-in, sign-out, and form fill are good examples of where web UI automation is required.
• Insecure misconfiguration of software components, databases, or web services.
• Known third-party vulnerabilities.

Myth 2 – isn't it time-consuming to build an automation framework?

Our second myth is that it takes too much time to build an automation framework and that daily development changes may break the automation.

For a development team with mature security testing practices, the adoption of an automation framework such as BDD, data-driven testing (DDT), or Selenium can help with seamless security integration and improve security testing coverage. Adding security testing cases based on these existing automation frameworks won't necessarily take lots of effort and time, so long as the right tools and integration approaches are selected.

For security automation relating to UI flow, daily development changes may or may not break certain UI automation testing cases. It depends on how the UI components are located by the automation testing program. As a general rule of thumb, so long as the UI components are located by the automation testing framework, UI layout changes won't impact the automation.

Myth 3 – there are no automation frameworks that are really feasible for security testing

Our third myth is that there is no single automation framework that can cover the variety of security testing cases.

Now, due to the variety of security testing scenarios, it's true that there is no one single automation framework that can cover all security testing cases. Depending on the security testing scenario, however, we can plan related automation approaches with proper integration of security testing tools and an automation framework. The following table shows examples of security testing scenarios with different automation approaches:

Automation approaches	Mapping to security testing scenarios	Automation tools/frameworks
White box	Secure code inspection Secure configuration inspection	Secure code analysis with Visual Code Grepper (VCG)
API testing	Web/RESTful API security testing Data Input testing (also called parameterized testing or data-driven testing)	Robot Framework's requests library JMeter FuzzDB OWASP ZAP
Web UI automation	Logging in with different users or wrong accounts Logging users out for session management testing Creating a new user account Brute-forcing a user account login	Robot Framework Selenium OWASP ZAP

Automating web UI operations doesn't necessarily take lots of implementation effort or require you to be an expert in Selenium scripts. The Selenium IDE extension will help you to generate automated scripts when you operate UI flows:

• Kantu Selenium IDE: This generates HTML-format scripts. It supports parameterized testing by CSV, takes screenshots of each step for visual review, and can be executed from the command line.
• Katalon Recorder (Selenium IDE for Chrome/Firefox): The key advantage of Katalon is being able to generate various kinds of automation scripts, including Java, Python, Robot Framework, C#, and Ruby scripts. This makes Katalon very flexible for further integration with other tools. It can also support screenshots and DDT by importing CSV files.