Introduction

In a previous post, we discussed foundational practices for Software Security. Several of these practices require selecting and implementing purpose-built software security tools. These are the most frequently tools used to scan your source code and manifest files for vulnerabilities.

However, if your organization has not yet established code scanning tools and practices, one of the most difficult questions to answer is simply, “Which tools should I use?”

The software security product space only grows more crowded and complex from one year to the next, and knowing how to evaluate which tools are the right fit for your organization can be daunting. In this blog post, I will provide you with 11 simple guidelines you can use to approach the decision of which security tools to use for your needs.

Security Tool Selection Guidelines

1. Identify Your Specific Needs

Before you begin evaluating tools, it’s imperative to understand what your organization’s goals are. There are two possible starting points:

• If your goal is to identify issues in your own code, then you will want to primarily focus on SAST (Static Analysis Security Testing) tools.
• If you are interested in identifying vulnerable third-party dependencies, such as free and open-source software (FOSS) libraries, you will want to focus on Software Composition Analysis (SCA) tools.

In most cases, you will eventually want to use both types of tools, regardless of where you decide to start.

2. Evaluate Your Organizational Capability

One of the most important considerations for those standing up a new security scanning practice is to be well aware of the capability of your current team.

• If you have a dedicated security team and internal documentation that can act as a resource to your developers, it may be that your organization can tolerate a tool that does less hand-holding for the end user.
• If you have a small or nonexistent security team, you might want to look for tools that provide extensive built-in resources like articles, bug databases, and video learning. In this scenario, your developers will bear primary responsibility for understanding the tool output, and having more resources available will help them make better decisions while increasing their security understanding.

Remember, you can plan in your software security roadmap to switch tools as the organization matures.

3. Consider Language Support

Different tools excel at analyzing code written in specific programming languages. When evaluating tools, you may want to consider whether the tools you select support the languages used most in your projects.

A good general guideline is that tools tend to perform best when scanning the language that they are written in, even though most tools do a decent job at scanning other languages.

4. Review Integration Capabilities

At a minimum, you are going to want tools that integrate with your version control system, continuous integration/continuous deployment (CI/CD) pipeline, and ideally your integrated development environments (IDEs).

Security tools, by definition, introduce friction into the development process and you should look for every opportunity to reduce this friction through tight integration with the developer toolset. Also, consider the shift left rule. The earlier in the process, you find a problem, the less it costs to fix. This is why IDE integration and pipeline integration are important to consider.

5. Think About Accuracy and False Positives

The accuracy of the tool’s analysis is vital. This is usually a much larger concern for SAST tools than for SCA tools, but it does apply to both.

High false positive rates can be time-consuming and frustrating for development teams. Accuracy ratings provided by vendors are notoriously inflated, so Trailhead recommends evaluating products with a trial period allowing you to use it to scan your real codebases and review the results for noise and accuracy.

6. Up-To-Date Vulnerability Database

The effectiveness of SCA tools heavily relies on their vulnerability database. Ensure that the tool you choose has an up-to-date and comprehensive database of known vulnerabilities. At a minimum, we recommend tools that leverage the public NVD database, but if they leverage other sources it is a big bonus.

7. Policy and Rule Configuration

Every project is unique, and your tool should allow you to customize the ruleset to adapt to your specific requirements. This is especially true for SAST tools that scan your code.

Look for tools that let you configure rules and policies to align with your coding standards and security guidelines. Use your evaluation period to make sure they conform to your existing standards.

8. Check for Reporting and Visualization

Analyze the reporting and visualization capabilities of the tools. Clear and detailed reports help developers understand issues and prioritize fixes effectively. Visual representations, such as graphs and charts, can provide valuable insights into the codebase.

9. Think about Scalability

It is important to consider whether a given tool can support an organization of your size (or your anticipated future size).

Also, investigate how the tools are deployed. Are they on-prem or cloud-based? How easy is it to scale the deployment if you anticipate rapid growth in your use of the tool? What will the cost impact be of scaling?

10. Cost and Licensing Considerations

Budget constraints are always a consideration, and security tools can be particularly cost-prohibitive for smaller organizations. Understand the cost structure of the tools you’re evaluating, including licensing fees and ongoing maintenance costs.

Evaluate whether the investment aligns with the value it brings to your project. Evaluate whether it might be best to focus initially on only SAST or only SCA, and then plan to expand into the other practices as your organization grows.

11. Support and Community

The availability of support and an active user community can be invaluable. A responsive support team can assist with issues, while a thriving user community can provide insights and best practices. Check for available documentation, forums, and user groups.

Getting You Started

Selecting the right tools for the job is one of the most critical first steps in implementing or growing the security practice in your organization. To get you started I’ve compiled a list of options in the table below:

Conclusion

If you are new to software security or new to software security tooling, reach out to Trailhead for a consultation and we will be happy to walk with you as you begin this journey.