Code security scanning with Amazon Q Developer
October 20, 2024A primary objective of software developers is to develop products that uphold the highest standards of data privacy and security, fostering trust and confidence among their users and customers. Developers seek to secure their software by identifying and mitigating security vulnerabilities in their codebase, thereby enhancing its resilience against cyber threats. Amazon Q Developer, a generative AI–powered assistant, helps shift left and prioritize security earlier in the software development life cycle (SDLC) by providing guidance to developers as they write code within their integrated development environment (IDE).
As a developer, you can use Amazon Q Developer code security scanning to help you proactively identify and remove security vulnerabilities in your codebase, both in existing codebase and in new code as you write it in the IDE. Amazon Q Developer is powered by thousands of security detectors across several programming languages that help you create software that helps meet security requirements and provides an experience that your customers can trust. Addressing the findings generated by Amazon Q Developer reduces the number of security vulnerabilities in the software, and reduces costs by eliminating vulnerabilities early in the development cycle before they propagate to later stages like testing.
This blog post explores the code security scanning feature of Amazon Q Developer and the security detectors that Amazon Q uses to scan your code. First, we demonstrate the auto-scan feature of Amazon Q Developer that assesses the code as you write it. Then, we walk through how to initiate a security scan of an active project and its dependencies in the IDE, review the findings about detected security vulnerabilities, and use the automated remediations provided by Amazon Q Developer to fix the detected vulnerabilities. Finally, this post provides an analysis of the performance of security scans by Amazon Q and a comparison with the performance of similar tools on respected public benchmarks.
Code Security Scanning
Amazon Q Developer assists you to follow secure coding practices by providing two ways of scanning your code: Scan your project and Scan as you code. Amazon Q Developer can run on-demand scans of your entire project. It can also scan your code in real-time as you write it in the IDE.
Amazon Q Developer code security scanning feature currently incorporates thousands of security detectors across more than a dozen programming languages which all have unique benefits to provide a wide array of vulnerability detection capabilities. The scan generates a detection message that includes a description of the issue and a recommended fix. Some of the security vulnerability detections have an accompanying suggested code fix that Amazon Q Developer provides within the IDE. If you choose the option to fix the code, Amazon Q Developer updates your code.
Running Security Scans
The prerequisite for running a security scan is to install the Amazon Q Developer plugin in a supported IDE of your choice. In this walkthrough, we are using the JetBrains IntelliJ IDE. Once you are authenticated to the Amazon Q Developer service, you’ll see the section for Security Scans, including a selection for Run Project Scan, in the Amazon Q Developer menu. If you have subscribed to Amazon Q Developer Pro, auto-scans are enabled by default and you’ll see the additional selection to Pause Auto-Scans in the Amazon Q Developer menu.
When the auto-scan feature is enabled, the security scans will run in the background periodically and highlight the vulnerabilities that are detected in the file where you are actively writing code. Let’s explore this functionality in the context of an example of code where a hard-coded password is used in the credentials to establish a connection with a database. This is a critical security vulnerability because as soon as this code is checked into the repository, an attacker can use this password to gain unauthorized access to the database.
As the developer is writing the code, after several seconds, Amazon Q highlights the method call. If you hover the cursor over the highlighted code, a detection message generated by the security scan is provided in an informational window. It includes the link to the specific Common Weakness Enumeration (CWE) associated with the vulnerability and the detector library used. It may also include a code fix, if available.
Scan as you code
While the auto-scan feature is exclusive to Amazon Q Developer Pro Tier, the feature to run scans manually is available in both the Pro Tier and the Free Tier. You can evaluate the entire codebase by selecting the Run Project Scan option in the Amazon Q Developer menu. This runs all the detectors on your project.
Scan your project
Once Amazon Q has completed the scan of the active project and its dependencies, a list of all the vulnerabilities appears in a new tab named Amazon Q Security Issues. Selecting an item from the list will open the file where the vulnerability was found and cursor is placed on the location of the issue in the codebase. In this case, the hard-coded password in the file is highlighted. When you hover your cursor on the highlighted issue, it will bring up a window with information on the detected vulnerability including the CWE, in this case CWE-798, and options of how to resolve the problem.
Locate the code where security vulnerability is detected
By selecting the Amazon Q: Explain option in the information window, Amazon Q will explain the vulnerability in detail. This will help your understanding of what the flaw is, how it may be harmful, and may provide advice for fixing the issue. Here, we see Amazon Q describing the vulnerability, and following up with a suggestion to pull the password from environment variables along with an explanation about how this circumvents the issue.
Explanation of the security vulnerability
When Amazon Q has a supported remediation for the detected vulnerability, it is indicated by the green Yes under Code fix available in the information window. The Suggested code fix preview section displays all the code changes that will be made in the file as part of the suggested fix. When code fix is available for the detected vulnerability, you can review what the fix will entail with a code difference image. Once you are satisfied with the changes, you can ask Amazon Q to Apply fix by selecting the button. This will apply the suggested patch automatically into your code. In the current example, Amazon Q Developer replaces the hard-coded password with an access to environment variables, keeping the value secret when uploaded to a code repository.
Fix the code to remediate the security vulnerability
Detection Accuracy & Benchmarking
We discuss the correctness of detectors through the lens of false positives and false negatives; a false positive is when the detector claims a vulnerability exists where one does not, and a false negative means a vulnerability does exist, but the detector did not find it. We use the two fundamental metrics, Precision and Recall, to evaluate the performance of security scans by Amazon Q. Precision measures how correct, or precise, are the positive predictions. A precision of 1.0 means there are no false positives. In other words, precision evaluates the question that of all the vulnerabilities detected by a detection tool, how many vulnerabilities did actually exist? Recall, on the other hand, measures the correct prediction, or recall, of all the actual positive instances in the dataset. A recall of 1.0 means there are no false negatives. In other words, recall answers the question that of all the existing vulnerabilities, how many were correctly identified by the detection tool?
The security detectors in Amazon Q are created with a bias for precision without sacrificing recall. This means it aims to have a low false positive rate first and foremost but still tries to minimize the number of false negatives. We evaluate the security detectors of Amazon Q Developer against other state-of-the-art detection tools that permit benchmarking. The test includes running all these detection tools on both respected datasets and projects such as OWASP Top 10, RailsGoat, WebGoat, and CredData in addition to our own internal datasets.
With a bias for precision, Amazon Q accomplishes that goal by exceeding or matching the precision of top benchmarkable detection tools. As the table shows, Amazon Q surpasses other state of the art detection tools in terms of precision on all the benchmarks, and surpasses them on recall in two of the four benchmarks. This means that when Amazon Q detects a vulnerability in the code, it is correct in a larger percentage of instances as compared to other tools. At the same time, Amazon Q is able to detect a large percentage of all existing vulnerabilities in a code.
Some comparisons on notable public benchmarks:
| Benchmark | Language | Metric | Amazon Q code scanning score | Best competitor code scanning score |
| OWASP Top Rules | Java | Precision | 84.7 | 75.7 |
| Recall | 100 | 92.1 | ||
| RailsGoat | Ruby | Precision | 100 | 100 |
| Recall | 13.3 | 26.6 | ||
| WebGoat | Java | Precision | 100 | 100 |
| Recall | 28.7 | 28.7 | ||
| CredData | All | Precision | 88.2 | 82.3 |
| Recall | 83.3 | 77.7 |
In addition to these popular benchmarks, we also create internal benchmarks for all our supported languages. These internal benchmarks include various public datasets, such as the examples in the table above. In addition, we add to these internal benchmarks the publicly available test datasets that other state-of-the-art detection tools share to evaluate their detectors. Further, we enhance these internal benchmarks using internal findings that we identify as false positives or false negatives. These additions help us make our coverage of vulnerabilities more complete. We then periodically benchmark against similar detection tools, to verify that the quality of our detection is consistently high. Similar to the table above, we surpass other state-of-the-art detection tools in terms of precision on all our internal benchmarks and surpass them on recall in ten of the thirteen benchmarks.
Conclusion
The security scanning feature of Amazon Q Developer enables developers to remediate security vulnerabilities in the codebase. Developers get the recommendations and information within their working environment in the IDE, enabling them to integrate secure coding practices early in the software development lifecycle. Developers can proactively scan their existing code using Amazon Q and remediate the security vulnerabilities found in the code. As for the new code, Amazon Q will auto-scan the code as it is being written in the IDE and provide recommendations for remediation and in some cases a code fix as well. The natural language explanation of the detected vulnerabilities helps developers understand secure coding practices and its impact. Amazon Q helps developers achieve their objective of developing secure and resilient software applications and reduces the associated risk and cost of addressing security incidents later in the software lifecycle.
To learn more about Amazon Q Developer, see the following resources:
About the authors
