Hello folks, In this post we’ll discuss about Sonarqube which is an open source platform developed by SonarSource for the purpose of continuous inspection of code quality to perform automatic reviews with static analysis of code to detect bugs, code smells and security vulnerabilities. The source code for the example project is available here.
Why SonarQube?
In software development life cycle Code Review system plays a crucial role.
Delivering high-quality code is the topmost responsibility of all the developers. To ensure this, developers will again and again refactor the code .The delivering must be reliable and threat free. Achieving code reviews manually is not at all a best practice. Because it is more time consuming process.
To overcome this situation and deliver high-quality code we need a tool to Automatic code review with every release.
What exactly SonarQube do?
- Sonarqube performs static code analysis with a predefined set of rules under four domains.
- Sonarqube reduce the time and effort and also ensures a high-level code quality and performance for large complex applications.
- It identifies the bugs, security threats, code smells and vulnerabilities before the release of an application.
- Let the work flow(in case of CI/CD).
- Performs Security analysis.
- As of now sonarqube can generate code quality report for 27 languages.
Architectural Flow:
The SonarQube Platform is made of 4 components:
- SonarQube Server :
- This is where the Sonarqube compute engine works and takes the input from UI.
- It consists of three parts
- i. Web Server : for developers, managers to browse quality snapshots and configure the SonarQube instance
- ii. Search Server : It is based on Elasticsearch to back searches from the UI.
- iii. Compute Engine Server in charge of processing code analysis reports and saving them in the SonarQube Database
2. SonarQube Database to store:
- The configuration of the SonarQube instance (security, plugins settings, etc.)
- the quality snapshots of projects, views, etc.
- We can also configure our local Databases to store the generated reports.
3. Multiple SonarQube Plugins installed on the server, possibly including language, SCM, integration, authentication, and governance plugins
4. One or more SonarQube Scanners running on your Build / Continuous Integration Servers to analyze projects.
What is Static Code Analysis:
Static Code Analysis address the weaknesses in source code that might lead to bugs, vulnerabilities and security hotspots.
This analysis process can also be achieved manually, but it takes more time and the result may or may not reliable.
Static Code Analysis in Sonarqube commonly follows the rules defined in different domains like maintainability, security, reliability domains.
Static code analysis stage comes in DevOps before the release of a software version, so that the developers can know the in early if there are any threats in the code.
- Static code analysis is very fast and it does automated reviews for developers.
- Testing may or may not cover every possible execution path whereas Static code analyzer can go in depth an covers all parts of the code.
- Manual code review system is prone to errors but a static code analyzer gives a high-level quality code without any threats and errors.
Rules in Sonarqube:
Rule: A Rule is a coding standard or practice which should be followed. Not complying with coding rules leads to Bugs, Vulnerabilities, Code smells and Security Threats.
Rules in sonarqube are categorized into three based on there respective domains.
- Reliability Domain(Bugs):
- A Bug is an issue that represents something wrong in the code. If this hasn’t broken yet, it will and probably at the worst possible moment.
- It is actually a coding error.
- It will be caused by incorrect or erroneous logic.
- A problem is said to be a buggy when it contains large no.of bugs.
- Examples:
- Arithmetic & Exception handling cases,Division by zero, Arithmetic overflow or Underflow.
- Infinite loop and infinite recursion
- Deadlocks, Starvation, Race Condition
2. Maintainability Domain(Code Smells):
- Code smells are certain structures in the code that indicate the violation of fundamental design principles and negatively impact the design quality.
- Code smells are usually not bugs; They aren’t technically incorrect and don’t prevent the program from functioning. Instead, they indicate the weakness in the design that may slow down the development or increase the risk of bugs or failures in the future.
- Bad code smells can be indicator of factors that contribute to Technical Debt.
- Technical Debt: The estimated time it will take to fix all the code smells. To avoid this code should be refactored.
- Examples:
- Duplicated code, Defining large classes, lazy classes(which does too little)
- Method level: Too many parameters, long method, Excessive return of data.
- Contrived Complexity: Forced usage of over complicated design patterns where simpler would suffice.
- Shotgun Surgery: A single change needs to be applied to multiple classes at the same time.
3. Security Domain(Vulnerabilities & Security Hotspots):
- Vulnerability: A security related issue which represents a backdoor for attackers.
- Vulnerabilities are points in the code which are open to attack.
- Security Hotspot: It is a security sensitive pieces of code that need to be manually reviewed. Upon review, you’ll either find that there is no threat or there exists a vulnerable code which needs to be fixed.
- A vulnerability rule highlights the security threats only when it has a high level of confidence.
- Where as a Hotspot rule guides secure code reviews by showing code where those issues might lurk, even if it could not detect any vulnerability.
- The main goal of security Hotspot is to help focus the efforts of the security auditors who manually review the application source code.
- The second goal is to educate the developers and increase their security awareness.
Prerequisites:
- openjdk 11
- The only prerequisite for running SonarQube is to have Java (Oracle JRE 11 or OpenJDK 11) installed on your machine.
2. SonarQube
- Download and install the latest version of sonarqube here. There are a total of four editions available. Each edition provides differenet kind of features. Out of all community edition is available for free.
3. SonarScanner
- SonarScanner is the one which does the actual process of scanning the projects and generate the reports. Download the sonarscanner from here.
- There are several kinds SonarScanners available for different build systems like Sonarscanners for Maven, MSBuild, Gradle, Ant, Azure Devops, Jenkins.
- If our project is not specific to any of this build system., we can go ahead with common SonarScanner.
Adding Paths to Environmental Variables:
- We need to add the paths of SonarQube platform and SonarScanner to our system/user Environment Variables in order to access the batch files or exe files from anywhere through terminal.
- Inside SonarQube\bin folder , nested folders are available for different OS, choose accordingly and set the path.
Sonarqube Quality Gate:
- Sonarqube Quality Gate is defined as a set of threshold measures set on our projects like Security Rating, Code Coverage, Maintainability Rating , Reliability Rating etc..
- Using Quality Gates is the best approach to ensure that the coding standards are met in all of our projects.
- When SonarQube runs it will check if the code meets all the quality thresholds which we have set in a Quality Gate and is applied to our project, else it will fail the Quality Gate and will not allow you to check in code to source control.
- This is a very powerful feature since it enforces code quality in your projects and gives reviews to the Developers during the CI/CD process. Based on the Quality Gate output CI/CD process will decide to Abort the all operations or continue to execute. Want to know how this happens?, Checkout my ci/cd article here.
- The output of the Quality Gate is Passed or Failed.
Code Coverage:
- It is a metric used to check the quality of testcases. Basically it represents the percentage of production code that has been tested successfully.
- There are different plugins for each language to generate code coverage report.
- Example : Jacoco (for java, maven, gradle)
Example Project:
- Here we have an example project to understood about how the SonarQube platform will generate the code quality report.
- To achieve this, we need to write a properties file called “sonar-project.properties”
- sonar-project.properties:
sonar.projectKey=demoapp
sonar.projectName= ProjectName
sonar.projectVersion=1.0
sonar.sources=.
sonar.sourceEncoding=UTF-8sonar.host.url=http://localhost:9000
sonar.tests=src/test/
sonar.coverage.jacoco.xmlReportPaths=target/jacoco/test/jacoco.xml,target/jacoco/integrationTest/jacoco.xmlsonar.java.codeCoveragePlugin=jacocosonar.junit.reportPaths=target/test-results/test,target/test-results/integrationTestsonar.testExecutionReportPaths=target/test-results/jest/TESTS-results-sonar.xmlsonar.exclusions=src/main/webapp/content/**/*.*, src/main/webapp/i18n/*.js, target/classes/static/**/*.*
- Let’s Understand the file,
- sonar.projectKey refers to the project key
- sonar.projectName the name of our project which will be displayed in the sonarqube dashboard.
- sonar.projectVersion is to identify the release version of the project.
- sonar.sources is to tell the SonarScanner about which files it should scan and generate the report. path “.” represent the current working directory.
- sonar.sourceEncoding is for mentioning Encoding which is to be UTF-8.
- These five properties are mandatory for any project.
- Remaining all optional and based on your project.
- sonar.host.url is by default works on 9000 port , it is where the generated report will be represented.
- sonar.tests → To mention the place where we wrote the testcases foru our project.
- sonar.coverage.jacoco → To generate the code coverage report.
- sonar.java.codeCoveragePlugin → code coverage generating plugin name.
- sonar.junit.reportPaths & testExecutionReportPaths → Test Results paths
- We may often required to exclude some files which are not part of code quality , sonar.exclusions property is useful to exclude those files.
Report Generation:
- Start the Sonarqube by hitting StartSonar batch file from terminal. Just type in “StartSonar” to start it. Then sonarqube will start on it’s default port 9100. Go to browser and enter “localhost:9100” to view the dashboard.
- open the terminal inside the project folder where the sonar-project.properties files of that project is placed.
- Type in the command “sonar-scanner” to scan the project using the sonar-project file configuration details.
- Here I am taking a sample spring-boot project and generating the report for it.
- After it’s scanning , if it gives the result as Execution success, then it means that the report is successfully, else it means that there is something problem in your code or in configuration file(sonar-project.props)
- Go to the dashboard now and see the results of the generated report.
- In the Dashboard we can observer that 3 project have been passed the quality gate and is indicated with the status “passed”. One project hasn’t been qualified the quality gate since, it didn’t met threshold measures set by me.
- We can also observe that it showing the No.of bugs, Code smells,Duplications %, Hotspots etc… It also showing the Ratings for each domain there, to understand this look at the left side of the image also. Here, Code Coverage is 0.0% since I didn’t wrote test cases for my project.
- Click on your project to check where the issues have generated. It will also shows us the Debt, which is the expexted time to resolve thise issues.
- Here we can see the issues for overall code as well as for new code. If we go to down we can observer an activity chart of all versions of our project.
- To findout the issues and fix them click on any issue like bug or vulnerabilities. We’ll be redirected to a page where all the issues will be mentioned and it also provides the suggestions to fix them.
- This report will be generated based on the rules defined inside a Quality Profile. Each Quality profile consists of number of rules for the respective language.
Setting Up a New SonarQube Quality Gate:
- By default sonarqube provides a default Quality Gate called as Sonar way and is applied to all the projects. We can create new one and set to desired projects.
- In practical world, there quality gate criteria will defer so that we need to setup new one according to the threshold measures for the generated metrics.
- Here, we have setup a new quality gate on Overall code and on New code by adding conditions on metrics.
- Whenever the sonar-scanner runs the project, the mentioned Quality gate will be passed onto the project to get the Status of the Quality gate.
That’s it about the Sonarqube setup and execution guys, we can also write our own rules for our projects and can be embedded into our personal sonarqube or can also be released onto community. We’ll discuss about this concept in the next article.
For any queries on this article do comment here or reach my out at narsimhulu.464@gmail.com and at LinkedIn B Narsimha.
Thank You!!!