Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Branching is broken (again) when using a project name Groovy script #1312

Closed
nleach999 opened this issue Jan 30, 2024 · 1 comment
Closed

Branching is broken (again) when using a project name Groovy script #1312

nleach999 opened this issue Jan 30, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@nleach999
Copy link
Contributor

Description

Bug #1086 described a problem where a project naming script is used. A specific issue related to project naming appears to have been fixed in Bug #1086, but the root cause appears to not have been fixed.

In this reported issue, the protected branch has a / in the name. When checkmarx.cx-branch: true is set, there is an attempt to find the parent project but the name of the project is not generated by the project name script so it fails.

The way this is described to recreate uses a branch named release/1.0 and a project naming Groovy script that converts / to _.

Keep in mind that this issue and Bug #1086 described this as a problem for any case where the naming script modifies the name of the project, so the real issue is that names for projects for all SAST requests involving project names should be generated using the same logic that is used for scanning (i.e. using the project naming script if it is configured).

Expected Behavior

The parent project should be detected/created properly by following the naming convention implemented in the Groovy project name script.

Actual Behavior

The parent project for a branch project is not found, causing the entire scan to fail since the branch project has parent project id of -1.

Reproduction

  1. Use CxFlow in webhook mode with checkmarx.cx-branch: true and the project naming groovy script:
if (request.project != null)
    return request.project.replace('/', '_')
else
    return request.project

  1. Your SAST system should have no projects when you open the PR in a later step so that this is demonstrated properly.

  2. My config-as-code file in the root of my repo is like so:

{
	"version": 1.0,
	"project" : "SimplyVulnerable-[${branch}]",
	"team": "/CxServer",
	"sast": {
		"preset": "Checkmarx Default",
		"engineConfiguration": "Default Configuration",
		"incremental": "false",
		"forceScan": "true"
	},
	"additionalProperties": {
		"cxFlow": {
			"application": "Simply Vulnerable Test",
			"branches": ["demo-master", "release/1.0"],
			"jira": {
				"project": "SV"
			}
		}
	},
	"sca" : {
	  "team": "/CxServer/SCA-PM/Champions/NA-Partners"
	},
	 "customFields": {
	    "InventoryId": "ABC123"
	  }
}

The important part to note is the protected branches and the project name where the branch name is embedded into the project name by the config-as-code.

  1. Create the protected branch release/1.0 in your demo repository. You can use my example project if needed, but any project will suffice as long as it has the correct config-as-code file in the repo root.

  2. Create a branch named feature/1.0_myfeature, make a change to it, open a PR targeting release/1.0.

This is the log of the PR request handling:

cxflow-webhook_1  | 2024-01-30 19:52:05.326  INFO 17 --- [nio-8585-exec-1] c.c.f.c.GitHubController                  [igGEvMnd] : Processing GitHub PULL request
cxflow-webhook_1  | 2024-01-30 19:52:05.500  INFO 17 --- [nio-8585-exec-1] c.c.c.RemoteRepoDownloader                [igGEvMnd] : Searching for a config-as-code file in a remote repo
cxflow-webhook_1  | 2024-01-30 19:52:06.110  INFO 17 --- [nio-8585-exec-1] c.c.f.c.GitHubController                  [igGEvMnd] : Signature verified
cxflow-webhook_1  | 2024-01-30 19:52:06.115  INFO 17 --- [nio-8585-exec-1] c.c.f.c.GitHubController                  [igGEvMnd] : Using url: https://github.com/nleach999/SimplyVulnerable.git
cxflow-webhook_1  | 2024-01-30 19:52:06.463  INFO 17 --- [nio-8585-exec-1] c.c.f.s.ScaFilterFactory                  [igGEvMnd] : Initializing SCA filters.
cxflow-webhook_1  | 2024-01-30 19:52:06.493  INFO 17 --- [nio-8585-exec-1] c.c.f.s.ConfigurationOverrider            [igGEvMnd] : The following properties were overridden by config-as-code file: {scan configuration=Default Configuration, force scan=true, application=Simply Vulnerable Test, active branches=[demo-master, release/1.0], project=SimplyVulnerable-[feature/1.0_myfeature], includeSources=false, team=/CxServer/SCA-PM/Champions/NA-Partners, incremental=false, scan preset=Checkmarx Default}
cxflow-webhook_1  | 2024-01-30 19:52:06.508  INFO 17 --- [      flow-web1] c.c.f.s.ExternalScriptService             [igGEvMnd] : executing external script to determine the project in Checkmarx to be used (name.groovy)
cxflow-webhook_1  | 2024-01-30 19:52:07.102  INFO 17 --- [      flow-web1] c.c.f.s.ExternalScriptService             [igGEvMnd] : executing external script to determine the project in Checkmarx to be used (name.groovy)
cxflow-webhook_1  | 2024-01-30 19:52:07.137  INFO 17 --- [      flow-web1] c.c.f.s.ProjectNameGenerator              [igGEvMnd] : Project name (SimplyVulnerable-[feature_1.0_myfeature]) has not been normalized.
cxflow-webhook_1  | 2024-01-30 19:52:07.138  INFO 17 --- [      flow-web1] c.c.f.s.ProjectNameGenerator              [igGEvMnd] : Project name being used: SimplyVulnerable-[feature_1.0_myfeature]
cxflow-webhook_1  | 2024-01-30 19:52:07.143  INFO 17 --- [      flow-web1] c.c.f.s.AbstractVulnerabilityScanner      [igGEvMnd] : --------------------- Initiating new sast scan ---------------------
cxflow-webhook_1  | 2024-01-30 19:52:07.154  INFO 17 --- [      flow-web1] c.c.f.s.EmailService                      [igGEvMnd] : cx-flow.mail.notification not set or set to false. Skipping Scan Submitted e-mail...
cxflow-webhook_1  | 2024-01-30 19:52:07.155  INFO 17 --- [      flow-web1] c.c.f.s.ScanRequestConverter              [igGEvMnd] : Overriding team with /CxServer
cxflow-webhook_1  | 2024-01-30 19:52:07.155  INFO 17 --- [      flow-web1] c.c.s.s.CxAuthService                     [igGEvMnd] : Logging into Checkmarx http://sast9x.cxsecure.nl/cxrestapi/auth/identity/connect/token
cxflow-webhook_1  | 2024-01-30 19:52:07.406  INFO 17 --- [      flow-web1] c.c.s.s.CxService                         [igGEvMnd] : Retrieving Cx teams
cxflow-webhook_1  | 2024-01-30 19:52:07.465  INFO 17 --- [      flow-web1] c.c.s.s.CxService                         [igGEvMnd] : Found team /CxServer with ID 1
cxflow-webhook_1  | 2024-01-30 19:52:07.524  INFO 17 --- [      flow-web1] c.c.f.s.AbstractVulnerabilityScanner      [igGEvMnd] : Checking if there is any existing scan for Project: -1
cxflow-webhook_1  | 2024-01-30 19:52:07.575  INFO 17 --- [      flow-web1] c.c.s.s.CxService                         [igGEvMnd] : No scans in the queue that are in progress
cxflow-webhook_1  | 2024-01-30 19:52:07.575  INFO 17 --- [      flow-web1] c.c.s.s.CxService                         [igGEvMnd] : Creating scan...

Here is where it tries to create the parent project and fails because the name is not generated from the project name Groovy script:

cxflow-webhook_1  | 2024-01-30 19:52:07.686  INFO 17 --- [      flow-web1] c.c.s.s.CxService                         [igGEvMnd] : Creating Project SimplyVulnerable-[release/1.0] for ownerId 1
cxflow-webhook_1  | 2024-01-30 19:52:07.739 ERROR 17 --- [      flow-web1] c.c.s.s.CxService                         [igGEvMnd] : HTTP error code 400 BAD_REQUEST while creating project with name SimplyVulnerable-[release/1.0] under owner id 1
cxflow-webhook_1  | 2024-01-30 19:52:07.742 ERROR 17 --- [      flow-web1] c.c.s.s.CxService                         [igGEvMnd] : org.springframework.web.client.HttpClientErrorException$BadRequest: 400 Bad Request: "{<EOL><EOL>  "message": "The request is invalid.",<EOL><EOL>  "modelState": {<EOL><EOL>    "project.Name": [<EOL><EOL>      "The field Name is invalid."<EOL><EOL>    ]<EOL><EOL>  }<EOL><EOL>}"
cxflow-webhook_1  |     at org.springframework.web.client.HttpClientErrorException.create(HttpClientErrorException.java:101)
cxflow-webhook_1  |     at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:168)
cxflow-webhook_1  |     at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:122)
cxflow-webhook_1  |     at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63)
cxflow-webhook_1  |     at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:819)
cxflow-webhook_1  |     at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:777)
cxflow-webhook_1  |     at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:711)
cxflow-webhook_1  |     at org.springframework.web.client.RestTemplate.postForObject(RestTemplate.java:437)
cxflow-webhook_1  |     at com.checkmarx.sdk.service.CxService.createProject(CxService.java:1045)
cxflow-webhook_1  |     at com.checkmarx.sdk.service.CxService.createScan(CxService.java:1879)
cxflow-webhook_1  |     at com.checkmarx.flow.service.AbstractVulnerabilityScanner.scan(AbstractVulnerabilityScanner.java:98)
cxflow-webhook_1  |     at com.checkmarx.flow.service.FlowService.lambda$runScanRequest$3(FlowService.java:64)
cxflow-webhook_1  |     at java.util.ArrayList.forEach(ArrayList.java:1259)
cxflow-webhook_1  |     at com.checkmarx.flow.service.FlowService.runScanRequest(FlowService.java:62)
cxflow-webhook_1  |     at com.checkmarx.flow.service.FlowService.initiateAutomation(FlowService.java:41)
cxflow-webhook_1  |     at com.checkmarx.flow.service.FlowService$$FastClassBySpringCGLIB$$3503db27.invoke(<generated>)
cxflow-webhook_1  |     at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
cxflow-webhook_1  |     at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
cxflow-webhook_1  |     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
cxflow-webhook_1  |     at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
cxflow-webhook_1  |     at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58)
cxflow-webhook_1  |     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
cxflow-webhook_1  |     at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
cxflow-webhook_1  |     at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
cxflow-webhook_1  |     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
cxflow-webhook_1  |     at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
cxflow-webhook_1  |     at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115)
cxflow-webhook_1  |     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
cxflow-webhook_1  |     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
cxflow-webhook_1  |     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
cxflow-webhook_1  |     at java.lang.Thread.run(Thread.java:750)
cxflow-webhook_1  |

It then tries to create the branch project (properly named by the Groovy script) but fails since it won't branch from parent project -1 due to the previous failure. The scan is then reported as failed:

cxflow-webhook_1  | 2024-01-30 19:52:07.743  INFO 17 --- [      flow-web1] c.c.s.s.CxService                         [igGEvMnd] : Creating branched project with name 'SimplyVulnerable-[feature_1.0_myfeature]' from existing project with ID -1
cxflow-webhook_1  | 2024-01-30 19:52:07.795 ERROR 17 --- [      flow-web1] c.c.s.s.CxService                         [igGEvMnd] : HTTP error code 400 BAD_REQUEST while creating branched project with name 'SimplyVulnerable-[feature_1.0_myfeature]' from existing project with ID -1
cxflow-webhook_1  | 2024-01-30 19:52:07.796 ERROR 17 --- [      flow-web1] c.c.s.s.CxService                         [igGEvMnd] : org.springframework.web.client.HttpClientErrorException$BadRequest: 400 Bad Request: "{<EOL><EOL>  "messageCode": 47102,<EOL><EOL>  "messageDetails": "Project with id -1 was not found"<EOL><EOL>}"
cxflow-webhook_1  |     at org.springframework.web.client.HttpClientErrorException.create(HttpClientErrorException.java:101)
cxflow-webhook_1  |     at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:168)
cxflow-webhook_1  |     at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:122)
cxflow-webhook_1  |     at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63)
cxflow-webhook_1  |     at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:819)
cxflow-webhook_1  |     at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:777)
cxflow-webhook_1  |     at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:711)
cxflow-webhook_1  |     at org.springframework.web.client.RestTemplate.postForObject(RestTemplate.java:437)
cxflow-webhook_1  |     at com.checkmarx.sdk.service.CxService.branchProject(CxService.java:1084)
cxflow-webhook_1  |     at com.checkmarx.sdk.service.CxService.createScan(CxService.java:1881)
cxflow-webhook_1  |     at com.checkmarx.flow.service.AbstractVulnerabilityScanner.scan(AbstractVulnerabilityScanner.java:98)
cxflow-webhook_1  |     at com.checkmarx.flow.service.FlowService.lambda$runScanRequest$3(FlowService.java:64)
cxflow-webhook_1  |     at java.util.ArrayList.forEach(ArrayList.java:1259)
cxflow-webhook_1  |     at com.checkmarx.flow.service.FlowService.runScanRequest(FlowService.java:62)
cxflow-webhook_1  |     at com.checkmarx.flow.service.FlowService.initiateAutomation(FlowService.java:41)
cxflow-webhook_1  |     at com.checkmarx.flow.service.FlowService$$FastClassBySpringCGLIB$$3503db27.invoke(<generated>)
cxflow-webhook_1  |     at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
cxflow-webhook_1  |     at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
cxflow-webhook_1  |     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
cxflow-webhook_1  |     at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
cxflow-webhook_1  |     at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58)
cxflow-webhook_1  |     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
cxflow-webhook_1  |     at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
cxflow-webhook_1  |     at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
cxflow-webhook_1  |     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
cxflow-webhook_1  |     at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
cxflow-webhook_1  |     at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115)
cxflow-webhook_1  |     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
cxflow-webhook_1  |     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
cxflow-webhook_1  |     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
cxflow-webhook_1  |     at java.lang.Thread.run(Thread.java:750)
cxflow-webhook_1  | 
cxflow-webhook_1  | 2024-01-30 19:52:07.800 ERROR 17 --- [      flow-web1] c.c.f.s.AbstractVulnerabilityScanner      [igGEvMnd] : SAST scan failed
cxflow-webhook_1  | 
cxflow-webhook_1  | com.checkmarx.sdk.exception.CheckmarxException: Project was not created successfully: SimplyVulnerable-[feature_1.0_myfeature]
cxflow-webhook_1  |     at com.checkmarx.sdk.service.CxService.createScan(CxService.java:1889)
cxflow-webhook_1  |     at com.checkmarx.flow.service.AbstractVulnerabilityScanner.scan(AbstractVulnerabilityScanner.java:98)
cxflow-webhook_1  |     at com.checkmarx.flow.service.FlowService.lambda$runScanRequest$3(FlowService.java:64)
cxflow-webhook_1  |     at java.util.ArrayList.forEach(ArrayList.java:1259)
cxflow-webhook_1  |     at com.checkmarx.flow.service.FlowService.runScanRequest(FlowService.java:62)
cxflow-webhook_1  |     at com.checkmarx.flow.service.FlowService.initiateAutomation(FlowService.java:41)
cxflow-webhook_1  |     at com.checkmarx.flow.service.FlowService$$FastClassBySpringCGLIB$$3503db27.invoke(<generated>)
cxflow-webhook_1  |     at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
cxflow-webhook_1  |     at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
cxflow-webhook_1  |     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
cxflow-webhook_1  |     at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
cxflow-webhook_1  |     at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58)
cxflow-webhook_1  |     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
cxflow-webhook_1  |     at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
cxflow-webhook_1  |     at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
cxflow-webhook_1  |     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
cxflow-webhook_1  |     at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
cxflow-webhook_1  |     at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115)
cxflow-webhook_1  |     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
cxflow-webhook_1  |     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
cxflow-webhook_1  |     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
cxflow-webhook_1  |     at java.lang.Thread.run(Thread.java:750)
cxflow-webhook_1  | 
cxflow-webhook_1  | 2024-01-30 19:52:07.887  INFO 17 --- [      flow-web1] jsonLogger                                [igGEvMnd] : 
cxflow-webhook_1  | 2024-01-30 19:52:07.952  INFO 17 --- [  scan-results1] c.c.f.s.ResultsService                    [igGEvMnd] : Finished processing the request

Environment Details

Confirmed in 1.6.37 and 1.6.46

CxFlow running in webhook mode

Confirmed with both BitBucket Enterprise and GitHub cloud.
@nleach999 nleach999 added the bug Something isn't working label Jan 30, 2024
@satyamchaurasiapersistent
Copy link
Contributor

Fixed in Release 1.7.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@nleach999 @satyamchaurasiapersistent