Synth NodejsFunction: Error: ENOTEMPTY: directory not empty, rmdir

[rupe120]

Describe the bug

cdk synth fails if a prior synth exists in cdk.out with the error:

Error: ENOTEMPTY: directory not empty, rmdir '\\?\C:\CodeCommit\project\group-document-processing\cdk.out\bundling-temp-dc725a4c4531f07cc12d4a763596738b6f71449ea4146dc0c69217eb5b7c5f27'
    at Object.rmdirSync (node:fs:1219:10)
    at _rmdirSync (node:internal/fs/rimraf:260:21)
    at rimrafSync (node:internal/fs/rimraf:193:7)
    at Object.rmSync (node:fs:1268:10)
    at Object.removeSync (C:\CodeCommit\project\group-document-processing\node_modules\aws-cdk-lib\node_modules\fs-extra\lib\remove\index.js:11:6)   
    at AssetStaging.stageAsset (C:\CodeCommit\project\group-document-processing\node_modules\aws-cdk-lib\core\lib\asset-staging.js:1:6427)
    at AssetStaging.stageByBundling (C:\CodeCommit\project\group-document-processing\node_modules\aws-cdk-lib\core\lib\asset-staging.js:1:5777)      
    at stageThisAsset (C:\CodeCommit\project\group-document-processing\node_modules\aws-cdk-lib\core\lib\asset-staging.js:1:2728)
    at Cache.obtain (C:\CodeCommit\project\group-document-processing\node_modules\aws-cdk-lib\core\lib\private\cache.js:1:242)
    at new AssetStaging (C:\CodeCommit\project\group-document-processing\node_modules\aws-cdk-lib\core\lib\asset-staging.js:1:3125) {
  errno: -4051,
  syscall: 'rmdir',
  code: 'ENOTEMPTY',
  path: '\\\\?\\C:\\CodeCommit\\project\\group-document-processing\\cdk.out\\bundling-temp-dc725a4c4531f07cc12d4a763596738b6f71449ea4146dc0c69217eb5b7c5f27'

Regression Issue

• Select this option if this issue appears to be a regression.

Last Known Working CDK Version

Unknown, I tried prior versions and was not able to find one that worked. This was working for me about a week ago and then suddenly stopped.

Expected Behavior

A successful synth

Current Behavior

See description

Reproduction Steps

I have not had the time to rebuild a project to see if it happens with the lambda definition in a fresh project.

This is the lambda definition though:

    const testGetTextractTextDetectionLambda = new NodejsFunction(this, 'test-get-textract-text-detection',
    {
      codeSigningConfig: props.currentEnvironment.deployEnvironment === "dev" ? undefined : codeSigningConfig,
      functionName:`test-get-textract-text-detection-${props.currentEnvironment.deployEnvironment}`,
      description: `Get Textract text detection`,
      runtime: Runtime.NODEJS_20_X,
      entry: 'lib/lambda-stack.test-get-from-textract-text-detection.ts',
      handler: "lambdaHandler",
      timeout: cdk.Duration.minutes(15),
      tracing: Tracing.ACTIVE,
      insightsVersion: LambdaInsightsVersion.VERSION_1_0_143_0,
      deadLetterQueue: dlQueue,
      deadLetterQueueEnabled: true,
      memorySize: 1024,
      environment: {
        REGION_ID: props.env!.region!,
        SOURCE_BUCKET_PATH: props.currentEnvironment.sourceBucketPrefix,
        DOCUMENT_DATE_ANALYSIS_QUESTION_ALIAS: documentDateAnalysisQuestionAlias,
        DOCUMENT_DATE_ANALYSIS_QUESTION: props.appConfig.documentDateAnalysisQuestion,
        EMPLOYER_NAME_ANALYSIS_QUESTION_ALIAS: employerNameAnalysisQuestionAlias,
        EMPLOYER_NAME_ANALYSIS_QUESTION: props.appConfig.employerNameAnalysisQuestion,
        CHAMBER_OF_COMMERCE_ANALYSIS_QUESTION_ALIAS: chamberOfCommerceAnalysisQuestionAlias,
        CHAMBER_OF_COMMERCE_ANALYSIS_QUESTION: props.appConfig.chamberOfCommerceAnalysisQuestion,
        ENVIRONMENT: props.currentEnvironment.deployEnvironment,
        NODE_ENV: "test"
      }
    });
    sourceBucket.grantReadWrite(testGetTextractTextDetectionLambda);
    testGetTextractTextDetectionLambda.addToRolePolicy(new PolicyStatement(
      { 
        actions: [
          'textract:GetDocumentTextDetection'
        ],
        resources: ['*']
      }
    ));

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.166.0

Framework Version

2.166.0

Node.js Version

v20.10.0

OS

Windows 11 Enterprise 22621.4169

Language

TypeScript

Language Version

4.9.5

Other information

No response

[khushail]

Hi @rupe120 , thanks for reaching out.

I tried to repro the issue with this minimal code of Lambda node js function and its succeeding with synth as well as deployment. Sharing the details -

Code -

const dlQueue = new sqs.Queue(this, 'dlQueue', {
      queueName: 'dlQueue'
    });

    const testGetTextractTextDetectionLambda = new NodejsFunction(this, 'test-get-textract-text-detection',
    {
      codeSigningConfig: undefined,
      functionName:'test-get-textract-text-detection',
      description: `Get Textract text detection`,
      runtime: Runtime.NODEJS_20_X,
      timeout: cdk.Duration.minutes(15),
      entry: 'lib/index.ts',
      handler: 'index.handler',
      tracing: Tracing.ACTIVE,
      insightsVersion: LambdaInsightsVersion.VERSION_1_0_143_0,
      deadLetterQueue: dlQueue,
      deadLetterQueueEnabled: true,
      memorySize: 1024,
      environment: {
        NODE_ENV: "test"
      }
    });
    const sourceBucket = new s3.Bucket(this, 'sourceBuckettestDeploy', {
      bucketName: 'source-bucket-test-deploy',
      removalPolicy: cdk.RemovalPolicy.DESTROY
    });

    sourceBucket.grantReadWrite(testGetTextractTextDetectionLambda);
    testGetTextractTextDetectionLambda.addToRolePolicy(new PolicyStatement(
      { 
        actions: [
          'textract:GetDocumentTextDetection'
        ],
        resources: ['*']
      }
    ));
    //added these statement after 1st synth
    new cdk.CfnOutput(this, 'testGetTextractTextDetectionLambda', { value: testGetTextractTextDetectionLambda.functionArn });
    new cdk.CfnOutput(this, 'sourceBucket', { value: sourceBucket.bucketName });
  }

the code synthesized successfully and I am able to see the cdk diff as well -

used cdk version - 2.166.0 Node - v20.17.0

As I am not able to repro the scenario, could you please share the minimal self contained code to repro the issue.
Also I would suggest you to check the NodeJS version and CDK version as well.

[github-actions]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[rupe120]

Hi @rupe120 , thanks for reaching out.

I tried to repro the issue with this minimal code of Lambda node js function and its succeeding with synth as well as deployment. Sharing the details -

Code -

const dlQueue = new sqs.Queue(this, 'dlQueue', {
      queueName: 'dlQueue'
    });

    const testGetTextractTextDetectionLambda = new NodejsFunction(this, 'test-get-textract-text-detection',
    {
      codeSigningConfig: undefined,
      functionName:'test-get-textract-text-detection',
      description: `Get Textract text detection`,
      runtime: Runtime.NODEJS_20_X,
      timeout: cdk.Duration.minutes(15),
      entry: 'lib/index.ts',
      handler: 'index.handler',
      tracing: Tracing.ACTIVE,
      insightsVersion: LambdaInsightsVersion.VERSION_1_0_143_0,
      deadLetterQueue: dlQueue,
      deadLetterQueueEnabled: true,
      memorySize: 1024,
      environment: {
        NODE_ENV: "test"
      }
    });
    const sourceBucket = new s3.Bucket(this, 'sourceBuckettestDeploy', {
      bucketName: 'source-bucket-test-deploy',
      removalPolicy: cdk.RemovalPolicy.DESTROY
    });

    sourceBucket.grantReadWrite(testGetTextractTextDetectionLambda);
    testGetTextractTextDetectionLambda.addToRolePolicy(new PolicyStatement(
      { 
        actions: [
          'textract:GetDocumentTextDetection'
        ],
        resources: ['*']
      }
    ));
    //added these statement after 1st synth
    new cdk.CfnOutput(this, 'testGetTextractTextDetectionLambda', { value: testGetTextractTextDetectionLambda.functionArn });
    new cdk.CfnOutput(this, 'sourceBucket', { value: sourceBucket.bucketName });
  }

the code synthesized successfully and I am able to see the cdk diff as well -

used cdk version - 2.166.0 Node - v20.17.0

As I am not able to repro the scenario, could you please share the minimal self contained code to repro the issue.
Also I would suggest you to check the NodeJS version and CDK version as well.

This seems to be a Windows issue. It works fine, using the same path in WSL

[ashishdhingra]

This seems to be a Windows issue. It works fine, using the same path in WSL
@rupe120 Good afternoon. Apologies for late response. Somehow, I'm unable to reproduce the issue using code code provided by @khushail in #32098 (comment). The CDK code successfully synthesizes after exiting the environment and re-running the cdk synth command without removing the cdk.out directory.

[+] Building 2.0s (16/16) FINISHED                                                                                                                                                                     docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                                                                                   0.0s
 => => transferring dockerfile: 1.62kB                                                                                                                                                                                 0.0s 
 => [internal] load metadata for public.ecr.aws/sam/build-nodejs20.x:latest                                                                                                                                            1.9s 
 => [internal] load .dockerignore                                                                                                                                                                                      0.0s
 => => transferring context: 2B                                                                                                                                                                                        0.0s 
 => [ 1/12] FROM public.ecr.aws/sam/build-nodejs20.x:latest@sha256:561b314d0659c0d3ffa01b1a1fd132386597c34d9202bcd70c8168fc9c4552b5                                                                                    0.0s 
 => CACHED [ 2/12] RUN npm install --global yarn@1.22.5                                                                                                                                                                0.0s 
 => CACHED [ 3/12] RUN npm install --global pnpm@7.30.5                                                                                                                                                                0.0s 
 => CACHED [ 4/12] RUN npm install --global bun@1.1.30                                                                                                                                                                 0.0s 
 => CACHED [ 5/12] RUN npm install --global typescript                                                                                                                                                                 0.0s 
 => CACHED [ 6/12] RUN npm install --global --unsafe-perm=true esbuild@0.21                                                                                                                                            0.0s 
 => CACHED [ 7/12] RUN mkdir /tmp/npm-cache &&     chmod -R 777 /tmp/npm-cache &&     npm config --global set cache /tmp/npm-cache                                                                                     0.0s 
 => CACHED [ 8/12] RUN mkdir /tmp/yarn-cache &&     chmod -R 777 /tmp/yarn-cache &&     yarn config set cache-folder /tmp/yarn-cache                                                                                   0.0s 
 => CACHED [ 9/12] RUN mkdir /tmp/pnpm-cache &&     chmod -R 777 /tmp/pnpm-cache &&     pnpm config --global set store-dir /tmp/pnpm-cache                                                                             0.0s 
 => CACHED [10/12] RUN npm config --global set update-notifier false                                                                                                                                                   0.0s 
 => CACHED [11/12] RUN /sbin/useradd -u 1000 user && chmod 711 /                                                                                                                                                       0.0s 
 => CACHED [12/12] RUN mkdir /tmp/bun-cache &&     chmod -R 777 /tmp/bun-cache &&     echo -e "[install.cache]\ndir = "/tmp/bun-cache"\ndisable = true" >> /home/user/.bunfig.toml                                     0.0s 
 => exporting to image                                                                                                                                                                                                 0.0s 
 => => exporting layers                                                                                                                                                                                                0.0s 
 => => writing image sha256:cba5f17200a36daa93cbf3f0236912934ab34879c38d27adaf5cc416ea24b90d                                                                                                                           0.0s 
 => => naming to docker.io/library/cdk-bd11bc77f379761d3523ff5406f9893601c198eb2a6cb450624d1bf01278b7fd                                                                                                                0.0s 

View build details: docker-desktop://dashboard/build/desktop-linux/desktop-linux/s009j71hbl4e7jyifeddpvjxs

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview 
Bundling asset CdktestStack/test-get-textract-text-detection/Code/Stage...
esbuild cannot run locally. Switching to Docker bundling.

  asset-output/index.js  293b

⚡ Done in 21ms
Resources:
  dlQueue819AE35E:
    Type: AWS::SQS::Queue
    Properties:
      QueueName: dlQueue
    UpdateReplacePolicy: Delete
    DeletionPolicy: Delete
    Metadata:
      aws:cdk:path: CdktestStack/dlQueue/Resource
  testgettextracttextdetectionServiceRoleCF1935A8:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Action: sts:AssumeRole
            Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
        Version: "2012-10-17"
      ManagedPolicyArns:
        - Fn::Join:
            - ""
            - - "arn:"
              - Ref: AWS::Partition
              - :iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
        - Fn::Join:
            - ""
            - - "arn:"
              - Ref: AWS::Partition
              - :iam::aws:policy/CloudWatchLambdaInsightsExecutionRolePolicy
    Metadata:
      aws:cdk:path: CdktestStack/test-get-textract-text-detection/ServiceRole/Resource
  testgettextracttextdetectionServiceRoleDefaultPolicy559BFF9C:
    Type: AWS::IAM::Policy
    Properties:
      PolicyDocument:
        Statement:
          - Action: sqs:SendMessage
            Effect: Allow
            Resource:
              Fn::GetAtt:
                - dlQueue819AE35E
                - Arn
          - Action:
              - textract:GetDocumentTextDetection
              - xray:PutTelemetryRecords
              - xray:PutTraceSegments
            Effect: Allow
            Resource: "*"
          - Action:
              - s3:Abort*
              - s3:DeleteObject*
              - s3:GetBucket*
              - s3:GetObject*
              - s3:List*
              - s3:PutObject
              - s3:PutObjectLegalHold
              - s3:PutObjectRetention
              - s3:PutObjectTagging
              - s3:PutObjectVersionTagging
            Effect: Allow
            Resource:
              - Fn::GetAtt:
                  - sourceBuckettestDeploy8AC14122
                  - Arn
              - Fn::Join:
                  - ""
                  - - Fn::GetAtt:
                        - sourceBuckettestDeploy8AC14122
                        - Arn
                    - /*
        Version: "2012-10-17"
      PolicyName: testgettextracttextdetectionServiceRoleDefaultPolicy559BFF9C
      Roles:
        - Ref: testgettextracttextdetectionServiceRoleCF1935A8
    Metadata:
      aws:cdk:path: CdktestStack/test-get-textract-text-detection/ServiceRole/DefaultPolicy/Resource
  testgettextracttextdetection08D3AF7A:
    Type: AWS::Lambda::Function
    Properties:
      Code:
        S3Bucket:
          Fn::Sub: cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}
        S3Key: 2f38cab1b79e0e9d98aaa1ce8509cb252935198c1c17eec6bac8484e8e57b07c.zip
      DeadLetterConfig:
        TargetArn:
          Fn::GetAtt:
            - dlQueue819AE35E
            - Arn
      Description: Get Textract text detection
      Environment:
        Variables:
          NODE_ENV: test
      FunctionName: test-get-textract-text-detection
      Handler: index.handler
      Layers:
        - Fn::FindInMap:
            - CloudwatchlambdainsightsversionMap
            - Ref: AWS::Region
            - 1x0x143x0xx86x64
      MemorySize: 1024
      Role:
        Fn::GetAtt:
          - testgettextracttextdetectionServiceRoleCF1935A8
          - Arn
      Runtime: nodejs20.x
      Timeout: 900
      TracingConfig:
        Mode: Active
    DependsOn:
      - testgettextracttextdetectionServiceRoleDefaultPolicy559BFF9C
      - testgettextracttextdetectionServiceRoleCF1935A8
    Metadata:
      aws:cdk:path: CdktestStack/test-get-textract-text-detection/Resource
      aws:asset:path: asset.2f38cab1b79e0e9d98aaa1ce8509cb252935198c1c17eec6bac8484e8e57b07c
      aws:asset:is-bundled: true
      aws:asset:property: Code
  sourceBuckettestDeploy8AC14122:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: source-bucket-test-deploy
    UpdateReplacePolicy: Delete
    DeletionPolicy: Delete
    Metadata:
      aws:cdk:path: CdktestStack/sourceBuckettestDeploy/Resource
  CDKMetadata:
    Type: AWS::CDK::Metadata
    Properties:
      Analytics: v2:deflate64:H4sIAAAAAAAA/z2P3W7CMAxGn4X71KPlYtyOSrvb2MoDVCZNkfvjFJwMoajvPiUZuzrHkr/PcgXlawXbDd6l0N1YTHSGcHKoR4V3aYNcBcK3N96ouuckq5pwPnfYsu3MIPCZ8O5ZO7KsCGcIjZ1SIPHLTqQfccy2Ktm1KGKcwFuEkh2Eg9ejcQcUo7LGQLbnRah7ft5ZVWPE+ps2KnWcHF6ILzH0gcvyp0fvFp+a/rfrnmvLHeWS+AMM8vJTbaHcQ7kZhKi4eXY0G2gyfwFssIdnJQEAAA==
    Metadata:
      aws:cdk:path: CdktestStack/CDKMetadata/Default
    Condition: CDKMetadataAvailable
Mappings:
  CloudwatchlambdainsightsversionMap:
    af-south-1:
      1x0x143x0xx86x64: arn:aws:lambda:af-south-1:012438385374:layer:LambdaInsightsExtension:13
    ap-east-1:
      1x0x143x0xx86x64: arn:aws:lambda:ap-east-1:519774774795:layer:LambdaInsightsExtension:13
    ap-northeast-1:
      1x0x143x0xx86x64: arn:aws:lambda:ap-northeast-1:580247275435:layer:LambdaInsightsExtension:32
    ap-northeast-2:
      1x0x143x0xx86x64: arn:aws:lambda:ap-northeast-2:580247275435:layer:LambdaInsightsExtension:20
    ap-northeast-3:
      1x0x143x0xx86x64: arn:aws:lambda:ap-northeast-3:194566237122:layer:LambdaInsightsExtension:2
    ap-south-1:
      1x0x143x0xx86x64: arn:aws:lambda:ap-south-1:580247275435:layer:LambdaInsightsExtension:21
    ap-southeast-1:
      1x0x143x0xx86x64: arn:aws:lambda:ap-southeast-1:580247275435:layer:LambdaInsightsExtension:21
    ap-southeast-2:
      1x0x143x0xx86x64: arn:aws:lambda:ap-southeast-2:580247275435:layer:LambdaInsightsExtension:21
    ca-central-1:
      1x0x143x0xx86x64: arn:aws:lambda:ca-central-1:580247275435:layer:LambdaInsightsExtension:20
    cn-north-1:
      1x0x143x0xx86x64: arn:aws-cn:lambda:cn-north-1:488211338238:layer:LambdaInsightsExtension:14
    cn-northwest-1:
      1x0x143x0xx86x64: arn:aws-cn:lambda:cn-northwest-1:488211338238:layer:LambdaInsightsExtension:14
    eu-central-1:
      1x0x143x0xx86x64: arn:aws:lambda:eu-central-1:580247275435:layer:LambdaInsightsExtension:21
    eu-north-1:
      1x0x143x0xx86x64: arn:aws:lambda:eu-north-1:580247275435:layer:LambdaInsightsExtension:20
    eu-south-1:
      1x0x143x0xx86x64: arn:aws:lambda:eu-south-1:339249233099:layer:LambdaInsightsExtension:13
    eu-west-1:
      1x0x143x0xx86x64: arn:aws:lambda:eu-west-1:580247275435:layer:LambdaInsightsExtension:21
    eu-west-2:
      1x0x143x0xx86x64: arn:aws:lambda:eu-west-2:580247275435:layer:LambdaInsightsExtension:21
    eu-west-3:
      1x0x143x0xx86x64: arn:aws:lambda:eu-west-3:580247275435:layer:LambdaInsightsExtension:20
    me-south-1:
      1x0x143x0xx86x64: arn:aws:lambda:me-south-1:285320876703:layer:LambdaInsightsExtension:13
    sa-east-1:
      1x0x143x0xx86x64: arn:aws:lambda:sa-east-1:580247275435:layer:LambdaInsightsExtension:20
    us-east-1:
      1x0x143x0xx86x64: arn:aws:lambda:us-east-1:580247275435:layer:LambdaInsightsExtension:21
    us-east-2:
      1x0x143x0xx86x64: arn:aws:lambda:us-east-2:580247275435:layer:LambdaInsightsExtension:21
    us-west-1:
      1x0x143x0xx86x64: arn:aws:lambda:us-west-1:580247275435:layer:LambdaInsightsExtension:20
    us-west-2:
      1x0x143x0xx86x64: arn:aws:lambda:us-west-2:580247275435:layer:LambdaInsightsExtension:21
Outputs:
  testGetTextractTextDetectionLambda:
    Value:
      Fn::GetAtt:
        - testgettextracttextdetection08D3AF7A
        - Arn
  sourceBucket:
    Value:
      Ref: sourceBuckettestDeploy8AC14122
Conditions:
  CDKMetadataAvailable:
...

So, I'm wondering if in your use case, the previous cdk.out directory was created using other Windows account and running cdk synth might be having issues in cleaning up the directory and building fresh assets. You might want to try running cdk synth from command prompt with Administrator privileges to rule out the same.

Thanks,
Ashish

[github-actions]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.