run using SharedINI credentials causing issue - Need nonuserid/iam role assume

[gmmurugan]

As its running using CLI Keys as credentials(SharedIniFileCredentials), some scans are not successful.
It will be better if we can enable the EC2MetadataCredentials support with a flag.

I can understand development needs SharedIni Mode and workaround as of now that ec2 uses instance role by fetching from API and saving them in ~/.aws/credentials. But its not successful for all scenarios(reference)

Errors:

Running aws.iam.MFADevicesCollector[ERROR] { ValidationError: **Must specify userName when calling with non - User credentials at Request.extractError**(/home/cloud - reports / node_modules / aws - sdk / lib / protoco l / query.js: 47: 29) at Request.callListeners(/home/cloud - reports / node_modules / aws - sdk / lib / sequen tial_executor.js: 109: 20) at Request.emit(/home/cloud - reports / node_modules / aws - sdk / lib / sequential_exec utor.js: 81: 10) at Request.emit(/home/cloud - reports / node_modules / aws - sdk / lib / request.js: 683:14) at Request.transition(/home/cloud - reports / node_modules / aws - sdk / lib / request.j s: 22: 10) at AcceptorStateMachine.runTo(/home/cloud - reports / node_modules / aws - sdk / lib / s tate_machine.js: 14: 12) at / home / cloud - reports / node_modules / aws - sdk / lib / state_machine.js: 26: 10 at Request. < anonymous > (/home/cloud - reports / node_modules / aws - sdk / lib / request.js: 38: 9) at Request. < anonymous > (/home/cloud - reports / node_modules / aws - sdk / lib / request.js: 685: 12) at Request.callListeners(/home/cloud - reports / node_modules / aws - sdk / lib / sequen tial_executor.js: 119: 18) **message: 'Must specify userName when calling with non-User credentials', code: 'ValidationError',** time: 2018 - 11 - 27 T19: 51: 17.587 Z, requestId: '123435ddf-f27d-11e8-8118-bf1c41531853', statusCode: 400, retryable: false, retryDelay: 87.51518271316361 } aws.iam.MFADevicesCollector completed

Running aws.iam.PasswordPolicyCollector **[ERROR] { NoSuchEntity: The Password Policy with domain name 123456789012 cannot be found. at Request.extractError (/home/cloud-reports/node_modules/aws-sdk/lib/protoco l/query.js:47:29)** at Request.callListeners (/home/cloud-reports/node_modules/aws-sdk/lib/sequen tial_executor.js:109:20) at Request.emit (/home/cloud-reports/node_modules/aws-sdk/lib/sequential_exec utor.js:81:10) at Request.emit (/home/cloud-reports/node_modules/aws-sdk/lib/request.js:683: 14) at Request.transition (/home/cloud-reports/node_modules/aws-sdk/lib/request.j s:22:10) at AcceptorStateMachine.runTo (/home/cloud-reports/node_modules/aws-sdk/lib/s tate_machine.js:14:12) at /home/cloud-reports/node_modules/aws-sdk/lib/state_machine.js:26:10 at Request.<anonymous> (/home/cloud-reports/node_modules/aws-sdk/lib/request. js:38:9) at Request.<anonymous> (/home/cloud-reports/node_modules/aws-sdk/lib/request. js:685:12) at Request.callListeners (/home/cloud-reports/node_modules/aws-sdk/lib/sequen tial_executor.js:119:18) message: **'The Password Policy with domain name 123456789012 cannot be found.', code: 'NoSuchEntity',** time: 2018-11-27T19:51:18.089Z, requestId: '1233345ed-f27d-11e8-94af-3578535a537a', statusCode: 404, retryable: false, retryDelay: 9.32401654223225 } aws.iam.PasswordPolicyCollector completed

[koladilip]

@gmmurugan I have added support ec2 credentials.

[gmmurugan]

the reference i gave is the workaround. the reason i created this issue is to bring out and that also not solving the issue. some resource scan needs nonuser-id(NO NEY BASED CREDENTIALS). im not sure what it meant.

may be below reference might help you .
https://docs.aws.amazon.com/IAM/latest/APIReference/API_GetUser.html
hashicorp/terraform#5030

in the end, im expecting the tool should run without specifying the profile flag in npm command so, it will assume the ec2 role and fetch keys/user(nonuserid) for each usecases

https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html

[koladilip]

Can you tell exact steps to reproduce the issue?

[gmmurugan]

How to reproduce? i ran the scan inside ec2 and those were the console logs. not stopping the file/html output run. try --module iam . It could be access issue for my role.

moreover, the ec2credentials from URL is a workaround not the fix . we have to pass the arn and userid somehow for above issues. refer the comment

@gmmurugan I have added support ec2 credentials.
its not fetching from the env variables. is it fetching runtime? i tried and its nothing in env

> process.env.AWS_CONTAINER_CREDENTIALS_RELATIVE_URI
undefined

[agnelNandapurapu]

@gmmurugan we found the issue in ListMfaDevices API call, we have resolved it and we tested on (Ubuntu 18 and RAM 2 GB), please check now.

[gmmurugan]

Great ! - its running assuming the EC2 role. i tested by removing the ~/.aws/

However, i still see 2 errors(not stopping run anyway). correct me if this is caused by access on my end or something we can handle in code.

[ERROR] aws.iam.PasswordPolicyCollector failed[{
    NoSuchEntity: The Password Policy with domain name 123456789012 cannot be found.
    at Request.extractError(/home/cloud-reports/node_modules/aws-sdk/lib/proto col / query.js: 47: 29)
    at Request.callListeners(/home/cloud-reports/node_modules/aws-sdk/lib/sequ ential_executor.js: 109: 20)
    at Request.emit(/home/cloud-reports/node_modules/aws-sdk/lib/sequential_ex ecutor.js: 81: 10)
    at Request.emit(/home/cloud-reports/node_modules/aws-sdk/lib/request.js: 68 3: 14)
    at Request.transition(/home/cloud-reports/node_modules/aws-sdk/lib/request.js: 22: 10)
    at AcceptorStateMachine.runTo(/home/ cloud - reports / node_modules / aws - sdk / lib /state_machine.js: 14: 12)
    at / home /  cloud - reports / node_modules / aws - sdk / lib / state_machine.js: 26: 10
    at Request. < anonymous > (/home/cloud-reports/node_modules/aws-sdk/lib/reques t.js: 38: 9)
    at Request. < anonymous > (/home/cloud-reports/node_modules/aws-sdk/lib/reques t.js: 685: 12)
    at Request.callListeners(/home/cloud-reports/node_modules/aws-sdk/lib/sequ ential_executor.js: 119: 18)
    message: 'The Password Policy with domain name 123456789012 cannot be found.',
    code: 'NoSuchEntity',
    time: 2018 - 12 - 04 T16: 23: 15.762 Z,
    requestId: 'e7250cce-f7e0-11e8-9ec9-bf77700720d8',
    statusCode: 404,
    retryable: false,
    retryDelay: 42.70464297811456
}]

Running aws.vpc.VpcsCollector[ERROR] aws.lambda.LambdaFunctionVersionsCollector failed[[{
    TooManyRequestsException: Rate exceeded
    at Object.extractError(/home/cloud-reports/node_modules/aws-sdk/lib/protocol / json.js: 48: 27)
    at Request.extractError(/home/cloud-reports/node_modules/aws-sdk/lib/pro tocol / rest_json.js: 52: 8)
    at Request.callListeners(/home/cloud-reports/node_modules/aws-sdk/lib/se quential_executor.js: 109: 20)
    at Request.emit(/home/cloud-reports/node_modules/aws-sdk/lib/sequential_ executor.js: 81: 10)
    at Request.emit(/home/cloud-reports/node_modules/aws-sdk/lib/request.js:683: 14)
    at Request.transition(/home/cloud-reports/node_modules/aws-sdk/lib/reque st.js: 22: 10)
    at AcceptorStateMachine.runTo(/home/ cloud - reports / node_modules / aws - sdk / l ib / state_machine.js: 14: 12)
    at / home /  cloud - reports / node_modules / aws - sdk / lib / state_machine.js: 26: 10
    at Request. < anonymous > (/home/cloud-reports/node_modules/aws-sdk/lib/requ est.js: 38: 9)
    at Request. < anonymous > (/home/cloud-reports/node_modules/aws-sdk/lib/requ est.js: 685: 12)
    message: 'Rate exceeded',
    code: 'TooManyRequestsException',
    time: 2018 - 12 - 04 T16: 23: 33.565 Z,
    requestId: 'f1c3194f-f7e0-11e8-b301-f7c8f85027c8',
    statusCode: 429,
    retryable: true
}]]
aws.sns.TopicsCollector completed

[koladilip]

Can you confirm if you have password policy on the AWS account you are testing?

[koladilip]

@gmmurugan add a check for PasswordPolicyCollector if doesn't exist then it won't give the error.
LambdaFunctionVersionsCollector added some wait time to avoid this error.

[gmmurugan]

1. 1. i ran with root access and this policy check collector ran without issues . look like i need some read only access for iam to avoid throwing that error. if possible document what role is best to cover all collectors.

2. lambda ran without issue with there is 100+secs pause

aws.lambda.LambdaFunctionAliasesCollector completed aws.lambda.LambdaFunctionVersionsCollector completed

rest is good.

[koladilip]

We have updated the read me.