Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Profile to id#34 #36

Merged
merged 3 commits into from
Dec 21, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 5 additions & 7 deletions DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ Each Provider implements the below described `schema.Provider` interface. The co
type Provider interface {
// Name returns the name of the provider
Name() string
// ProfileName returns the name of the provider profile
ProfileName() string
// ID returns the name of the provider id
ID() string
// Resources returns the provider for an resource deployment source.
Resources(ctx context.Context) (*Resources, error)
}
Expand Down Expand Up @@ -62,7 +62,7 @@ switch value {

### Resource

A resource is a single unit in cloud belonging to an Organization. Some metadata is provided, like whether is the asset public facing or private, provider, profile name, as well as any IP addresses and DNS Names (Either among IP or DNS must always be provided).
A resource is a single unit in cloud belonging to an Organization. Some metadata is provided, like whether is the asset public facing or private, provider, id name, as well as any IP addresses and DNS Names (Either among IP or DNS must always be provided).

Providers return `schema.Resource` structure that contains an array of resources and provides some convenience wrappers on top of the array like `Append` and `Merge`. These can be used during the resource collection phase to minimize boilerplate.

Expand All @@ -73,10 +73,8 @@ type Resource struct {
Public bool `json:"public"`
// Provider is the name of provider for instance
Provider string `json:"provider"`
// Profile is the profile name of the resource provider
Profile string `json:"profile,omitempty"`
// ProfileName is the name of the key profile
ProfileName string `json:"profile_name,omitempty"`
// ID is the id the resource provider
ID string `json:"id,omitempty"`
// PublicIPv4 is the public ipv4 address of the instance.
PublicIPv4 string `json:"public_ipv4,omitempty"`
// PrivateIpv4 is the private ipv4 address of the instance
Expand Down
46 changes: 23 additions & 23 deletions PROVIDERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ Amazon Web Services can be integrated by using the following configuration block
```yaml
- # provider is the name of the provider
provider: aws
# profile is the name of the provider profile
profile: staging
# id is the name of the provider id
id: staging
# aws_access_key is the access key for AWS account
aws_access_key: AKIAXXXXXXXXXXXXXX
# aws_secret_key is the secret key for AWS account
Expand All @@ -35,8 +35,8 @@ Google Cloud Platform can be integrated by using the following configuration blo
```yaml
- # provider is the name of the provider
provider: gcp
# profile is the name of the provider profile
profile: staging
# id is the name of the provider id
id: staging
# gcp_service_account_key is the key token of service account.
gcp_service_account_key: '{}'
```
Expand All @@ -58,8 +58,8 @@ Microsoft Azure can be integrated by using the following configuration block.
```yaml
- # provider is the name of the provider
provider: azure
# profile is the name of the provider profile
profile: staging
# id is the name of the provider id
id: staging
# client_id is the client ID of registered application of the azure account (not requuired if using cli auth)
client_id: xxxxxxxxxxxxxxxxxxxxxxxxx
# client_secret is the secret ID of registered application of the zure account (not requuired if using cli auth)
Expand Down Expand Up @@ -148,8 +148,8 @@ Heroku can be integrated by using the following configuration block.
```yaml
- # provider is the name of the provider
provider: heroku
# profile is the name of the provider profile
profile: staging
# id is the name of the provider id
id: staging
# heroku_api_token is the authorization token for Heroku account
heroku_api_token: cf0e05d9-4eca-4948-a012-b9xxxxxxxxxx
```
Expand Down Expand Up @@ -179,8 +179,8 @@ Fastly can be integrated by using the following configuration block.
```yaml
- # provider is the name of the provider
provider: fastly
# profile is the name of the provider profile
profile: staging
# id is the name of the provider id
id: staging
# fastly_api_key is the personal API token for fastly account
fastly_api_key: XX-XXXXXXXXXXXXXXXXXXXXXX-
```
Expand All @@ -198,13 +198,13 @@ Linode can be integrated by using the following configuration block.
```yaml
- # provider is the name of the provider
provider: linode
# profile is the name of the provider profile
profile: staging
# id is the name of the provider id
id: staging
# linode_personal_access_token is the personal access token for Linode account
linode_personal_access_token: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
```

`linode_personal_access_token` can be created from https://cloud.linode.com/profile/tokens. Minimum scope needed is `Read Only` for `Linodes` resource.
`linode_personal_access_token` can be created from https://cloud.linode.com/id/tokens. Minimum scope needed is `Read Only` for `Linodes` resource.

References -
1. https://www.linode.com/docs/guides/getting-started-with-the-linode-api/#get-an-access-token
Expand All @@ -217,8 +217,8 @@ Namecheap can be integrated by using the following configuration block.
```yaml
- # provider is the name of the provider
provider: namecheap
# profile is the name of the provider profile
profile: staging
# id is the name of the provider id
id: staging
# namecheap_api_key is the api key for namecheap account
namecheap_api_key: xxxxxxxxxxxxxxxxxx
# namecheap_user_name is the username of the namecheap account
Expand All @@ -244,8 +244,8 @@ Alibaba Cloud can be integrated by using the following configuration block.
```yaml
- # provider is the name of the provider
provider: alibaba
# profile is the name of the provider profile
profile: staging
# id is the name of the provider id
id: staging
# alibaba_region_id is the region id of the resources
alibaba_region_id: ap-XXXXXXX
# alibaba_access_key is the access key ID for alibaba cloud account
Expand All @@ -269,8 +269,8 @@ Terraform can be integrated by using the following configuration block.
```yaml
- # provider is the name of the provider
provider: terraform
# profile is the name of the provider profile
profile: staging
# id is the name of the provider id
id: staging
#tf_state_file is the location of terraform state file (terraform.tfsate)
tf_state_file: path/to/terraform.tfstate
```
Expand All @@ -283,8 +283,8 @@ Hashicorp Consul can be integrated by using the following configuration block.
provider: consul
# consul_url is the url for consul server
consul_url: http://localhost:8500/
# profile is the name of the provider profile
profile: staging
# id is the name of the provider id
id: staging
# consul_ca_file is the path to consul CA file
# consul_ca_file: <path-to-ca-file>.pem
# consul_cert_file is the path to consul Certificate file
Expand All @@ -311,8 +311,8 @@ Hashicorp Nomad can be integrated by using the following configuration block.
provider: nomad
# nomad_url is the url for nomad server
nomad_url: http://127.0.0.1:4646/
# profile is the name of the provider profile
profile: staging
# id is the name of the provider id
id: staging
# nomad_ca_file is the path to nomad CA file
# nomad_ca_file: <path-to-ca-file>.pem
# nomad_cert_file is the path to nomad Certificate file
Expand Down
16 changes: 8 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,8 +80,8 @@ The default config file should be located in `$HOME/.config/cloudlist/config.yam
# Configuration file for cloudlist enumeration agent
- # provider is the name of the provider (Digitalocean)
provider: do
# profile is the name of the provider profile
profile: xxxx
# id is the name of the provider id
id: xxxx
# digitalocean_token is the API key for digitalocean cloud platform
digitalocean_token: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Expand All @@ -94,8 +94,8 @@ The default config file should be located in `$HOME/.config/cloudlist/config.yam

- # provider is the name of the provider (Amazon Web Services)
provider: aws
# profile is the name of the provider profile
profile: staging
# id is the name of the provider id
id: staging
# aws_access_key is the access key for AWS account
aws_access_key: AKIAXXXXXXXXXXXXXX
# aws_secret_key is the secret key for AWS account
Expand All @@ -105,15 +105,15 @@ The default config file should be located in `$HOME/.config/cloudlist/config.yam

- # provider is the name of the provider (Google Cloud Platform)
provider: gcp
# profile is the name of the provider profile
profile: logs
# id is the name of the provider id
id: logs
# gcp_service_account_key is the minified json of a google cloud service account with list permissions
gcp_service_account_key: '{xxxxxxxxxxxxx}'

- # provider is the name of the provider
provider: fastly
# profile is the name of the provider profile
profile: staging
# id is the name of the provider id
id: staging
# fastly_api_key is the personal API token for fastly account
fastly_api_key: XX-XXXXXXXXXXXXXXXXXXXXXX-
```
Expand Down
36 changes: 18 additions & 18 deletions internal/runner/options.go
Original file line number Diff line number Diff line change
Expand Up @@ -116,8 +116,8 @@ const defaultConfigFile = `# #Configuration file for cloudlist enumeration agen

#- # provider is the name of the provider
# provider: do
# # profile is the name of the provider profile
# profile: xxxx
# # id is the name of the provider id
# id: xxxx
# # digitalocean_token is the API key for digitalocean cloud platform
# digitalocean_token: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
#
Expand All @@ -130,8 +130,8 @@ const defaultConfigFile = `# #Configuration file for cloudlist enumeration agen
#
#- # provider is the name of the provider
# provider: aws
# # profile is the name of the provider profile
# profile: staging
# # id is the name of the provider id
# id: staging
# # aws_access_key is the access key for AWS account
# aws_access_key: AKIAXXXXXXXXXXXXXX
# # aws_secret_key is the secret key for AWS account
Expand All @@ -141,8 +141,8 @@ const defaultConfigFile = `# #Configuration file for cloudlist enumeration agen

#- # provider is the name of the provider
# provider: azure
# # profile is the name of the provider profile
# profile: staging
# # id is the name of the provider id
# id: staging
# # client_id is the client ID of registered application of the azure account (not requuired if using cli auth)
# client_id: xxxxxxxxxxxxxxxxxxxxxxxxx
# # client_secret is the secret ID of registered application of the zure account (not requuired if using cli auth)
Expand All @@ -163,29 +163,29 @@ const defaultConfigFile = `# #Configuration file for cloudlist enumeration agen

#- # provider is the name of the provider
# provider: heroku
# # profile is the name of the provider profile
# profile: staging
# # id is the name of the provider id
# id: staging
# # heroku_api_token is the api key for Heroku account
# heroku_api_token: xxxxxxxxxxxxxxxxxxxx

#- # provider is the name of the provider
# provider: linode
# # profile is the name of the provider profile
# profile: staging
# # id is the name of the provider id
# id: staging
# # linode_personal_access_token is the personal access token for linode account
# linode_personal_access_token: XXXXXXXXXXXXXXXXXXXXXXXX

#- # provider is the name of the provider
# provider: fastly
# # profile is the name of the provider profile
# profile: staging
# # id is the name of the provider id
# id: staging
# # fastly_api_key is the personal API token for fastly account
# fastly_api_key: XX-XXXXXXXXXXXXXXXXXXXXXX-

#- # provider is the name of the provider
# provider: alibaba
# # profile is the name of the provider profile
# profile: staging
# # id is the name of the provider id
# id: staging
# # alibaba_region_id is the region id of the resources
# alibaba_region_id: ap-XXXXXXX
# # alibaba_access_key is the access key ID for alibaba cloud account
Expand All @@ -195,17 +195,17 @@ const defaultConfigFile = `# #Configuration file for cloudlist enumeration agen

# - # provider is the name of the provider
# provider: namecheap
# # profile is the name of the provider profile
# profile: staging
# # id is the name of the provider id
# id: staging
# # namecheap_api_key is the api key for namecheap account
# namecheap_api_key: XXXXXXXXXXXXXXXXXXXXXXX
# # namecheap_user_name is the username of the namecheap account
# namecheap_user_name: XXXXXXX

# - # provider is the name of the provider
# provider: terraform
# # profile is the name of the provider profile
# profile: staging
# # id is the name of the provider id
# id: staging
# #tf_state_file is the location of terraform state file (terraform.tfsate)
# tf_state_file: path/to/terraform.tfstate

Expand Down
12 changes: 6 additions & 6 deletions internal/runner/runner.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,8 @@ func (r *Runner) Enumerate() {
if item == nil {
continue
}
if _, ok := item["profile"]; !ok {
item["profile"] = ""
if _, ok := item["id"]; !ok {
item["id"] = ""
}
// Validate and only pass the correct items to input
if r.options.Provider != "" {
Expand Down Expand Up @@ -74,10 +74,10 @@ func (r *Runner) Enumerate() {
}
}

gologger.Info().Msgf("Listing assets from %s (%s) provider\n", provider.Name(), provider.ProfileName())
gologger.Info().Msgf("Listing assets from %s (%s) provider\n", provider.Name(), provider.ID())
instances, err := provider.Resources(context.Background())
if err != nil {
gologger.Warning().Msgf("Could not get resources for provider %s %s: %s\n", provider.Name(), provider.ProfileName(), err)
gologger.Warning().Msgf("Could not get resources for provider %s %s: %s\n", provider.Name(), provider.ID(), err)
continue
}
var hostsCount, ipCount int
Expand Down Expand Up @@ -177,9 +177,9 @@ func (r *Runner) Enumerate() {
logBuilder.WriteString(" IP Addresses")
}
if hostsCount == 0 && ipCount == 0 {
gologger.Warning().Msgf("No results found for %s (%s)\n", provider.Name(), provider.ProfileName())
gologger.Warning().Msgf("No results found for %s (%s)\n", provider.Name(), provider.ID())
} else {
gologger.Info().Msgf("Found %s for %s (%s)\n", logBuilder.String(), provider.Name(), provider.ProfileName())
gologger.Info().Msgf("Found %s for %s (%s)\n", logBuilder.String(), provider.Name(), provider.ID())
}
}
}
4 changes: 2 additions & 2 deletions pkg/inventory/inventory.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,10 @@ func New(options schema.Options) (*Inventory, error) {
if !ok {
continue
}
profile, _ := block.GetMetadata("profile")
id, _ := block.GetMetadata("id")
provider, err := nameToProvider(value, block)
if err != nil {
gologger.Warning().Msgf("Could not initialize provider %s %s: %s\n", value, profile, err)
gologger.Warning().Msgf("Could not initialize provider %s %s: %s\n", value, id, err)
continue
}
inventory.Providers = append(inventory.Providers, provider)
Expand Down
16 changes: 8 additions & 8 deletions pkg/providers/alibaba/alibaba.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ const (

// Provider is a data provider for alibaba API
type Provider struct {
profile string
client *ecs.Client
id string
client *ecs.Client
}

// New creates a new provider client for alibaba API
Expand All @@ -35,7 +35,7 @@ func New(options schema.OptionBlock) (*Provider, error) {
return nil, &schema.ErrNoSuchKey{Name: accessKeySecret}
}

profile, _ := options.GetMetadata("profile")
id, _ := options.GetMetadata("id")

client, err := ecs.NewClientWithAccessKey(
regionID, // region ID
Expand All @@ -46,22 +46,22 @@ func New(options schema.OptionBlock) (*Provider, error) {
return nil, err
}

return &Provider{client: client, profile: profile}, nil
return &Provider{client: client, id: id}, nil
}

// Name returns the name of the provider
func (p *Provider) Name() string {
return providerName
}

// ProfileName returns the name of the provider profile
func (p *Provider) ProfileName() string {
return p.profile
// ID returns the name of the provider id
func (p *Provider) ID() string {
return p.id
}

// Resources returns the provider for an resource deployment source.
func (p *Provider) Resources(ctx context.Context) (*schema.Resources, error) {
ecsprovider := &instanceProvider{client: p.client, profile: p.profile}
ecsprovider := &instanceProvider{client: p.client, id: p.id}
list, err := ecsprovider.GetResource(ctx)
if err != nil {
return nil, err
Expand Down
Loading