Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Unable to add an existing EFS file system via Lookup #773

Closed
lorienx0 opened this issue Aug 12, 2024 · 3 comments · Fixed by #774
Closed

[BUG] Unable to add an existing EFS file system via Lookup #773

lorienx0 opened this issue Aug 12, 2024 · 3 comments · Fixed by #774
Assignees
Labels
bug Something isn't working

Comments

@lorienx0
Copy link
Contributor

Describe the bug
ecs compose x can successfully find EFS file systems using Lookup, but fails to bind them to services because of an error in one of the Python modules. efs.efs_ecs.looked_up_efs_family_hook line 236 tries to pass family.stack into add_security_group_ingress, but that function expects just the family object itself, as it tries to access .stack attribute again, which fails and causes the tool to terminate execution. Removing .stack and passing the family object solves the issue completely.

To Reproduce
Steps to reproduce the behavior:
Create a simple VPC and EFS file system.
Use the shared test case yaml file with your VPC and EFS
Try to render it using the ecs compose x CLI tool from venv+pip installation

  1. How you installed ECS ComposeX: virutalenv + pip
  2. How you are executing it ? : As a CLI tool, as a library, in lambda? CLI
  3. Link / share your Docker compose file
volumes:
  efs-test:
    x-efs:
      Lookup:
        Tags:
          - Name: efs-test

services:
  nginxd:
    image: nginx:1.27.0
    ports:
      - protocol: tcp
        target: 80
    volumes:
      - efs-test:/usr/share/nginx/html

x-vpc:
  Lookup:
    VpcId:
      Tags:
        - Name: minimal-vpc
    PublicSubnets:
      Tags:
        - Name: minimal-subnet-public-a
        - Name: minimal-subnet-public-b
    AppSubnets:
      Tags:
        - Name: minimal-subnet-app-a
        - Name: minimal-subnet-app-b
    StorageSubnets:
      Tags:
        - Name: minimal-subnet-storage-a
        - Name: minimal-subnet-storage-b
  1. See error
    AttributeError: stack

Expected behavior
CLI tool rendering CFN stacks, not terminating abnormally.

Logs

ecs-compose-x render -f minimal-efs-test.yml -n nginxd-app --format yaml -d render3
/home/derpson/projects/ecsx/venv/lib/python3.10/site-packages/ecs_composex/compose/compose_services/service_image/docker_opts.py:33: UserWarning: Run pip install ecs-composex[ecrscan] to enable this functionality.
  warnings.warn(str(error))
2024-08-12 19:13:03 [    INFO] volumes.efs-test - Mapped to nginxd
2024-08-12 19:13:03 [    INFO] services.nginxd - No Launch Type defined. Using default: FARGATE
2024-08-12 19:13:04 [    INFO] Service families to process ['nginxd']
2024-08-12 19:13:04 [    INFO] No cluster information provided. Creating a new one
Loaded x-vpc vpc vpc /home/derpson/projects/ecsx/venv/lib/python3.10/site-packages/ecs_composex/vpc
Loaded x-efs efs efs /home/derpson/projects/ecsx/venv/lib/python3.10/site-packages/ecs_composex/efs
2024-08-12 19:13:04 [    INFO] Processing x-vpc
2024-08-12 19:13:07 [    INFO] x-vpc - Found VPC - vpc-xxxxxxxxxxxxxxxxx
2024-08-12 19:13:07 [    INFO] Processing x-efs
2024-08-12 19:13:07 [    INFO] x-efs.efs-test - Lookup via Tags
2024-08-12 19:13:13 [    INFO] x-vpc - Found VPC - vpc-xxxxxxxxxxxxxxxxx
2024-08-12 19:13:13 [    INFO] x-efs.efs-test - Linking to services
Traceback (most recent call last):
  File "/home/derpson/projects/ecsx/venv/lib/python3.10/site-packages/troposphere/__init__.py", line 228, in __getattr__
    return self.properties.__getitem__(name)
KeyError: 'stack'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/derpson/projects/ecsx/venv/bin/ecs-compose-x", line 8, in <module>
    sys.exit(main())
  File "/home/derpson/projects/ecsx/venv/lib/python3.10/site-packages/ecs_composex/cli.py", line 206, in main
    root_stack = generate_full_template(settings)
  File "/home/derpson/projects/ecsx/venv/lib/python3.10/site-packages/ecs_composex/ecs_composex.py", line 290, in generate_full_template
    apply_x_configs_to_ecs(settings, settings.root_stack, modules=settings.mod_manager)
  File "/home/derpson/projects/ecsx/venv/lib/python3.10/site-packages/ecs_composex/ecs_composex.py", line 144, in apply_x_configs_to_ecs
    invoke_x_to_ecs(res_type, root_stack, resource_stack[res_type], settings)
  File "/home/derpson/projects/ecsx/venv/lib/python3.10/site-packages/ecs_composex/ecs_composex.py", line 107, in invoke_x_to_ecs
    ecs_function(
  File "/home/derpson/projects/ecsx/venv/lib/python3.10/site-packages/ecs_composex/efs/efs_ecs.py", line 319, in efs_to_ecs
    expand_family_with_efs_volumes(None, resource, settings)
  File "/home/derpson/projects/ecsx/venv/lib/python3.10/site-packages/ecs_composex/efs/efs_ecs.py", line 260, in expand_family_with_efs_volumes
    looked_up_efs_family_hook(efs, family, settings)
  File "/home/derpson/projects/ecsx/venv/lib/python3.10/site-packages/ecs_composex/efs/efs_ecs.py", line 236, in looked_up_efs_family_hook
    add_security_group_ingress(
  File "/home/derpson/projects/ecsx/venv/lib/python3.10/site-packages/ecs_composex/rds_resources_settings.py", line 389, in add_security_group_ingress
    family.stack.stack_template,
  File "/home/derpson/projects/ecsx/venv/lib/python3.10/site-packages/troposphere/__init__.py", line 235, in __getattr__
    raise AttributeError(name)
AttributeError: stack

Desktop (please complete the following information):

  • OS: Ubuntu 22.04.4 LTS
  • Python version: 3.10.12
  • Tag/Version: 1.1.3
@lorienx0 lorienx0 added the bug Something isn't working label Aug 12, 2024
@JohnPreston
Copy link
Member

Thanks so much @lorienx0 for taking the time to report this and go into such details.
Is there a PR coming in from you to fix/address the bug? :)

@lorienx0
Copy link
Contributor Author

I can make a PR, I will need a bit of time to setup the dev environment though.

@JohnPreston
Copy link
Member

I can make a PR, I will need a bit of time to setup the dev environment though.

Happy either way :)
If I don't see a PR coming in later I will submit a PR and ping you the branch to test locally for yourself to confirm all is working properly.

lorienx0 pushed a commit to lorienx0/ecs_composex that referenced this issue Aug 12, 2024
@JohnPreston JohnPreston linked a pull request Aug 12, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants