Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[feature] allow lambda to run with podman as an alternative to docker #3276

Closed
yuvalif opened this issue Sep 2, 2020 · 5 comments · Fixed by #3702
Closed

[feature] allow lambda to run with podman as an alternative to docker #3276

yuvalif opened this issue Sep 2, 2020 · 5 comments · Fixed by #3702
Labels
enhancement

Comments

@yuvalif
Copy link

yuvalif commented Sep 2, 2020

  • allows you to containers not as root
  • cgroups v2
  • better support on centos8/fedora OSs

should be an alternative, and not a replacement, as docker is more widespread than podman
@bblommers
Copy link
Collaborator

Hi @yuvalif, thanks for raising this. Hadn't heard of Podman before.

Is there a specific use case for Lambdas where Podman would give a noticable improvement? From what I can read, it would behave exactly the same in our limited use case. (I.e., run t he container, execute some python code and return the result.)

@yuvalif
Copy link
Author

yuvalif commented Sep 2, 2020

the main improvement is around installation and setup of docker (especially in case of centos8/fedora OSs), as well as the fact that non-privileged containers don't need root permissions in podman.
don't think there is functional advantage.

@yan12125
Copy link
Contributor

I give it a try, and get blocked as Podman hasn't implemented all Docker APIs. After some patches to moto and docker-py, the blocker comes to the put_archive function [1][2].

I'm interested in this as I'm maintaining the Arch Linux package and I'd like to run the test suite of moto without extra privileges.

[1] https://github.com/spulec/moto/blob/9f0f230d130a839fb2de6bfc97af4182360fbcdb/moto/awslambda/models.py#L135
[2] containers/podman#6050

@yan12125 yan12125 mentioned this issue Dec 4, 2020
Merged
@yan12125 yan12125 mentioned this issue Feb 17, 2021
Merged
@yan12125
Copy link
Contributor

Podman 3.0 supports all needed Docker APIs, and only a few changes are needed in moto => #3702

bblommers pushed a commit that referenced this issue Feb 18, 2021
Support Podman for mocking Lambda (#3702)
2000f66 
* Support Podman for mocking Lambda

Podman supports all Docker APIs used in moto since version 3.0. Note
that Podman requires pulling the image before creating a container
using a fully-qualified image name (e.g., "docker.io/library/busybox"
instead of "busybox").

Test plan:
$ podman system service -t 0
$ DOCKER_HOST="unix://$XDG_RUNTIME_DIR/podman/podman.sock" pytest

Fixes #3276

* Run black

* Python 2 compatibility

* Address review comments and improve parse_image_ref
@bblommers
Copy link
Collaborator

This is now supported in moto >= 1.3.17.dev276

tony-dot-sh added a commit to tony-dot-sh/moto that referenced this issue Mar 3, 2021
@tony-dot-sh @bblommers
@chrisw-dev @photosbychris @elopez @bpandola @gforsythe-godaddy @umgelurgel @ccggeo @ikonst @ltagliamonte @nuwang @jbmchuck @rajendersaini @jordan-dimov @gruebel @jocke-l @usmangani1
update from master (#1)
16b3ddf 
* Fix Mock 4.0.3 compatibility - Unpatch only once during teardown (getmoto#3541)

* getmoto#3535 - Unpatch only once during teardown

* EnvVar patching - Fix Python2 bug

* Allow latest version of mock-library

* Add validation to API Key min length value (getmoto#3652)

* api gateway - add api key minimum length validation

* api gateway - support includeValue query parameter on api key apis

* [apigateway] code refactoring

* Cleanup - remove duplicate utility methods

* APIGateway - Dont send headers with error messsage

Co-authored-by: Jovan Zivanov <j.zivanov@levi9.com>

* Cloudformation - Create ApiGateway resources (getmoto#3659)

* Cloudformation - Create ApiGateway resources

* Cleanup

* Linting

* adding list_create_account_status to organization getmoto#3691 (getmoto#3692)

* adding list_create_account_status to organization getmoto#3691

* removing todo comment

Co-authored-by: Chris Walters <chriskwalters@hotmail.com>

* Enhancement: implement EC2 instance filtering by `subnet-id` (getmoto#3694)

Co-authored-by: Tony Greising-Murschel <tony@platform.sh>

* events: fix archive event pattern match check (getmoto#3671)

* events: fix archive event pattern match check

There is a missing `return True` for the positive match case in
matches_pattern, causing all valid patterns to fail.

* events: add test for valid, non-empty pattern match

* Pin RSA dependency for Py2 (getmoto#3697)

* Update build.yml

* Enable Py2 test coverage (getmoto#3698)

* Enable Py2 test coverage

* Coverage - exclude XRay as it's throwing errors

* Update requirements-tests.txt

* Revert "Coverage - exclude XRay as it's throwing errors"

This reverts commit bb58e0e.

* Enable coverage for all workflows

* Refactor ARNs to remove hardcoded account id (getmoto#3701)

* Fix: DynamoDB:GetItem throws wrong error when table doesn't exist (getmoto#3700)

* Fix: DynamoDB:GetItem throws wrong error when table doesn't exist

* Use unique exception for table not found, per PR feedback

* Just fix the reported issue, without touching anything else...

* Update integration test to use Pytest (getmoto#3703)

* Refactor int test to use pytest

* Add comments to int test script

* Support Podman for mocking Lambda (getmoto#3702)

* Support Podman for mocking Lambda

Podman supports all Docker APIs used in moto since version 3.0. Note
that Podman requires pulling the image before creating a container
using a fully-qualified image name (e.g., "docker.io/library/busybox"
instead of "busybox").

Test plan:
$ podman system service -t 0
$ DOCKER_HOST="unix://$XDG_RUNTIME_DIR/podman/podman.sock" pytest

Fixes getmoto#3276

* Run black

* Python 2 compatibility

* Address review comments and improve parse_image_ref

* pool domain should always return a domain (getmoto#3707)

* pool domain should always return a domain

Refs getmoto#3706

* set character encoding

* test CloudFrontDomain exists on pool domain

* describe pool domain does not return cloudfront domain

* Update stackset create operation to correctly return the resource's ID. (getmoto#3709)

* Update stackset create operation to correctly return the resource's ID.

* Add test confirming create_stack_set response.

* feature/refresh_trusted_advisor_check (getmoto#3705)

* Support - added refresh_trusted_advisor_check
- Returns a random check status
- Returns the check id in the response
- Testing for these two functionalities

* test_support - addressed PR comments, to cycle through a faked number of
check status responses

* dynamodb: fix deleting last set element (w/attr name) (getmoto#3708)

* dynamodb: deleting last set element

* add user-facing test

* Fix stackset describe/delete. Fix stackset update. (getmoto#3713)

* Fix stack set fetching and deleting.

* Also fix stackset update operation to accept previous values.

* Fix linting.

* Rename aliased test.

* add tags support to ECS tasks (getmoto#3715)

* add tags support to ECS tasks

* lint

* lint

* fmt

* fmt with same version

Co-authored-by: ltagliamonte <ltagliamonte@users.noreply.github.com>

* Fix unknown instance type (getmoto#3710)

* Use apiname when comparing unknown instance ids

* Use get_instance_types API to populate instance_types.json

* Fix scope of instances array when retrieving instance types

* Add installation-methods to docs (getmoto#3720)

* Changelog 2.0.0 (getmoto#3721)

* Changelog for upcoming release (..-27/01)

* Changelog for upcoming release (..-21-02)

* Update Implementation Coverage

* Update AWS Managed Policies

* Update release version

* Add the status field to ECS task definitions (getmoto#3723)

Also I found extra tests for describe_task_definition and deregister_task_definition that were not being run,
so I changed their names so they are found by pytest and made them pass.  I also added checks to them for the new
status field.

* Automate release - Test Github Action (getmoto#3726)

* Test custom action with custom input

* Run test workflow on push

* Only allow release on manual trigger

* Fixes for IAM Role Description field in responses from list_roles and create_roles (getmoto#3724)

* Add IAM Role Description field to list_roles responses

The IAM ListRoles IAM API call will return the Description key/value
for each role if it exists.  If it does not exist the Description
key is not included.

* fix handling in create_role resp

* blackg

* Combine two tests using pytest.mark.parametrize

* consistency

* added feature update_usage_plan and fixed some lint errors (getmoto#3727)

Co-authored-by: rajinder saini <rajinder.saini@c02vt5k2htd6.corp.climate.com>

* Implements core MediaConnect endpoints (getmoto#3718)

* Adds initial models and implements create_flow endpoint.

* Updates README with a Releases section

* Removes unused import

* Adds __init__.py files to MediaLive and MediaConnect test directories.

* Adds list_flows endpoint and tests.

* Adds describe_flow endpoint and tests.

* Adds delete_flow endpoint and tests.

* Adds URLs

* Adds start_flow and stop_flow endpoints and tests.

* Adds tag_resource and list_tags_for_resource endpoints and tests.

* Uses unquote for ARNs

* 3730 update api key - doc update  (getmoto#3731)

* updated doc for added update_api_key / update_usage_plan feature

* updated doc for added update_api_key / update_usage_plan feature

Co-authored-by: rajinder saini <rajinder.saini@c02vt5k2htd6.corp.climate.com>

* Update ImplementationCoverage for APIGateway (getmoto#3733)

* Add eventbridge replay (getmoto#3735)

* Add events.start_replay

* Add events.describe_replay

* Add events.list_replays

* Add events.cancel_replay

* implement actual replay functionality

* Fix Python 2.7 issues

* Support SSM Parameter tags (getmoto#3739)

* Fix:update rule (getmoto#3734)

* Fix:SQS-message retention period consideration

* Fix:SQS-message retention period consideration

* Fix:SQS-message retention period consideration

* Added comments &linting

* Fixed tests

* Fix:update rule

* Linting and comments

Co-authored-by: Bert Blommers <bblommers@users.noreply.github.com>
Co-authored-by: Jovan Zivanov <j.zivanov@levi9.com>
Co-authored-by: chrisw-dev <68233262+chrisw-dev@users.noreply.github.com>
Co-authored-by: Chris Walters <chriskwalters@hotmail.com>
Co-authored-by: Tony Greising-Murschel <tony@platform.sh>
Co-authored-by: Emilio López <2642849+elopez@users.noreply.github.com>
Co-authored-by: Brian Pandola <bpandola@gmail.com>
Co-authored-by: Chih-Hsuan Yen <yan12125@gmail.com>
Co-authored-by: Gordon Forsythe <gordon.forsythe@pagely.com>
Co-authored-by: Jakub Musko <umgelurgel@users.noreply.github.com>
Co-authored-by: Connor <6322551+ccggeo@users.noreply.github.com>
Co-authored-by: Ilya Konstantinov <ilya.konstantinov@gmail.com>
Co-authored-by: Luigi Tagliamonte <10259338+ltagliamonte@users.noreply.github.com>
Co-authored-by: ltagliamonte <ltagliamonte@users.noreply.github.com>
Co-authored-by: Nuwan Goonasekera <2070605+nuwang@users.noreply.github.com>
Co-authored-by: William Richard <wrichard@indigoag.com>
Co-authored-by: Jon Michaelchuck <5964742+jbmchuck@users.noreply.github.com>
Co-authored-by: rajinder <rajendersaini@gmail.com>
Co-authored-by: rajinder saini <rajinder.saini@c02vt5k2htd6.corp.climate.com>
Co-authored-by: Jordan Dimov <32362173+jordan-dimov@users.noreply.github.com>
Co-authored-by: Anton Grübel <anton.gruebel@gmail.com>
Co-authored-by: Joakim Saario <saario.joakim@gmail.com>
Co-authored-by: usmangani1 <sgosman_chem@yahoo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Support Podman for mocking Lambda
3 participants
@yuvalif @yan12125 @bblommers