Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add coverage for flow run with agent to integration test #8374

Merged
merged 16 commits into from
Feb 2, 2023
Merged

Add coverage for flow run with agent to integration test #8374

merged 16 commits into from
Feb 2, 2023

Conversation

zanieb
Copy link
Contributor

@zanieb zanieb commented Feb 2, 2023 •
edited
Loading

Adds an integration test that creates a deployment, creates a flow run, and starts an agent.

Intended to test cases like #8373

Example

❯ python flows/agent.py
11:44:05.976 | DEBUG   | prefect.client - Using ephemeral application with database at sqlite+aiosqlite:////Users/mz/.prefect/orion.db
11:44:06.049 | DEBUG   | prefect.client - Using ephemeral application with database at sqlite+aiosqlite:////Users/mz/.prefect/orion.db
11:44:06.106 | DEBUG   | prefect.client - Using ephemeral application with database at sqlite+aiosqlite:////Users/mz/.prefect/orion.db
11:44:06.215 | DEBUG   | prefect.client - Using ephemeral application with database at sqlite+aiosqlite:////Users/mz/.prefect/orion.db
Starting v2.7.10+46.gb3dcf538f2 agent with ephemeral API...
11:44:09.056 | DEBUG   | prefect.client - Using ephemeral application with database at sqlite+aiosqlite:////Users/mz/.prefect/orion.db

  ___ ___ ___ ___ ___ ___ _____     _   ___ ___ _  _ _____
 | _ \ _ \ __| __| __/ __|_   _|   /_\ / __| __| \| |_   _|
 |  _/   / _|| _|| _| (__  | |    / _ \ (_ | _|| .` | | |
 |_| |_|_\___|_| |___\___| |_|   /_/ \_\___|___|_|\_| |_|


Agent started! Looking for work from queue(s): default...
11:44:09.058 | DEBUG   | prefect.agent - Checking for scheduled flow runs...
11:44:09.059 | DEBUG   | prefect.agent - Checking for cancelled flow runs...
11:44:09.134 | INFO    | prefect.agent - Submitting flow run 'f47d98f6-11ff-4bb3-bc68-4929e5724f4f'
11:44:09.191 | INFO    | prefect.infrastructure.process - Opening process 'unbiased-yak'...
11:44:09.192 | DEBUG   | prefect.infrastructure.process - Process 'unbiased-yak' running command: /opt/homebrew/Caskroom/miniconda/base/envs/orion-dev-38/bin/python -m prefect.engine in /var/folders/q9/1myqgwxn2r3fvkvf6jy3npcm0000gn/T/tmpavriuf30prefect
11:44:09.216 | INFO    | prefect.agent - Completed submission of flow run 'f47d98f6-11ff-4bb3-bc68-4929e5724f4f'
/opt/homebrew/Caskroom/miniconda/base/envs/orion-dev-38/lib/python3.8/runpy.py:127: RuntimeWarning: 'prefect.engine' found in sys.modules after import of package 'prefect', but prior to execution of 'prefect.engine'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
11:44:11.597 | DEBUG   | prefect.client - Using ephemeral application with database at sqlite+aiosqlite:////Users/mz/.prefect/orion.db
11:44:11.641 | INFO    | Flow run 'unbiased-yak' - Downloading flow code from storage at '/Users/mz/dev/prefect'
11:44:13.648 | DEBUG   | prefect.client - Using ephemeral application with database at sqlite+aiosqlite:////Users/mz/.prefect/orion.db
11:44:28.313 | DEBUG   | Flow run 'unbiased-yak' - Importing flow code from 'flows/agent.py:hello'
11:44:28.326 | DEBUG   | Flow run 'unbiased-yak' - Starting 'ConcurrentTaskRunner'; submitted tasks will be run concurrently...
11:44:28.326 | DEBUG   | prefect.task_runner.concurrent - Starting task runner...
11:44:28.462 | DEBUG   | Flow run 'unbiased-yak' - Executing flow 'hello' for flow run 'unbiased-yak'...
11:44:28.462 | DEBUG   | Flow run 'unbiased-yak' - Beginning execution...
11:44:28.463 | INFO    | Flow run 'unbiased-yak' - Hello integration tests!
11:44:28.478 | DEBUG   | prefect.task_runner.concurrent - Shutting down task runner...
11:44:28.479 | INFO    | Flow run 'unbiased-yak' - Finished in state Completed()
11:44:28.480 | DEBUG   | prefect.client - Using ephemeral application with database at sqlite+aiosqlite:////Users/mz/.prefect/orion.db
11:44:30.781 | INFO    | prefect.infrastructure.process - Process 'unbiased-yak' exited cleanly.
Agent stopped!
11:44:31.154 | DEBUG   | prefect.client - Using ephemeral application with database at sqlite+aiosqlite:////Users/mz/.prefect/orion.db

@zanieb zanieb requested a review from a team as a code owner February 2, 2023 17:45
@zanieb zanieb added the development Tech debt, refactors, CI, tests, and other related work. label Feb 2, 2023
@netlify
Copy link

netlify bot commented Feb 2, 2023 •
edited
Loading

✅ Deploy Preview for prefect-orion ready!

Name Link
🔨 Latest commit 6788199
🔍 Latest deploy log https://app.netlify.com/sites/prefect-orion/deploys/63dc4683e8f9f10007c6e755
😎 Deploy Preview https://deploy-preview-8374--prefect-orion.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

github-advanced-security[bot]
github-advanced-security bot found potential problems Feb 2, 2023
View reviewed changes
flows/agent.py
subprocess.run(
["prefect", "agent", "start", "--work-queue", "default"], timeout=30
)
except subprocess.TimeoutExpired:

Check notice

Code scanning / CodeQL

Empty except

'except' clause does nothing but pass and there is no explanatory comment.
github-advanced-security[bot]
github-advanced-security bot found potential problems Feb 2, 2023
View reviewed changes
flows/agent.py Fixed Show fixed Hide fixed
github-advanced-security[bot]
github-advanced-security bot found potential problems Feb 2, 2023
View reviewed changes
flows/agent.py
subprocess.run(["prefect", "work-queue", "create", "default"])
try:
subprocess.run(["prefect", "agent", "start", "default"], timeout=30)
except subprocess.TimeoutExpired:

Check notice

Code scanning / CodeQL

Empty except

'except' clause does nothing but pass and there is no explanatory comment.
@zanieb zanieb merged commit 3f6c0c0 into main Feb 2, 2023
@zanieb zanieb deleted the agent-integration-test branch February 2, 2023 23:48
@zanieb zanieb mentioned this pull request Feb 3, 2023
Merged
3 tasks
zanieb added a commit that referenced this pull request Feb 3, 2023
@zanieb
Add coverage for flow run with agent to integration test (#8374)
6738bda
zanieb added a commit that referenced this pull request Feb 3, 2023
@zanieb
Add coverage for flow run with agent to integration test (#8374)
93869cf
serinamarie pushed a commit that referenced this pull request Feb 6, 2023
@zanieb @serinamarie
Add coverage for flow run with agent to integration test (#8374)
286e617
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@serinamarie serinamarie serinamarie approved these changes

@anticorrelator anticorrelator anticorrelator approved these changes

Assignees
No one assigned
Labels
development Tech debt, refactors, CI, tests, and other related work.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@zanieb @anticorrelator @serinamarie