Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Prefect Cloud Quickstart tutorial #8227

Merged
merged 9 commits into from
Jan 26, 2023
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/concepts/settings.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ PREFECT_API_URL="https://api.prefect.cloud/api/accounts/[ACCOUNT-ID]/workspaces/

### PREFECT_API_KEY

The `PREFECT_API_KEY` value specifies the [API key](/ui/cloud-getting-started/#create-an-api-key) used to authenticate with your Prefect Cloud workspace.
The `PREFECT_API_KEY` value specifies the [API key](/ui/cloud-api-keys/#create-an-api-key) used to authenticate with your Prefect Cloud workspace.

```bash
PREFECT_API_KEY="[API-KEY]"
Expand Down
2 changes: 1 addition & 1 deletion docs/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ To achieve this, we've leveraged the familiar tools of native Python: first clas

### If I’m using Prefect Cloud 2, do I still need to run Orion locally?

No, Prefect Cloud 2 hosts an instance of Orion for you. In fact, each workspace in Prefect Could 2 corresponds directly to a single instance of Prefect Orion. See [Getting Started with Prefect Cloud](/ui/cloud-getting-started/) for more information.
No, Prefect Cloud 2 hosts an instance of Orion for you. In fact, each workspace in Prefect Could 2 corresponds directly to a single instance of Prefect Orion. See the [Prefect Cloud Overview](/ui/cloud/) for more information.


## Features
Expand Down
2 changes: 1 addition & 1 deletion docs/getting-started/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ That's it! You're ready to [start writing local flows](/tutorials/first-steps/).

If you want to start running flows on a schedule, via the API, from the Prefect UI, or on distributed infrastructure, you'll need to understand a few additional concepts and perform some configuration.

- Start a [Prefect Orion API server](/ui/overview/) with `prefect orion start` or create a [free Prefect Cloud account](/ui/cloud-getting-started/).
- Start a [Prefect Orion API server](/ui/overview/) with `prefect orion start` or create a [free Prefect Cloud account](/ui/cloud-quickstart/).
- [Set `PREFECT_API_URL`](/tutorials/orion/#running-the-prefect-server) to enable communication between your execution environment and the Prefect Orion server or Prefect Cloud API.
- Configure [storage](/tutorials/storage/) to persist flow and task data.
- Create a [deployment](/tutorials/deployments/) for a flow, giving the API metadata about where your flow's code is stored and how your flow should be run.
Expand Down
Binary file added docs/img/ui/cloud-flow-run-details.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/img/ui/cloud-flow-run.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/img/ui/cloud-new-workspace.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/ui/cloud-sign-in.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/img/ui/cloud-workspace-details.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion docs/migration-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ Every time you run a flow, whether it is tracked by the API server or ad-hoc thr
With Prefect 2, your functions *are* your flows and tasks. Prefect 2 automatically detects your flows and tasks without the need to define a rigid DAG structure. While use of tasks is encouraged to provide you the maximum visibility into your workflows, they are no longer required. You can add a single `@flow` decorator to your main function to transform any Python script into a Prefect workflow.

### Incremental adoption
The built-in SQLite database automatically tracks all your locally executed flow runs. As soon as you start Prefect Orion and open the Prefect UI in your browser (or [authenticate your CLI with your Prefect Cloud workspace](../ui/cloud-getting-started/)), you can see all your locally executed flow runs in the UI. You don't even need to start an agent.
The built-in SQLite database automatically tracks all your locally executed flow runs. As soon as you start Prefect Orion and open the Prefect UI in your browser (or [authenticate your CLI with your Prefect Cloud workspace](/ui/cloud/)), you can see all your locally executed flow runs in the UI. You don't even need to start an agent.

Then, when you want to move toward scheduled, repeatable workflows, you can build a deployment and send it to the server by running a CLI command or a Python script.

Expand Down
41 changes: 36 additions & 5 deletions docs/tutorials/deployments.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
description: Learn how to create Prefect deployments and execute flow runs with agents and work queues.
description: Learn how Prefect flow deployments enable configuring flows for scheduled and remote execution.
tags:
- Orion
- work queues
Expand All @@ -11,11 +11,16 @@ tags:
- tutorial
---

# Deployments
# Flow deployments

In the tutorials leading up to this one, you've been able to explore Prefect capabilities like flows, tasks, retries, caching, and so on. But so far, you've run flows as scripts.

[Deployments](/concepts/deployments/) take your flows to the next level: adding the information needed for scheduling flow runs or triggering a flow run via an API call. Deployments elevate workflows from functions that you call manually to API-managed entities.
[Deployments](/concepts/deployments/) take your flows to the next level: adding the information needed for scheduling flow runs or triggering a flow run via an API call. Deployments elevate workflows from functions that you call manually to API-managed entities. Deployments also enable remote flow run execution.

!!! cloud-ad "Run deployments with Prefect Cloud"
The same steps demonstrated in this tutorial work to apply deployments and create flow runs from them with Prefect Cloud.

See the [Prefect Cloud Quickstart](/ui/cloud-quickstart/) for step-by-step instructions to log into Prefect Cloud, create a workspace, and configure your local environment to use Prefect Cloud as the API backend. Then run through this tutorial again, using Prefect Cloud instead of a local Prefect Orion server.

## Components of a deployment

Expand Down Expand Up @@ -266,7 +271,7 @@ To review, we have four files that make up the artifacts for this particular dep
- The ignore file `.prefectignore`
- The deployment definition in `log_flow-deployment.yaml`

Now we can _apply_ the settings in `log_flow-deployment.yaml` to create the deployment object on the Prefect Orion server API — or on a [Prefect Cloud workspace](/ui/cloud-getting-started/) if you had configured the Prefect Cloud API as your backend.
Now we can _apply_ the settings in `log_flow-deployment.yaml` to create the deployment object on the Prefect Orion server API — or on a [Prefect Cloud workspace](/ui/cloud/) if you had [configured the Prefect Cloud API](/ui/cloud-local-environment/) as your backend.

Use the `prefect deployment apply` command to create the deployment on the Prefect server, specifying the name of the `log_flow-deployment.yaml` file.

Expand Down Expand Up @@ -413,7 +418,7 @@ INFO: Uvicorn running on http://127.0.0.1:4200 (Press CTRL+C to quit)
</div>

!!! note "Set the `PREFECT_API_URL` for your server"
Note the message to set `PREFECT_API_URL` &mdash; configuring the URL of your Prefect Orion server or Prefect Cloud makes sure that you're coordinating flows with the correct API instance.
Note the message to set `PREFECT_API_URL`, configuring the URL of your Prefect Orion server or Prefect Cloud makes sure that you're coordinating flows with the correct API instance.

Go to your first terminal session and run this command to set the API URL to point to the Prefect Orion instance you just started:

Expand Down Expand Up @@ -607,6 +612,32 @@ Select the flow run to see details. In the flow run logs, you can see that the f

![The flow run logs show the expected Hello Trillian! log message](/img/tutorials/dep-flow-logs.png)

## Run deployments with Prefect Cloud

The same steps demonstrated in this tutorial work to apply deployments and create flow runs from them with Prefect Cloud.
tpdorsey marked this conversation as resolved.
Show resolved Hide resolved

See the [Prefect Cloud Quickstart](/ui/cloud-quickstart/) for step-by-step instructions to log into Prefect Cloud, create a workspace, and configure your local environment to use Prefect Cloud as the API backend. Then run through this tutorial again, using Prefect Cloud instead of a local Prefect Orion server.

Already have a Prefect Cloud account? Logging in from your local development environment is as easy as `prefect cloud login`:

<div class="terminal">
```shell
$ prefect cloud login
? How would you like to authenticate? [Use arrows to move; enter to select]
> Log in with a web browser
Paste an API key
Opening browser...
Waiting for response...
? Which workspace would you like to use? [Use arrows to move; enter to select]
> prefect/terry-prefect-workspace
g-gadflow/g-workspace
Authenticated with Prefect Cloud! Using workspace 'prefect/terry-prefect-workspace'.
```
</div>

!!! tip "Blocks and deployments are specific to a server or Prefect Cloud workspace"
Note that, if you ran through this tutorial on a local Prefect Orion server instance, the storage and infrastructure blocks you created would not also be configured on Prefect Cloud. You must configure new storage and infrastructure blocks for any Prefect Cloud workspace.

## Next steps

So far you've seen a simple example of a single deployment for a single flow. But a common and useful pattern is to create multiple deployments for a flow. By using tags, parameters, and schedules effectively, you can have a single flow definition that serves multiple purposes or can be configured to run in different environments.
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorials/orion.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ There are numerous ways to begin exploring the API:

- Navigate to [http://127.0.0.1:4200/docs](http://127.0.0.1:4200/docs) (or your corresponding API URL) to see the autogenerated Swagger API documentation.
- Navigate to [http://127.0.0.1:4200/redoc](http://127.0.0.1:4200/redoc) (or your corresponding API URL) to see the autogenerated Redoc API documentation.
- Instantiate [an asynchronous `OrionClient`][prefect.client] within Python to send requests to the API.
- Instantiate [an asynchronous `OrionClient`][/api-ref/prefect/client/orion/#prefect.client.orion.OrionClient] within Python to send requests to the API.

To stop an instance of the Orion API server, simply **CTRL+C** to end the process in your terminal, or close the terminal session.

Expand Down
3 changes: 3 additions & 0 deletions docs/tutorials/storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,9 @@ Select **Create** to create the new storage block. Prefect displays the details

![Viewing details of a new S3 storage block in the Prefect UI](/img/tutorials/new-storage-block.png)

!!! tip "Blocks and deployments are specific to a server or Prefect Cloud workspace"
Note that, if you ran through this tutorial on a local Prefect Orion server instance, the storage and infrastructure blocks you created would not also be configured on Prefect Cloud. You must configure new storage and infrastructure blocks for any Prefect Cloud workspace.

## Using storage blocks with deployments

To demonstrate using a storage block, we'll create a new variation of the deployment for the `log_flow` example from the [deployments tutorial](/tutorials/deployments/). For this deployment, we'll specify using the storage block created earlier by passing `-sb s3/log-test` or `--storage-block s3/log-test` to the `prefect deployment build` command.
Expand Down
36 changes: 36 additions & 0 deletions docs/ui/cloud-api-keys.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
---
description: Create an API key to access Prefect Cloud from a local execution environment.
icon: material/cloud-outline
tags:
- Prefect Cloud
- API keys
- configuration
---

# Manage Prefect Cloud API Keys <span class="badge cloud"></span>

API keys enable you to authenticate an a local environment to work with Prefect Cloud. See [Configure execution environment](#configure-execution-environment) for details on how API keys are configured in your execution environment.

See [Configure Local Environments](/ui/cloud-local-environment/) for details on setting up access to Prefect Cloud from a local development environment or remote execution environment.

## Create an API key

To create an API key, select the account icon at the bottom-left corner of the UI and select your account name. This displays your account profile.

Select the **API Keys** tab. This displays a list of previously generated keys and lets you create new API keys or delete keys.

![Viewing and editing API keys in the Cloud UI.](../img/ui/cloud-api-keys.png)

Select the **+** button to create a new API key. You're prompted to provide a name for the key and, optionally, an expiration date. Select **Create API Key** to generate the key.

![Creating an API key in the Cloud UI.](../img/ui/cloud-new-api-key.png)

Note that API keys cannot be revealed again in the UI after you generate them, so copy the key to a secure location.

## Service account API keys <span class="badge orgs"></span>

Service accounts are a feature of Prefect Cloud [organizations](/ui/organizations/) that enable you to create a Prefect Cloud API key that is not associated with a user account.

Service accounts are typically used to configure API access for running agents or executing flow runs on remote infrastructure. Events and logs for flow runs in those environments are then associated with the service account rather than a user, and API access may be managed or revoked by configuring or removing the service account without disrupting user access.

See the [service accounts](/ui/service-accounts/) documentation for more information about creating and managing service accounts in a Prefect Cloud organization.
Loading