Skip to content

Commit

Permalink
add defer page to cloud develop docs (#4229)
Browse files Browse the repository at this point in the history
@dave-connors-3's pr which adds a doc explaining how def works in the
ide and cloud cli

## Checklist
[ ] needs Dave Connors review
  • Loading branch information
mirnawong1 authored Oct 11, 2023
2 parents e21411c + e3e453f commit 639cb71
Show file tree
Hide file tree
Showing 7 changed files with 63 additions and 3 deletions.
55 changes: 55 additions & 0 deletions website/docs/docs/cloud/about-cloud-develop-defer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
---
title: Using defer in dbt Cloud
id: about-cloud-develop-defer
description: "Learn how to leverage defer to prod when developing with dbt Cloud."
sidebar_label: "Using defer in dbt Cloud"
pagination_next: "docs/cloud/cloud-cli-installation"
---


[Defer](/reference/node-selection/defer) is a powerful feature that allows developers to only build and run and test models they've edited without having to first run and build all the models that come before them (upstream parents). This is powered by using a production manifest for comparison, and dbt will resolve the `{{ ref() }}` function with upstream production artifacts.

Both the dbt Cloud IDE and the dbt Cloud CLI allow users to natively defer to production metadata directly in their development workflows, dramatically reducing development time and warehouse spend by preventing unnecessary model builds.

## Required setup

- You must select the **[Production environment](/docs/deploy/deploy-environments#set-as-production-environment-beta)** checkbox in the **Environment Settings** page.
- This can be set for one deployment environment per dbt Cloud project.
- You must have a successful job run first.

When using 'defer', it compares artifacts from the most recent successful production job, excluding CI jobs.

### Defer in the dbt Cloud IDE

To enable 'Defer' in the dbt Cloud IDE, toggle the **Defer to production** button on the command bar. Once enabled, dbt Cloud will:

1. Pull down the most recent manifest from the Production environment for comparison
2. Pass the `--defer` flag to the command (for any command that accepts the flag)

For example, if you were to start developing on a new branch with [nothing in your development schema](/reference/node-selection/defer#usage), edit a single model, and run `dbt build -s state:modified` — only the edited model would run. Any `{{ ref() }}` functions will point to the production location of the referenced models.

<Lightbox src="/img/docs/dbt-cloud/defer-toggle.jpg" width="100%" title="Select the 'Defer to production' toggle on the botom right of the command bar to enable defer in the dbt Cloud IDE."/>

### Defer in dbt Cloud CLI

One key difference between using `--defer` in the dbt Cloud CLI and the dbt Cloud IDE is that `--defer` is *automatically* enabled in the dbt Cloud CLI for all invocations, comparing with production artifacts. You can disable it with the `--no-defer` flag.

The dbt Cloud CLI offers additional flexibility by letting you choose the source environment for deferral artifacts. You can set a `defer-env-id` key in either your `dbt_project.yml` or `dbt_cloud.yml` file. If you do not provide a `defer-env-id` setting, the dbt Cloud CLI will use artifacts from your dbt Cloud environment marked 'Production'.

<File name="dbt_cloud.yml">

```yml
dever-env-id: '123456'
```
</File>
<File name="dbt_project.yml">
```yml
dbt_cloud:
dever-env-id: '123456'
```
</File>
2 changes: 2 additions & 0 deletions website/docs/docs/cloud/about-cloud-develop.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,7 @@ dbt Cloud offers a fast and reliable way to work on your dbt project. It runs db

The following sections provide detailed instructions on setting up the dbt Cloud CLI and dbt Cloud IDE. To get started with dbt development, you'll need a [developer](/docs/cloud/manage-access/seats-and-users) account. For a more comprehensive guide about developing in dbt, refer to our [quickstart guides](/quickstarts).


---------
**Note**: The dbt Cloud CLI and the open-sourced dbt Core are both command line tools that let you run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features).

2 changes: 1 addition & 1 deletion website/docs/docs/cloud/cloud-cli-installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ dbt Cloud natively supports developing using a command line (CLI), empowering te
dbt commands are run against dbt Cloud's infrastructure and benefit from:

* Secure credential storage in the dbt Cloud platform.
* Automatic deferral of build artifacts to your Cloud project's production environment.
* [Automatic deferral](/docs/cloud/about-cloud-develop-defer) of build artifacts to your Cloud project's production environment.
* Speedier, lower-cost builds.
* Support for dbt Mesh ([cross-project `ref`](/docs/collaborate/govern/project-dependencies)),
* Significant platform improvements, to be released over the coming months.
Expand Down
6 changes: 4 additions & 2 deletions website/docs/docs/cloud/dbt-cloud-ide/ide-user-interface.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,13 @@ The IDE streamlines your workflow, and features a popular user interface layout
* Added (A) — The IDE detects added files
* Deleted (D) — The IDE detects deleted files.

<Lightbox src="/img/docs/dbt-cloud/cloud-ide/ide-cmd-status.jpg" width="120%" title="Use the Command bar to write dbt commands and the status button for the current IDE status"/>
<Lightbox src="/img/docs/dbt-cloud/cloud-ide/ide-command-bar.jpg" width="120%" title="Use the Command bar to write dbt commands, toggle 'Defer', and view the current IDE status"/>

5. **Command bar &mdash;** The Command bar, located in the lower left of the IDE, is used to invoke [dbt commands](/reference/dbt-commands). When a command is invoked, the associated logs are shown in the Invocation History Drawer.

6. **IDE Status button &mdash;** The IDE Status button, located on the lower right of the IDE, displays the current IDE status. If there is an error in the status or in the dbt code that stops the project from parsing, the button will turn red and display "Error". If there aren't any errors, the button will display a green "Ready" status. To access the [IDE Status modal](#modals-and-menus), simply click on this button.
6. **Defer to production &mdash;** The **Defer to production** toggle allows developers to only build and run and test models they've edited without having to first run and build all the models that come before them (upstream parents). Refer to [Using defer in dbt Cloud](/docs/cloud/about-cloud-develop-defer#defer-in-the-dbt-cloud-ide) for more info.

7. **Status button &mdash;** The IDE Status button, located on the lower right of the IDE, displays the current IDE status. If there is an error in the status or in the dbt code that stops the project from parsing, the button will turn red and display "Error". If there aren't any errors, the button will display a green "Ready" status. To access the [IDE Status modal](#modals-and-menus), simply click on this button.


## Editing features
Expand Down
1 change: 1 addition & 0 deletions website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@ const sidebarSettings = {
link: { type: "doc", id: "docs/cloud/about-cloud-develop" },
items: [
"docs/cloud/about-cloud-develop",
"docs/cloud/about-cloud-develop-defer",
{
type: "category",
label: "dbt Cloud CLI",
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 639cb71

Please sign in to comment.