Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Docs rli bug #6008

Merged
merged 2 commits into from
Nov 14, 2024
Merged

Docs rli bug #6008

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3766d1a
add readthedocs env vars to conf.py
cosmicBboy Nov 13, 2024
069501e
use literalinclude instead of rli
cosmicBboy Nov 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,11 @@ This creates a new environment called `monodocs-env` with all the dependencies n

In the `flyteorg/flyte` root directory make sure you have activated the `monodocs-env` (or whatever you called it) environment and do:

```bash
# need to set this to a fake value to build the docs locally
$ export DOCSEARCH_API_KEY=fake-api-key
```

```bash
$ make docs
```
Expand Down
6 changes: 3 additions & 3 deletions docs/community/contribute/contribute_docs.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,14 +82,14 @@ The following are some tips to include various content:
* **Source code references (Embedded format)** <br>
`.rst` example:
```{code-block}
.. rli:: https://raw.githubusercontent.com/flyteorg/<source repo name>/<git sha>/<target file path>
.. literalinclude:: /examples/<target file path>
:lines: <from line>-<to line>
```

`.md` example:
````{code-block}
```{rli} https://raw.githubusercontent.com/flyteorg/<source repo name>/<git sha>/<target file path>
lines: <from line>-<to line>
```{literalinclude} /examples/<target file path>
:lines: <from line>-<to line>
```
````

Expand Down
33 changes: 25 additions & 8 deletions docs/conf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -343,6 +343,22 @@
"flytesnacks/README.md",
"flytekit/**/README.md",
"flytekit/_templates/**",
"examples/advanced_composition/**",
"examples/basics/**",
"examples/customizing_dependencies/**",
"examples/data_types_and_io/**",
"examples/development_lifecycle/**",
"examples/extending/**",
"examples/productionizing/**",
"examples/testing/**",
"flytesnacks/examples/advanced_composition/*.md",
"flytesnacks/examples/basics/*.md",
"flytesnacks/examples/customizing_dependencies/*.md",
"flytesnacks/examples/data_types_and_io/*.md",
"flytesnacks/examples/development_lifecycle/*.md",
"flytesnacks/examples/extending/*.md",
"flytesnacks/examples/productionizing/*.md",
"flytesnacks/examples/testing/*.md",
"api/flytectl/index.rst",
"protos/boilerplate/**",
"protos/tmp/**",
Expand Down Expand Up @@ -622,14 +638,6 @@
"flytesnacks/_build",
"flytesnacks/_tags",
"flytesnacks/index.md",
"examples/advanced_composition",
"examples/basics",
"examples/customizing_dependencies",
"examples/data_types_and_io",
"examples/development_lifecycle",
"examples/extending",
"examples/productionizing",
"examples/testing"
]
],
"local": flytesnacks_local_path is not None,
Expand Down Expand Up @@ -690,6 +698,15 @@
# Disable warnings from tensorflow
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3"

# Define the canonical URL if you are using a custom domain on Read the Docs
html_baseurl = os.environ.get("READTHEDOCS_CANONICAL_URL", "")

# Tell Jinja2 templates the build is running on Read the Docs
if os.environ.get("READTHEDOCS", "") == "True":
if "html_context" not in globals():
html_context = {}
html_context["READTHEDOCS"] = True


class CustomWarningSuppressor(logging.Filter):
"""Filter logs by `suppress_warnings`."""
Expand Down
10 changes: 5 additions & 5 deletions docs/user_guide/advanced_composition/decorating_workflows.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ To clone and run the example code on this page, see the [Flytesnacks repo][flyte

To begin, import the necessary libraries.

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/decorating_workflows.py
```{literalinclude} /examples/advanced_composition/advanced_composition/decorating_workflows.py
:caption: advanced_composition/decorating_workflows.py
:lines: 1-6
```
Expand All @@ -32,7 +32,7 @@ Let's define the tasks we need for setup and teardown. In this example, we use t
{py:class}`unittest.mock.MagicMock` class to create a fake external service that we want to initialize at the
beginning of our workflow and finish at the end.

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/decorating_workflows.py
```{literalinclude} /examples/advanced_composition/advanced_composition/decorating_workflows.py
:caption: advanced_composition/decorating_workflows.py
:lines: 9-21
```
Expand All @@ -45,7 +45,7 @@ external service and Flyte.

We create a decorator that we want to use to wrap our workflow function.

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/decorating_workflows.py
```{literalinclude} /examples/advanced_composition/advanced_composition/decorating_workflows.py
:caption: advanced_composition/decorating_workflows.py
:pyobject: setup_teardown
```
Expand All @@ -66,14 +66,14 @@ There are a few key pieces to note in the `setup_teardown` decorator above:

We define two tasks that will constitute the workflow.

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/decorating_workflows.py
```{literalinclude} /examples/advanced_composition/advanced_composition/decorating_workflows.py
:caption: advanced_composition/decorating_workflows.py
:lines: 63-70
```

And then create our decorated workflow:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/decorating_workflows.py
```{literalinclude} /examples/advanced_composition/advanced_composition/decorating_workflows.py
:caption: advanced_composition/decorating_workflows.py
:lines: 74-82
```
Expand Down
18 changes: 9 additions & 9 deletions docs/user_guide/advanced_composition/eager_workflows.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ using the `@eager` decorator.
To clone and run the example code on this page, see the [Flytesnacks repo][flytesnacks].
```

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/eager_workflows.py
```{literalinclude} /examples/advanced_composition/advanced_composition/eager_workflows.py
:caption: advanced_composition/eager_workflows.py
:lines: 1-21
```
Expand Down Expand Up @@ -116,7 +116,7 @@ One of the biggest benefits of eager workflows is that you can now materialize
task and subworkflow outputs as Python values and do operations on them just
like you would in any other Python function. Let's look at another example:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/eager_workflows.py
```{literalinclude} /examples/advanced_composition/advanced_composition/eager_workflows.py
:caption: advanced_composition/eager_workflows.py
:pyobject: another_eager_workflow
```
Expand All @@ -131,7 +131,7 @@ As you saw in the `simple_eager_workflow` workflow above, you can use regular
Python conditionals in your eager workflows. Let's look at a more complicated
example:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/eager_workflows.py
```{literalinclude} /examples/advanced_composition/advanced_composition/eager_workflows.py
:caption: advanced_composition/eager_workflows.py
:lines: 36-53
```
Expand All @@ -144,7 +144,7 @@ to check if `out` is negative, but we're also using the `gt_100` task in the

You can also gather the outputs of multiple tasks or subworkflows into a list:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/eager_workflows.py
```{literalinclude} /examples/advanced_composition/advanced_composition/eager_workflows.py
:caption: advanced_composition/eager_workflows.py
:lines: 58-69
```
Expand All @@ -153,7 +153,7 @@ You can also gather the outputs of multiple tasks or subworkflows into a list:

You can also invoke static workflows from within an eager workflow:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/eager_workflows.py
```{literalinclude} /examples/advanced_composition/advanced_composition/eager_workflows.py
:caption: advanced_composition/eager_workflows.py
:lines: 74-84
```
Expand All @@ -162,7 +162,7 @@ You can also invoke static workflows from within an eager workflow:

You can have nest eager subworkflows inside a parent eager workflow:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/eager_workflows.py
```{literalinclude} /examples/advanced_composition/advanced_composition/eager_workflows.py
:caption: advanced_composition/eager_workflows.py
:lines: 89-97
```
Expand All @@ -171,7 +171,7 @@ You can have nest eager subworkflows inside a parent eager workflow:

You can also catch exceptions in eager workflows through `EagerException`:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/eager_workflows.py
```{literalinclude} /examples/advanced_composition/advanced_composition/eager_workflows.py
:caption: advanced_composition/eager_workflows.py
:lines: 102-117
```
Expand All @@ -195,7 +195,7 @@ and remotely.
You can execute eager workflows locally by simply calling them like a regular
`async` function:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/eager_workflows.py
```{literalinclude} /examples/advanced_composition/advanced_composition/eager_workflows.py
:caption: advanced_composition/eager_workflows.py
:lines: 123-125
```
Expand Down Expand Up @@ -244,7 +244,7 @@ When using a sandbox cluster started with `flytectl demo start`, however, the
`client_secret_group` and `client_secret_key` are not required, since the
default sandbox configuration does not require key-based authentication.

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/eager_workflows.py
```{literalinclude} /examples/advanced_composition/advanced_composition/eager_workflows.py
:caption: advanced_composition/eager_workflows.py
:lines: 130-145
```
Expand Down
8 changes: 4 additions & 4 deletions docs/user_guide/advanced_composition/intratask_checkpoints.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,14 +51,14 @@ To clone and run the example code on this page, see the [Flytesnacks repo][flyte

To begin, import the necessary libraries and set the number of task retries to `3`:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/checkpoint.py
```{literalinclude} /examples/advanced_composition/advanced_composition/checkpoint.py
:caption: advanced_composition/checkpoint.py
:lines: 1-4
```

We define a task to iterate precisely `n_iterations`, checkpoint its state, and recover from simulated failures:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/checkpoint.py
```{literalinclude} /examples/advanced_composition/advanced_composition/checkpoint.py
:caption: advanced_composition/checkpoint.py
:pyobject: use_checkpoint
```
Expand All @@ -69,14 +69,14 @@ The checkpoint system offers additional APIs, documented in the code accessible
Create a workflow that invokes the task:
The task will automatically undergo retries in the event of a {ref}`FlyteRecoverableException <flytekit:exception_handling>`.

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/checkpoint.py
```{literalinclude} /examples/advanced_composition/advanced_composition/checkpoint.py
:caption: advanced_composition/checkpoint.py
:pyobject: checkpointing_example
```

The local checkpoint is not utilized here because retries are not supported:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/checkpoint.py
```{literalinclude} /examples/advanced_composition/advanced_composition/checkpoint.py
:caption: advanced_composition/checkpoint.py
:lines: 37-42
```
Expand Down
14 changes: 7 additions & 7 deletions docs/user_guide/advanced_composition/map_tasks.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,14 @@ To clone and run the example code on this page, see the [Flytesnacks repo][flyte

To begin, import the required libraries:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/map_task.py
```{literalinclude} /examples/advanced_composition/advanced_composition/map_task.py
:caption: advanced_composition/map_task.py
:lines: 1
```

Here's a simple workflow that uses {py:func}`map_task <flytekit:flytekit.map_task>`:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/map_task.py
```{literalinclude} /examples/advanced_composition/advanced_composition/map_task.py
:caption: advanced_composition/map_task.py
:lines: 4-19
```
Expand Down Expand Up @@ -82,7 +82,7 @@ When defining a map task, avoid calling other tasks in it. Flyte can't accuratel

In this example, the map task `suboptimal_mappable_task` would not give you the best performance:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/map_task.py
```{literalinclude} /examples/advanced_composition/advanced_composition/map_task.py
:caption: advanced_composition/map_task.py
:lines: 31-40
```
Expand All @@ -98,7 +98,7 @@ You might need to map a task with multiple inputs.

For instance, consider a task that requires three inputs:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/map_task.py
```{literalinclude} /examples/advanced_composition/advanced_composition/map_task.py
:caption: advanced_composition/map_task.py
:pyobject: multi_input_task
```
Expand All @@ -107,21 +107,21 @@ You may want to map this task with only the ``quantity`` input, while keeping th
Since a map task accepts only one input, you can achieve this by partially binding values to the map task.
This can be done using the {py:func}`functools.partial` function:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/map_task.py
```{literalinclude} /examples/advanced_composition/advanced_composition/map_task.py
:caption: advanced_composition/map_task.py
:lines: 52-58
```

Another possibility is to bind the outputs of a task to partials:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/map_task.py
```{literalinclude} /examples/advanced_composition/advanced_composition/map_task.py
:caption: advanced_composition/map_task.py
:lines: 63-72
```

You can also provide multiple lists as input to a `map_task`:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/map_task.py
```{literalinclude} /examples/advanced_composition/advanced_composition/map_task.py
:caption: advanced_composition/map_task.py
:pyobject: map_workflow_with_lists
```
Expand Down
12 changes: 6 additions & 6 deletions docs/user_guide/advanced_composition/subworkflows.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ To clone and run the example code on this page, see the [Flytesnacks repo][flyte

Here's an example illustrating the calculation of slope, intercept and the corresponding y-value:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/subworkflow.py
```{literalinclude} /examples/advanced_composition/advanced_composition/subworkflow.py
:caption: advanced_composition/subworkflow.py
:lines: 1-35
```
Expand All @@ -34,7 +34,7 @@ Subsequently, the `regression_line_wf` triggers `slope_intercept_wf` and then co

To execute the workflow locally, use the following:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/subworkflow.py
```{literalinclude} /examples/advanced_composition/advanced_composition/subworkflow.py
:caption: advanced_composition/subworkflow.py
:lines: 39-40
```
Expand All @@ -43,14 +43,14 @@ It's possible to nest a workflow that contains a subworkflow within another work
Workflows can be easily constructed from other workflows, even if they function as standalone entities.
Each workflow in this module has the capability to exist and run independently:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/subworkflow.py
```{literalinclude} /examples/advanced_composition/advanced_composition/subworkflow.py
:caption: advanced_composition/subworkflow.py
:pyobject: nested_regression_line_wf
```

You can run the nested workflow locally as well:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/subworkflow.py
```{literalinclude} /examples/advanced_composition/advanced_composition/subworkflow.py
:caption: advanced_composition/subworkflow.py
:lines: 52-53
```
Expand All @@ -71,7 +71,7 @@ external workflows may offer a way to distribute the workload of a workflow acro

Here's an example that illustrates the concept of external workflows:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/subworkflow.py
```{literalinclude} /examples/advanced_composition/advanced_composition/subworkflow.py
:caption: advanced_composition/subworkflow.py
:lines: 61-71
```
Expand All @@ -85,7 +85,7 @@ In the console screenshot above, note that the launch plan execution ID differs

You can run a workflow containing an external workflow locally as follows:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/subworkflow.py
```{literalinclude} /examples/advanced_composition/advanced_composition/subworkflow.py
:caption: advanced_composition/subworkflow.py
:lines: 75-76
```
Expand Down
Loading
Loading