Skip to content

Commit

Permalink
Default value for schedule_interval (github#606)
Browse files Browse the repository at this point in the history
* Notes default valur for schedule_interval, plus general light editing.

* Add section about parallel copy tool

* Update api/add_continuous_aggregate_policy.md

Co-authored-by: Jacob Prall <prall.jacob@gmail.com>
  • Loading branch information
Loquacity and jacobprall authored Jan 31, 2022
1 parent bccb18e commit 3cbe2c7
Show file tree
Hide file tree
Showing 6 changed files with 100 additions and 109 deletions.
46 changes: 25 additions & 21 deletions api/add_continuous_aggregate_policy.md
Original file line number Diff line number Diff line change
@@ -1,40 +1,44 @@
## add_continuous_aggregate_policy() <tag type="community">Community</tag>

## add_continuous_aggregate_policy() <tag type="community">Community</tag>
Create a policy that automatically refreshes a continuous aggregate.

### Required Arguments
### Required arguments

|Name|Type|Description|
|---|---|---|
| `continuous_aggregate` | REGCLASS | The continuous aggregate to add the policy for. |
| `start_offset` | INTERVAL or integer | Start of the refresh window as an interval relative to the time when the policy is executed |
| `end_offset` | INTERVAL or integer | End of the refresh window as an interval relative to the time when the policy is executed |
| `schedule_interval` | INTERVAL | Interval between refresh executions in wall-clock time. |
|-|-|-|
|`continuous_aggregate`|REGCLASS|The continuous aggregate to add the policy for|
|`start_offset`|INTERVAL or integer|Start of the refresh window as an interval relative to the time when the policy is executed|
|`end_offset`|INTERVAL or integer|End of the refresh window as an interval relative to the time when the policy is executed|
|`schedule_interval`|INTERVAL|Interval between refresh executions in wall-clock time. Defaults to 24 hours|

The `start_offset` should be greater than `end_offset`.
The `start_offset` and `end_offset` parameters should be specified differently depending on the type of the time column of the hypertable:
- For hypertables with TIMESTAMP, TIMESTAMPTZ, and DATE time columns: the offset should be an INTERVAL type
- For hypertables with integer-based timestamps: the offset should be an integer type.

### Optional Arguments
You must specify the `start_offset` and `end_offset` parameters differently,
depending on the type of the time column of the hypertable:
* For hypertables with `TIMESTAMP`, `TIMESTAMPTZ`, and `DATE` time columns,
set the offset as an `INTERVAL` type
* For hypertables with integer-based timestamps, set the offset as
an `INTEGER` type.

### Optional arguments

|Name|Type|Description|
|---|---|---|
| `if_not_exists` | BOOLEAN | Set to true to avoid throwing an error if the continuous aggregate policy already exists. A notice is issued instead. Defaults to false. |
|-|-|-|
|`if_not_exists`|BOOLEAN|Set to `true `to issue a notice instead of an error if the job does not exist. Defaults to false.|

### Returns
### Returns

|Column|Type|Description|
|---|---|---|
|`job_id`| INTEGER | TimescaleDB background job id created to implement this policy|

|-|-|-|
|`job_id`|INTEGER|TimescaleDB background job ID created to implement this policy|

### Sample Usage

Add a policy that refreshes the last month once an hour, excluding the latest hour from the aggregate (for performance reasons, it is recommended to exclude buckets that still see lots of writes):
### Sample use
Add a policy that refreshes the last month once an hour, excluding the latest
hour from the aggregate. For performance reasons, we recommend that you
exclude buckets that see lots of writes:
```sql
SELECT add_continuous_aggregate_policy('conditions_summary',
start_offset => INTERVAL '1 month',
end_offset => INTERVAL '1 hour',
schedule_interval => INTERVAL '1 hour');
```
```
31 changes: 13 additions & 18 deletions api/add_job.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,29 @@
## add_job() <tag type="community">Community</tag>
Register an action for scheduling by the automation framework. For more information about scheduling, including example actions, see the [actions section][using-actions].

Register an action to be scheduled by our automation framework.
Please read the [instructions][using-actions] for more details including
multiple example actions.

### Required Arguments
### Required arguments

|Name|Type|Description|
|---|---|---|
| `proc` | REGPROC | Name of the function or procedure to register as job|
| `schedule_interval` | INTERVAL | Interval between executions of this job|
|-|-|-|
|`proc`|REGPROC|Name of the function or procedure to register as job|
|`schedule_interval`|INTERVAL|Interval between executions of this job. Defaults to 24 hours|

### Optional Arguments

|Name|Type|Description|
|---|---|---|
| `config` | JSONB | Job-specific configuration (this is passed to the function when executed) |
| `initial_start` | TIMESTAMPTZ | Time of first execution of job |
| `scheduled` | BOOLEAN | Set to `FALSE` to exclude this job from scheduling. Defaults to `TRUE`. |
|-|-|-|
|`config`|JSONB|Job-specific configuration, passed to the function when it runs|
|`initial_start`|TIMESTAMPTZ|Time the job is first run|
|`scheduled`|BOOLEAN|Set to `FALSE` to exclude this job from scheduling. Defaults to `TRUE`. |

### Returns

|Column|Type|Description|
|---|---|---|
|`job_id`| INTEGER | TimescaleDB background job id |

### Sample Usage
|-|-|-|
|`job_id`|INTEGER|TimescaleDB background job ID|

### Sample use
Register the `user_defined_action` procedure to run every hour:
```sql
CREATE OR REPLACE PROCEDURE user_defined_action(job_id int, config jsonb) LANGUAGE PLPGSQL AS
$$
Expand All @@ -38,7 +35,5 @@ $$;
SELECT add_job('user_defined_action','1h');
```

Register the procedure `user_defined_action` to be run every hour.


[using-actions]: /timescaledb/:currentVersion:/overview/core-concepts/user-defined-actions
76 changes: 36 additions & 40 deletions api/alter_job.md
Original file line number Diff line number Diff line change
@@ -1,63 +1,59 @@
## alter_job() <tag type="community">Community</tag>
Actions scheduled using the TimescaleDB automation framework run periodically in
a background worker. You can change the schedule of these jobs with the
`alter_job` function. To alter an existing job, refer to it by `job_id`. The
`job_id` runs a given action, and its current schedule can be found in the
`timescaledb_information.jobs` view, which lists information about every
scheduled action, as well as in `timescaledb_information.job_stats`. The
`job_stats` view also gives information about when each job was last run and
other useful statistics for deciding what the new schedule should be.

Actions scheduled via TimescaleDB's automation framework run periodically in a
background worker. You can change the schedule of their execution using `alter_job`.
To alter an existing job, you must refer to it by `job_id`.
The `job_id` which executes a given action and its current schedule can be found
either in the `timescaledb_information.jobs` view, which lists information
about every scheduled action, as well as in `timescaledb_information.job_stats`.
The `job_stats` view additionally contains information about when each job was
last run and other useful statistics for deciding what the new schedule should be.

### Required Arguments
### Required arguments

|Name|Type|Description|
|---|---|---|
| `job_id` | INTEGER | the id of the policy job being modified |
|-|-|-|
|`job_id`|INTEGER|The ID of the policy job being modified|

### Optional Arguments
### Optional arguments

|Name|Type|Description|
|---|---|---|
| `schedule_interval` | INTERVAL | The interval at which the job runs |
| `max_runtime` | INTERVAL | The maximum amount of time the job is allowed to run by the background worker scheduler before it is stopped |
| `max_retries` | INTEGER | The number of times the job is retried should it fail |
| `retry_period` | INTERVAL | The amount of time the scheduler waits between retries of the job on failure |
| `scheduled` | BOOLEAN | Set to `FALSE` to exclude this job from being run as background job. |
| `config` | JSONB | Job-specific configuration (this is passed to the function when executed)|
| `next_start` | TIMESTAMPTZ | The next time at which to run the job. The job can be paused by setting this value to 'infinity' (and restarted with a value of now()). |
| `if_exists` | BOOLEAN | Set to true to avoid throwing an error if the job does not exist, a notice is issued instead. Defaults to false. |
|-|-|-|
|`schedule_interval`|INTERVAL|The interval at which the job runs. Defaults to 24 hours|
|`max_runtime`|INTERVAL|The maximum amount of time the job is allowed to run by the background worker scheduler before it is stopped|
|`max_retries`|INTEGER|The number of times the job is retried if it fails|
|`retry_period`|INTERVAL|The amount of time the scheduler waits between retries of the job on failure|
|`scheduled`|BOOLEAN|Set to `FALSE` to exclude this job from being run as background job|
|`config`|JSONB|Job-specific configuration, passed to the function when it runs|
|`next_start`|TIMESTAMPTZ|The next time at which to run the job. The job can be paused by setting this value to `infinity`, and restarted with a value of `now()`|
|`if_exists`|BOOLEAN|Set to `true `to issue a notice instead of an error if the job does not exist. Defaults to false.|

### Returns

|Column|Type|Description|
|---|---|---|
| `job_id` | INTEGER | the id of the job being modified |
| `schedule_interval` | INTERVAL | The interval at which the job runs |
| `max_runtime` | INTERVAL | The maximum amount of time the job is allowed to run by the background worker scheduler before it is stopped |
| `max_retries` | INTEGER | The number of times the job is retried should it fail |
| `retry_period` | INTERVAL | The amount of time the scheduler waits between retries of the job on failure |
| `scheduled` | BOOLEAN | True if this job is executed by the TimescaleDB scheduler. |
| `config` | JSONB | Job-specific configuration (this is passed to the function when executed)|
| `next_start` | TIMESTAMPTZ | The next time at which to run the job. |

### Sample Usage

|-|-|-|
|`job_id`|INTEGER|The ID of the job being modified|
|`schedule_interval`|INTERVAL|The interval at which the job runs. Defaults to 24 hours|
|`max_runtime`|INTERVAL|The maximum amount of time the job is allowed to run by the background worker scheduler before it is stopped|
|`max_retries`|INTEGER|The number of times the job is retried if it fails|
|`retry_period`|INTERVAL|The amount of time the scheduler waits between retries of the job on failure|
|`scheduled`|BOOLEAN|Returns `true` if the job is executed by the TimescaleDB scheduler|
|`config`|JSONB|Job-specific configuration, passed to the function when it runs|
|`next_start`|TIMESTAMPTZ|The next time to run the job|

### Sample use
Reschedules job ID `1000` so that it runs every two days:
```sql
SELECT alter_job(1000, schedule_interval => INTERVAL '2 days');
```
Reschedules the job with id 1000 so that it runs every two days.

Disables scheduling of the compression policy on the `conditions` hypertable:
```sql
SELECT alter_job(job_id, scheduled => false)
FROM timescaledb_information.jobs
WHERE proc_name = 'policy_compression' AND hypertable_name = 'conditions'
```
Disables scheduling of the compression policy on hypertable `conditions`.

Reschedules continuous aggregate job ID `1000` so that it next runs at 9:00:00 on 15 March, 2020:
```sql
SELECT alter_job(1015, next_start => '2020-03-15 09:00:00.0+00');
SELECT alter_job(1000, next_start => '2020-03-15 09:00:00.0+00');
```

Reschedules continuous aggregate job `1015` so that the next execution of the
job starts at the specified time (9:00:00 am on March 15, 2020).
47 changes: 21 additions & 26 deletions api/jobs.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,30 @@
## timescaledb_information.jobs
Shows information about all jobs registered with the automation framework.

### Available Columns
### Arguments

|Name|Type|Description|
|---|---|---|
|`job_id` | INTEGER | The id of the background job |
|`application_name` | TEXT | Name of the policy or user defined action |
|`schedule_interval` | INTERVAL | The interval at which the job runs |
|`max_runtime` | INTERVAL | The maximum amount of time the job is allowed to run by the background worker scheduler before it is stopped |
|`max_retries` | INTEGER | The number of times the job is retried should it fail |
|`retry_period` | INTERVAL | The amount of time the scheduler waits between retries of the job on failure |
|`proc_schema` | TEXT | Schema name of the function or procedure executed by the job |
|`proc_name` | TEXT | Name of the function or procedure executed by the job |
|`owner` | TEXT | Owner of the job |
|`scheduled` | BOOLEAN | | Is the job scheduled to run automatically? |
|`config` | JSONB | | Configuration passed to the function specified by `proc_name` at execution time |
|`next_start` | TIMESTAMP WITH TIME ZONE | | Next start time for the job, if it is scheduled to run automatically |
|`hypertable_schema` | TEXT | Schema name of the hypertable. NULL, if this is a user defined action.|
|`hypertable_name` | TEXT | Table name of the hypertable. NULL, if this is a user defined action. |
|-|-|-|
|`job_id`|INTEGER|The ID of the background job|
|`application_name`|TEXT|Name of the policy or user defined action|
|`schedule_interval`|INTERVAL|The interval at which the job runs. Defaults to 24 hours|
|`max_runtime`|INTERVAL|The maximum amount of time the job is allowed to run by the background worker scheduler before it is stopped|
|`max_retries`|INTEGER|The number of times the job is retried if it fails|
|`retry_period`|INTERVAL|The amount of time the scheduler waits between retries of the job on failure|
|`proc_schema`|TEXT|Schema name of the function or procedure executed by the job|
|`proc_name`|TEXT|Name of the function or procedure executed by the job|
|`owner`|TEXT|Owner of the job|
|`scheduled`|BOOLEAN|Set to `true` to run the job automatically|
|`config`|JSONB|Configuration passed to the function specified by `proc_name` at execution time|
|`next_start`|TIMESTAMP WITH TIME ZONE|Next start time for the job, if it is scheduled to run automatically|
|`hypertable_schema`|TEXT|Schema name of the hypertable. Set to `NULL` for user-defined action|
|`hypertable_name`|TEXT|Table name of the hypertable. Set to `NULL` for user-defined action|

### Sample Usage
### Sample use

Get information about jobs.
Shows a job associated with the refresh policy for continuous aggregates:
```sql

SELECT * FROM timescaledb_information.jobs;
--This shows a job associated with the refresh policy for continuous aggregates
job_id | 1001
application_name | Refresh Continuous Aggregate Policy [1001]
schedule_interval | 01:00:00
Expand All @@ -42,13 +40,12 @@ days", "mat_hypertable_id": 2}
next_start | 2020-10-02 12:38:07.014042-04
hypertable_schema | _timescaledb_internal
hypertable_name | _materialized_hypertable_2

```
Find all jobs related to compression policies.

Find all jobs related to compression policies:

```sql
SELECT * FROM timescaledb_information.jobs where application_name like 'Compression%';

-[ RECORD 1 ]-----+--------------------------------------------------
job_id | 1002
application_name | Compression Policy [1002]
Expand All @@ -64,13 +61,11 @@ config | {"hypertable_id": 3, "compress_after": "60 days"}
next_start | 2020-10-18 01:31:40.493764-04
hypertable_schema | public
hypertable_name | conditions

```
Find jobs that are executed by user defined actions.

Find jobs that are run by user defined actions:
```sql
SELECT * FROM timescaledb_information.jobs where application_name like 'User-Define%';

-[ RECORD 1 ]-----+------------------------------
job_id | 1003
application_name | User-Defined Action [1003]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,11 @@ it is removed from the hypertable.
You can change the way your continuous aggregate is refreshed by adjusting the
[add_continuous_aggregate_policy][api-add-continuous-aggregate-policy].
The policy takes three arguments:
* `start_offset`: the start of the refresh window relative to when the policy runs
* `start_offset`: the start of the refresh window relative to when the policy
runs
* `end_offset`: the end of the refresh window relative to when the policy runs
* `schedule_interval`: the refresh interval in minutes or hours
* `schedule_interval`: the refresh interval in minutes or hours. Defaults to
24 hours.

If you set the `start_offset` or `end_offset` to NULL, the range is open-ended
and extends to the beginning or end of time. However, we recommend that you
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -418,8 +418,7 @@ automatically reconfigure your continuous aggregates to use the new framework.
In particular, the update process should:

* Maintain the same view schema, name, owner, and view definition as before, as well as the setting for `materialized_only`.
* Schedule the refresh to run at the same interval as before (although the parameter is now named `schedule_interval` r
ather than `refresh_interval`).
* Schedule the refresh to run at the same interval as before (although the parameter is now named `schedule_interval` rather than `refresh_interval`).
* Automatically configure `start_offset` to the same offset as specified by the old `ignore_invalidation_older_than` setting.
* Automatically configure `end_offset` to have an offset from `now()` equivalent to the old `refresh_lag` setting.
* Mark all the data older than the interval `ignore_invalidation_older_than` as out-of-date, so that it can be refreshed.
Expand Down

0 comments on commit 3cbe2c7

Please sign in to comment.