Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Automatically set --event-start-time and --event-end-time values if not provided #10636

Closed
1 task done
Tracked by #10624
QMalcolm opened this issue Aug 29, 2024 · 1 comment · Fixed by #10594
Closed
1 task done
Tracked by #10624
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@QMalcolm
Copy link
Contributor

QMalcolm commented Aug 29, 2024

Housekeeping

  • I am a maintainer of dbt-core

Short description

In #10624 we're building a new type of incremental model strategy, microbatch. In #10635 we allowed people to specify event_time_start and event_time_end via the CLI. However, the more common situation will be that neither are specified (or perhaps only one of them is specified).

In the case where event_time_end isn't specified, then the value of event_time_end should become now. In the case where event_time_start isn't specified, then the value of event_time_start should become the value of event_time_end truncated to the start of the batch_size definition.

In the case where neither is specified, what is listed above still applies. That is to say that if now is 2024-09-04 13:11:00 and my batch_size is day, then event_time_end would be 2024-09-04 13:11:00 and event_time_start would be 2024-09-04 00:00:00.

Acceptance criteria

  • event_time_end and event_time_start provided from the CLI is preferred
  • if event_time_end is not specified, then the value of event_time_end becomes now
  • if event_time_start is not specified, then the value of event_time_start becomes event_time_end truncated by the batch_size

Suggested Tests

  1. Values of event_time_end and event_time_end provided from the CLI are preffered
  2. If event_time_end is not provided, it is correctly defaulted
  3. If event_time_start is not provided it is correctly defaulted

Impact to Other Teams

N/A

Will backports be required?

No

Context

Depends on:

@QMalcolm QMalcolm added the enhancement New feature or request label Sep 4, 2024
@QMalcolm QMalcolm changed the title Automatically set --event-start-time and --event-end-time values if not provided [Feature] Automatically set --event-start-time and --event-end-time values if not provided Sep 4, 2024
@QMalcolm QMalcolm added this to the v1.9 milestone Sep 4, 2024
@QMalcolm
Copy link
Contributor Author

Resolved by #10594

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants