Skip to content

Commit

Permalink
feat(sensor)!: start & stop is required, and allow stop < start in ti…
Browse files Browse the repository at this point in the history
…me-filter (#737)

* feat(sensor): time-filter support time ranges span midnight #737

* style: parse time first, then if-then logics

* doc: comprehensive time filter docs including stop < start behavior

* style: parse time first, then if-then logics (again)

* doc: add start > stop time filter example

* feat(sensor)!: make start/stop of time filter required
  • Loading branch information
tmshn authored Jul 19, 2020
1 parent 073995c commit 6ebca1a
Show file tree
Hide file tree
Showing 15 changed files with 234 additions and 151 deletions.
10 changes: 7 additions & 3 deletions api/openapi-spec/swagger.json
Original file line number Diff line number Diff line change
Expand Up @@ -2548,15 +2548,19 @@
}
},
"io.argoproj.sensor.v1alpha1.TimeFilter": {
"description": "TimeFilter describes a window in time. DataFilters out event events that occur outside the time limits. In other words, only events that occur after Start and before Stop will pass this filter.",
"description": "TimeFilter describes a window in time. It filters out events that occur outside the time limits. In other words, only events that occur after Start and before Stop will pass this filter.",
"type": "object",
"required": [
"start",
"stop"
],
"properties": {
"start": {
"description": "Start is the beginning of a time window. Before this time, events for this event are ignored and format is hh:mm:ss",
"description": "Start is the beginning of a time window in UTC. Before this time, events for this dependency are ignored. Format is hh:mm:ss.",
"type": "string"
},
"stop": {
"description": "StopPattern is the end of a time window. After this time, events for this event are ignored and format is hh:mm:ss",
"description": "Stop is the end of a time window in UTC. After or equal to this time, events for this dependency are ignored and Format is hh:mm:ss. If it is smaller than Start, it is treated as next day of Start (e.g.: 22:00:00-01:00:00 means 22:00:00-25:00:00).",
"type": "string"
}
}
Expand Down
16 changes: 9 additions & 7 deletions api/sensor.html
Original file line number Diff line number Diff line change
Expand Up @@ -2388,7 +2388,7 @@ <h3 id="argoproj.io/v1alpha1.TimeFilter">TimeFilter
</p>
<p>
<p>TimeFilter describes a window in time.
DataFilters out event events that occur outside the time limits.
It filters out events that occur outside the time limits.
In other words, only events that occur after Start and before Stop
will pass this filter.</p>
</p>
Expand All @@ -2408,9 +2408,9 @@ <h3 id="argoproj.io/v1alpha1.TimeFilter">TimeFilter
</em>
</td>
<td>
<p>Start is the beginning of a time window.
Before this time, events for this event are ignored and
format is hh:mm:ss</p>
<p>Start is the beginning of a time window in UTC.
Before this time, events for this dependency are ignored.
Format is hh:mm:ss.</p>
</td>
</tr>
<tr>
Expand All @@ -2421,9 +2421,11 @@ <h3 id="argoproj.io/v1alpha1.TimeFilter">TimeFilter
</em>
</td>
<td>
<p>StopPattern is the end of a time window.
After this time, events for this event are ignored and
format is hh:mm:ss</p>
<p>Stop is the end of a time window in UTC.
After or equal to this time, events for this dependency are ignored and
Format is hh:mm:ss.
If it is smaller than Start, it is treated as next day of Start
(e.g.: 22:00:00-01:00:00 means 22:00:00-25:00:00).</p>
</td>
</tr>
</tbody>
Expand Down
16 changes: 9 additions & 7 deletions api/sensor.md
Original file line number Diff line number Diff line change
Expand Up @@ -4671,9 +4671,9 @@ TimeFilter

<p>

TimeFilter describes a window in time. DataFilters out event events that
occur outside the time limits. In other words, only events that occur
after Start and before Stop will pass this filter.
TimeFilter describes a window in time. It filters out events that occur
outside the time limits. In other words, only events that occur after
Start and before Stop will pass this filter.

</p>

Expand Down Expand Up @@ -4715,8 +4715,8 @@ Description

<p>

Start is the beginning of a time window. Before this time, events for
this event are ignored and format is hh:mm:ss
Start is the beginning of a time window in UTC. Before this time, events
for this dependency are ignored. Format is hh:mm:ss.

</p>

Expand All @@ -4736,8 +4736,10 @@ this event are ignored and format is hh:mm:ss

<p>

StopPattern is the end of a time window. After this time, events for
this event are ignored and format is hh:mm:ss
Stop is the end of a time window in UTC. After or equal to this time,
events for this dependency are ignored and Format is hh:mm:ss. If it is
smaller than Start, it is treated as next day of Start (e.g.:
22:00:00-01:00:00 means 22:00:00-25:00:00).

</p>

Expand Down
4 changes: 0 additions & 4 deletions common/common.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,6 @@ import (

// Defaults
const (
// StandardTimeFormat is time format reference for golang
StandardTimeFormat = "2006-01-02 15:04:05"
// StandardYYYYMMDDFormat formats date in yyyy-mm-dd format
StandardYYYYMMDDFormat = "2006-01-02"
// DefaultControllerNamespace is the default namespace where the sensor and gateways controllers are installed
DefaultControllerNamespace = "argo-events"
)
Expand Down
28 changes: 28 additions & 0 deletions common/time.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
/*
Copyright 2020 BlackRock, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package common

import (
"fmt"
"time"
)

// ParseTime parses time string in "HH:MM:SS" format into time.Time, which date is same as baseDate in UTC.
func ParseTime(t string, baseDate time.Time) (time.Time, error) {
date := baseDate.UTC().Format("2006-01-02")
return time.Parse("2006-01-02 15:04:05", fmt.Sprintf("%s %s", date, t))
}
36 changes: 36 additions & 0 deletions common/time_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
/*
Copyright 2018 BlackRock, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package common

import (
"testing"
"time"

"github.com/stretchr/testify/assert"
)

func TestParseTime(t *testing.T) {
timeStr := "12:34:56"
base := time.Date(2020, 7, 8, 1, 2, 03, 123456789, time.FixedZone("UTC+9", 9*60*60))

parsed, err := ParseTime(timeStr, base)
if assert.NoError(t, err) {
assert.Equal(t, timeStr, parsed.Format("15:04:05"))
assert.Zero(t, parsed.Nanosecond())
assert.Equal(t, base.UTC().Truncate(24*time.Hour), parsed.Truncate(24*time.Hour))
}
}
37 changes: 12 additions & 25 deletions controllers/sensor/validate.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ limitations under the License.
package sensor

import (
"fmt"
"net/http"
"strings"
"time"
Expand Down Expand Up @@ -469,31 +468,19 @@ func validateEventFilter(filter *v1alpha1.EventDependencyFilter) error {

// validateEventTimeFilter validates time filter
func validateEventTimeFilter(tFilter *v1alpha1.TimeFilter) error {
currentT := time.Now().UTC()
currentT = time.Date(currentT.Year(), currentT.Month(), currentT.Day(), 0, 0, 0, 0, time.UTC)
currentTStr := currentT.Format(common.StandardYYYYMMDDFormat)
if tFilter.Start != "" && tFilter.Stop != "" {
startTime, err := time.Parse(common.StandardTimeFormat, fmt.Sprintf("%s %s", currentTStr, tFilter.Start))
if err != nil {
return err
}
stopTime, err := time.Parse(common.StandardTimeFormat, fmt.Sprintf("%s %s", currentTStr, tFilter.Stop))
if err != nil {
return err
}
if stopTime.Before(startTime) || startTime.Equal(stopTime) {
return errors.Errorf("invalid event time filter: stop '%s' is before or equal to start '%s", tFilter.Stop, tFilter.Start)
}
now := time.Now().UTC()
// Parse start and stop
startTime, err := common.ParseTime(tFilter.Start, now)
if err != nil {
return err
}
if tFilter.Stop != "" {
stopTime, err := time.Parse(common.StandardTimeFormat, fmt.Sprintf("%s %s", currentTStr, tFilter.Stop))
if err != nil {
return err
}
stopTime = stopTime.UTC()
if stopTime.Before(currentT.UTC()) {
return errors.Errorf("invalid event time filter: stop '%s' is before the current time '%s'", tFilter.Stop, currentT)
}
stopTime, err := common.ParseTime(tFilter.Stop, now)
if err != nil {
return err
}

if stopTime.Equal(startTime) {
return errors.Errorf("invalid event time filter: stop '%s' is equal to start '%s", tFilter.Stop, tFilter.Start)
}
return nil
}
Expand Down
29 changes: 25 additions & 4 deletions docs/tutorials/07-filters.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,28 @@ Change the subscriber in the webhook gateway to point it to `context-filter` sen
either `custom-webhook` as the value of the `source`.

## Time Filter
Time filter is specially helpful when you need to make sure an event occurs between a
certain time-frame. Time filter takes a `start` and `stop` time but you can also define just the
`start` time, meaning, there is no `end time` constraint or just the `stop` time, meaning, there is
no `start time` constraint. An example of time filter is available under `examples/sensors`.

You can also use time filter, which is applied on event time.
It filters out events that occur outside the specified time range, so it is specially helpful when
you need to make sure an event occurs between a certain time-frame.

Time filter takes a `start` and `stop` time in `HH:MM:SS` format in UTC. If `stop` is smaller than `start`,
the stop time is treated as next day of `start`. Note that `start` is inclusive while `stop` is exclusive.
The diagrams below illustlate these behavior.

An example of time filter is available under `examples/sensors`.

1. if `start` < `stop`: event time must be in `[start, stop)`

00:00:00 00:00:00 00:00:00
┃ start stop ┃ start stop ┃
─┸─────●───────────────────────○─────┸─────●───────────────────────○─────┸─
╰───────── OK ──────────╯ ╰───────── OK ──────────╯

2. if `stop` < `start`: event time must be in `[start, stop@Next day)`
(this is equivalent to: event time must be in `[00:00:00, stop) || [start, 00:00:00@Next day)`)

00:00:00 00:00:00 00:00:00
┃ stop start ┃ stop start ┃
─┸───────────○───────────●───────────┸───────────○───────────●───────────┸─
─── OK ──────╯ ╰───────── OK ──────────╯ ╰────── OK ───
9 changes: 9 additions & 0 deletions examples/sensors/time-filter-webhook.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,15 @@ spec:
time:
start: "22:14:05"
stop: "23:04:05"
- name: test-another-dep
gatewayName: webhook
eventName: example
filters:
time:
# start > stop, stop is treated as next day of start,
# so in this case, events can pass only during 50 minutes from 23:34:05
start: "23:34:05"
stop: "00:24:05" # == "24:24:05"
triggers:
- template:
name: hello-world-workflow-trigger
Expand Down
13 changes: 1 addition & 12 deletions manifests/install.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -265,18 +265,7 @@ subjects:
---
apiVersion: v1
data:
_example: "#\n# This is not real configuration but an example.\n# Copy this block
out of \"_example\" and unindent it \n# under \"data\" to make it functional.\n#\nconfig:
|\n # instanceID is a label selector to limit the controller's watch to a specific
instance. It\n # contains an arbitrary value that is carried forward into its
pod labels, under the key\n # gateways.argoproj.io/gateway-controller-instanceid,
for the purposes of date segregation.\n # This enables a controller to only watch
the gateway that it is interested about,\n # in order to support multiple controllers
in a single cluster, and ultimately allows the\n # controller itself to be bundled
as part of a higher level application. If omitted, the\n # controller watches
gateways that *are not* labeled with an instance id.\n instanceID: <instanceID>\n\n
\ # namespace is used for namespaced installation, if it is specified, the controller
only \n # watches gateways in this namespace.\n namespace: <your_namespace>\n"
_example: "#\n# This is not real configuration but an example.\n# Copy this block out of \"_example\" and unindent it \n# under \"data\" to make it functional.\n#\nconfig: |\n # instanceID is a label selector to limit the controller's watch to a specific instance. It\n # contains an arbitrary value that is carried forward into its pod labels, under the key\n # gateways.argoproj.io/gateway-controller-instanceid, for the purposes of date segregation.\n # This enables a controller to only watch the gateway that it is interested about,\n # in order to support multiple controllers in a single cluster, and ultimately allows the\n # controller itself to be bundled as part of a higher level application. If omitted, the\n # controller watches gateways that *are not* labeled with an instance id.\n instanceID: <instanceID>\n\n # namespace is used for namespaced installation, if it is specified, the controller only \n # watches gateways in this namespace.\n namespace: <your_namespace>\n"
kind: ConfigMap
metadata:
name: gateway-controller-configmap
Expand Down
16 changes: 9 additions & 7 deletions pkg/apis/sensor/v1alpha1/generated.proto

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 4 additions & 3 deletions pkg/apis/sensor/v1alpha1/openapi_generated.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 11 additions & 9 deletions pkg/apis/sensor/v1alpha1/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -211,18 +211,20 @@ type EventDependencyFilter struct {
}

// TimeFilter describes a window in time.
// DataFilters out event events that occur outside the time limits.
// It filters out events that occur outside the time limits.
// In other words, only events that occur after Start and before Stop
// will pass this filter.
type TimeFilter struct {
// Start is the beginning of a time window.
// Before this time, events for this event are ignored and
// format is hh:mm:ss
Start string `json:"start,omitempty" protobuf:"bytes,1,opt,name=start"`
// StopPattern is the end of a time window.
// After this time, events for this event are ignored and
// format is hh:mm:ss
Stop string `json:"stop,omitempty" protobuf:"bytes,2,opt,name=stop"`
// Start is the beginning of a time window in UTC.
// Before this time, events for this dependency are ignored.
// Format is hh:mm:ss.
Start string `json:"start" protobuf:"bytes,1,opt,name=start"`
// Stop is the end of a time window in UTC.
// After or equal to this time, events for this dependency are ignored and
// Format is hh:mm:ss.
// If it is smaller than Start, it is treated as next day of Start
// (e.g.: 22:00:00-01:00:00 means 22:00:00-25:00:00).
Stop string `json:"stop" protobuf:"bytes,2,opt,name=stop"`
}

// JSONType contains the supported JSON types for data filtering
Expand Down
Loading

0 comments on commit 6ebca1a

Please sign in to comment.