Skip to content

Commit

Permalink
Squashed commit of the following:
Browse files Browse the repository at this point in the history 
commit 2565ce8
Merge: 0ee043f 2f568ee
Author: Chris Chinchilla <chris@chronosphere.io>
Date:   Thu Sep 17 10:29:16 2020 +0200

    Merge branch 'master' into braskin/hugo_site

commit 2f568ee
Author: Matt Schallert <mattschallert@gmail.com>
Date:   Wed Sep 16 15:00:11 2020 -0700

    [coord] Configurable blackholed SP's back from agg (#2641)

    There are cases where we want to ignore metrics for a given storage
    policy at the coordinator as they come back from the aggregator over
    m3msg. Specifically, the coordinators may still receive aggregated
    metrics with storage policies that they no longer have namespace
    configuration for. This PR allows dropping those metrics as they come
    back from the aggregator based on their storage policy.

commit 273b2e3
Author: Alex Bublichenko <46664526+abliqo@users.noreply.github.com>
Date:   Wed Sep 16 17:10:18 2020 -0400

    [dbnode] Emit metric with dbnode health status (#2588)

    Emit metric with dbnode health status

    Problem:

    In a large m3db cluster, when a database node becomes non-functional (service fails to start or host is down), it may go unnoticed. If it goes unnoticed long enough, and one more node that owns the same shard(s) becomes non-functional, a quorum may be lost and block writes to the database.

    Solution:

    The connection pool in `src/dbnode/client/connection_pool.go` already does periodic health check from the client's node/process. Let that code emit a gauge metric with the result of the healthcheck. The metrics scope passed to `newConnectionPool` is already tagged with `hostID`.

    Since the healthcheck is done from the client, it implies that node is in M3DB placement and expected to be functional. Thus, alerting can be set up based on this metric alone.

    This behavior is optional, and disabled by default, to prevent accidental explosion of metric cardinality. When enabled, the callsites must ensure that the tags they set on the scope passed to m3db node client will not cause high cardinality of combinations with `hostID` tag.

    Considered Alternatives:

    1. Emit a heartbeat metric from `src/dbnode/network/server/tchannelthrift/node/service.go`. Alerting on lost heartbeat requires knowledge about whether the node is in placement, i.e. expected to be functional.

    2. Let independent monitoring/canary system actively probe healthcheck endpoint of every database node, determine whether the node is expected to be functional by comparing to M3DB placement data, and alert operator. Such solution would be ideal but has much higher cost.

commit 0ee043f
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Wed Sep 16 18:11:50 2020 +0200

    Add Netlify TOML file

    Signed-off-by: ChrisChinchilla <chris@chronosphere.io>

commit a91c5ec
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Wed Sep 16 18:00:16 2020 +0200

    Final

    Signed-off-by: ChrisChinchilla <chris@chronosphere.io>

commit 356abb8
Merge: febe502 47bd03b
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Wed Sep 16 17:55:50 2020 +0200

    Merge branch 'master' into braskin/hugo_site

    Signed-off-by: ChrisChinchilla <chris@chronosphere.io>

    # Conflicts:
    #	docs/content/coordinator/_index.md
    #	docs/content/coordinator/api/remote.md
    #	docs/content/how_to/query.md
    #	docs/content/m3query/api/query.md
    #	docs/content/m3query/architecture/_index.md
    #	docs/content/m3query/architecture/blocks.md
    #	docs/content/m3query/architecture/fanout.md
    #	docs/content/m3query/config/annotated_config.md
    #	docs/content/m3query/config/annotated_config.yaml
    #	docs/content/m3query/roadmap.md
    #	docs/content/operational_guide/bootstrapping_crash_recovery.md
    #	docs/content/operational_guide/monitoring.md
    #	docs/content/operational_guide/repairs.md
    #	docs/content/operational_guide/upgrading_m3.md
    #	docs/content/overview/components.md
    #	docs/coordinator/api/remote.md
    #	docs/includes/headers_optional_read_all.md
    #	docs/includes/headers_optional_read_limits.md
    #	docs/includes/headers_optional_write_all.md
    #	docs/index.md
    #	docs/m3coordinator/api/remote.md
    #	docs/m3db/architecture/storage.md
    #	docs/m3query/api/index.md
    #	docs/m3query/architecture/blocks.md
    #	docs/m3query/architecture/fanout.md
    #	docs/m3query/architecture/functions.md
    #	docs/m3query/config/annotated_config.md
    #	docs/m3query/config/annotated_config.yaml
    #	docs/m3query/roadmap.md
    #	docs/operational_guide/index.md
    #	docs/operational_guide/mapping_rollup.md
    #	docs/overview/media.md
    #	docs/query_engine/api/index.md
    #	docs/query_engine/architecture/blocks.md
    #	docs/query_engine/architecture/fanout.md
    #	docs/query_engine/config/annotated_config.md
    #	docs/query_engine/config/annotated_config.yaml
    #	docs/query_engine/roadmap.md
    #	docs/static/aggregator.md

commit febe502
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Wed Sep 16 15:34:17 2020 +0200

    Fix paths

    Signed-off-by: ChrisChinchilla <chris@chronosphere.io>

commit 8c300d5
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Wed Sep 16 14:55:57 2020 +0200

    Link fixes

    Signed-off-by: ChrisChinchilla <chris@chronosphere.io>

commit a4ca3db
Merge: 3325ed8 45ad2f6
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Wed Sep 16 14:17:18 2020 +0200

    Merge commit '45ad2f6d6f1c23e387b8232b678e8c0d1093a5ca' into braskin/hugo_site

commit 45ad2f6
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Wed Sep 16 14:17:18 2020 +0200

    Squashed 'docs/themes/docs-theme/' changes from cc1e7c4b7..5103e6cc3

    5103e6cc3 Remove centering

    git-subtree-dir: docs/themes/docs-theme
    git-subtree-split: 5103e6cc3427e97c469c2914246d4b46f7e27bd9

commit 3325ed8
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Wed Sep 16 14:17:06 2020 +0200

    Fix links

    Signed-off-by: ChrisChinchilla <chris@chronosphere.io>

commit 47bd03b
Author: Matt Schallert <mattschallert@gmail.com>
Date:   Tue Sep 15 18:47:31 2020 -0700

    [docs] Add recent talks (#2639)

commit 705f4f0
Author: Ryan Allen <rallen090@gmail.com>
Date:   Tue Sep 15 20:54:45 2020 -0400

    [dbnode] query limits - missed commit with feedback (#2640)

commit b71785e
Author: Ryan Allen <rallen090@gmail.com>
Date:   Tue Sep 15 15:19:25 2020 -0400

    [dbnode] additional query limit on bytes read (#2627)

commit 3aa2f51
Author: Linas Medžiūnas <linasm@users.noreply.github.com>
Date:   Tue Sep 15 21:13:41 2020 +0300

    [large-tiles] Cross block reader changes plus Cross block iterator (#2612)

    * [dbnode] Add OrderedByIndex option for DataFileSetReader.Open (#2465)

    * [dbnode] Cross-block series reader (#2481)

    * [dbnode] AggregateTiles RPC - minimal E2E flow (#2466)

    * [large-tiles] Cross block reader changes plus Cross block iterator

    * Fix TestReadAggregateWrite

    * Fix TestShardAggregateTiles

    * Dummy

    * Fix TestReadAggregateWrite

    * typo

    * Convert spaces to tabs in rpc.thrift

    * Change capitalization of source/target namespace

    * Fix copyright year in large_tiles_test.go

    * Remove a noop

    * PR feedback

    * PR 2617 feedback

    * PR feedback

    * More PR feedback

    * Fix unit test

    * Reuse read objects during aggregation

    * Renamed StreamingMode to StreamingEnabled

    * PR feedback

    * Indentation

    * Address PR feedback

    * Skip flaky TestReadAggregateWrite

    Co-authored-by: arnikola <artem@chronosphere.io>

commit 8bcf5ac
Author: Rob Skillington <rob.skillington@gmail.com>
Date:   Tue Sep 15 12:35:09 2020 -0400

    [changelog] Update changelog for 0.15.14 release (#2638)

commit 4b0fcb3
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Tue Sep 15 16:40:53 2020 +0200

    Squashed 'docs/themes/docs-theme/' content from commit cc1e7c4b7

    git-subtree-dir: docs/themes/docs-theme
    git-subtree-split: cc1e7c4b7be99f7dd8ceaacf11e6094f297c5989

commit 0d8b1fa
Merge: 99b541e 4b0fcb3
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Tue Sep 15 16:40:53 2020 +0200

    Merge commit '4b0fcb301a028aa8cecd46c100d8cde119737ccc' as 'docs/themes/docs-theme'

commit 99b541e
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Tue Sep 15 16:40:42 2020 +0200

    Finalise glossary section

    Signed-off-by: ChrisChinchilla <chris@chronosphere.io>

commit 90e38bc
Merge: 6d88723 2b5490b
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Tue Sep 15 13:44:15 2020 +0200

    Merge commit '2b5490b211202f7959328b4f7e9b387ef5e4b6ab' into braskin/hugo_site

commit 2b5490b
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Tue Sep 15 13:44:15 2020 +0200

    Squashed 'docs/themes/docs-theme/' changes from 06ca4188b..dcbf4649b

    dcbf4649b Change glossary path
    REVERT: 06ca4188b Change glossary path

    git-subtree-dir: docs/themes/docs-theme
    git-subtree-split: dcbf4649bbaa43fc21c12cbdb936999d4b6719d8

commit 6d88723
Merge: 957fb39 ce857e0
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Tue Sep 15 12:28:23 2020 +0200

    Merge commit 'ce857e005246025a90ec4ddd933cee7cfab4488a' into braskin/hugo_site

commit ce857e0
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Tue Sep 15 12:28:23 2020 +0200

    Squashed 'docs/themes/docs-theme/' changes from 7ac79e398..06ca4188b

    06ca4188b Change glossary path
    REVERT: 7ac79e398 Change glossary path

    git-subtree-dir: docs/themes/docs-theme
    git-subtree-split: 06ca4188b1b040ce0f6334c50aeaecf8352c789f

commit 957fb39
Merge: 571aed1 0ee3f03
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Tue Sep 15 12:23:18 2020 +0200

    Merge commit '0ee3f033d96a0af227c42edb71274a5e081a6e04' into braskin/hugo_site

commit 0ee3f03
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Tue Sep 15 12:23:18 2020 +0200

    Squashed 'docs/themes/docs-theme/' changes from d84a9b2e3..7ac79e398

    7ac79e398 Change glossary path
    REVERT: d84a9b2e3 Change glossary path

    git-subtree-dir: docs/themes/docs-theme
    git-subtree-split: 7ac79e398305a2dbdbd88b3cd36e3b4c31e7483a

commit 571aed1
Merge: 8ed0032 fa9fe8b
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Tue Sep 15 12:13:12 2020 +0200

    Merge commit 'fa9fe8b2f4055b89cd23bd9f0a8b50fb70918fe3' into braskin/hugo_site

commit fa9fe8b
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Tue Sep 15 12:13:12 2020 +0200

    Squashed 'docs/themes/docs-theme/' changes from 4dba9dd2e..d84a9b2e3

    d84a9b2e3 Change glossary path

    git-subtree-dir: docs/themes/docs-theme
    git-subtree-split: d84a9b2e3d94bc46382e79bb7b14775f5f9d29ff

commit 8ed0032
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Tue Sep 15 12:13:03 2020 +0200

    Remove defunct files

    Signed-off-by: ChrisChinchilla <chris@chronosphere.io>

commit 2ae9792
Author: Rob Skillington <rob.skillington@gmail.com>
Date:   Tue Sep 15 00:39:21 2020 -0400

    [dbnode] Add ability to configure writes to initializing shards (#2615)

commit f83f576
Author: Rob Skillington <rob.skillington@gmail.com>
Date:   Mon Sep 14 22:19:15 2020 -0400

    [changelog] Update out of date changelog for 0.15.13 (#2636)

commit 876cee8
Author: Linas Medžiūnas <linasm@users.noreply.github.com>
Date:   Mon Sep 14 22:11:29 2020 +0300

    [large-tiles] Large tiles aggregation flow (#2617)

commit d38ff54
Author: Matt Schallert <mattschallert@gmail.com>
Date:   Mon Sep 14 07:22:09 2020 -0700

    [m3admin] Allow /set for new placements (#2625)

    Previously `/set` required that a placement already exists in order to
    set it. This hinders operations such as preemptively creating a
    placement or replacing one that had been deleted via safe APIs.

    This PR allows using `/set` even if a placement didn't previously exist
    at that key.

commit d5fff76
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Mon Sep 14 15:33:05 2020 +0200

    Squashed 'docs/themes/docs-theme/' content from commit 4dba9dd2e

    git-subtree-dir: docs/themes/docs-theme
    git-subtree-split: 4dba9dd2e0320cf0b9df05be01808a567781d5e3

commit 74ae18a
Merge: d707793 d5fff76
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Mon Sep 14 15:33:05 2020 +0200

    Merge commit 'd5fff76fac215da9f96e751e916bb82cca6dc785' as 'docs/themes/docs-theme'

commit d707793
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Mon Sep 14 15:32:55 2020 +0200

    Move

    Signed-off-by: ChrisChinchilla <chris@chronosphere.io>

commit 93f3cd1
Merge: 42dbf19 8b276ed
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Mon Sep 14 12:57:19 2020 +0200

    Merge commit '8b276ed170a886058c276535a127567a38210426' as 'docs/themes/docs-theme'

commit 8b276ed
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Mon Sep 14 12:57:19 2020 +0200

    Squashed 'docs/themes/docs-theme/' content from commit 4bb3340df

    git-subtree-dir: docs/themes/docs-theme
    git-subtree-split: 4bb3340df8857eb8cd07b70515b546287617ca60

commit 42dbf19
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Mon Sep 14 12:57:15 2020 +0200

    Continue

commit 07c034d
Author: teddywahle <69990143+teddywahle@users.noreply.github.com>
Date:   Mon Sep 14 01:03:57 2020 -0700

    [query] Implemented movingSum, movingMax, movingMin (graphite functions) (#2570)

commit e66e016
Author: teddywahle <69990143+teddywahle@users.noreply.github.com>
Date:   Mon Sep 14 00:48:27 2020 -0700

    [query] Add Graphite aliasByTags function (#2626)

commit d3cd566
Author: teddywahle <69990143+teddywahle@users.noreply.github.com>
Date:   Mon Sep 14 00:46:44 2020 -0700

    [query] Implemented the Graphite `exponentialMovingAverage` function (#2622)

commit 47eaa98
Author: arnikola <artem@chronosphere.io>
Date:   Thu Sep 10 18:18:01 2020 -0400

    [query] Expose Graphite Compile method. (#2624)

commit 8223666
Author: nate <nbroyles@gmail.com>
Date:   Thu Sep 10 11:00:49 2020 -0400

    [dbnode] Fail if FetchTagged partially retrieves results due to error (#2610)

commit d103bae
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Wed Sep 9 12:36:58 2020 +0200

    Remove netlify build from branch

commit 7759733
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Wed Sep 9 12:35:32 2020 +0200

    Attempt to fix Netlify build

commit 7440118
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Tue Sep 8 17:05:41 2020 +0200

    More

commit 76372de
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Fri Sep 4 16:43:26 2020 +0200

    Struct

commit e74c381
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Fri Aug 28 12:42:48 2020 +0200

    Further fixing

commit 3e357ac
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Fri Aug 28 12:10:14 2020 +0200

    Continue

commit 6b50f87
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Fri Aug 28 11:44:52 2020 +0200

    Continue

commit efddf37
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Fri Aug 28 11:37:08 2020 +0200

    Start migration to Hugo

commit 1bbca4b
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Mon Aug 17 16:29:09 2020 +0200

    Start quickstart rewrite

commit 97279a4
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Thu Aug 13 16:33:02 2020 +0200

    Add hugo publish dir to gitignore

commit fdd275d
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Thu Aug 13 16:25:58 2020 +0200

    Update publish path

commit e942b4c
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Thu Aug 13 16:22:20 2020 +0200

    Miss-formed YAML broke build

commit 62b2fac
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Thu Aug 13 16:20:25 2020 +0200

    Add netlify config for this preview branch

commit 3232961
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Thu Aug 13 16:06:43 2020 +0200

    Remove rogue submodule

commit d884269
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Thu Aug 13 15:56:55 2020 +0200

    Change submodule path

commit 3e42ed8
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Thu Aug 13 15:53:59 2020 +0200

    Add theme as submodule

commit ce92f32
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Thu Aug 13 14:37:49 2020 +0200

    Testing content

commit 40d6f9f
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Thu Aug 13 14:37:33 2020 +0200

    Update config

commit accedc1
Author: ChrisChinchilla <chris@chronosphere.io>
Date:   Thu Aug 13 14:37:25 2020 +0200

    remove folder

commit 6ed118b
Author: Gibbs Cullen <gibbs@chronosphere.io>
Date:   Fri May 8 12:56:50 2020 -0400

    edits to operator section

commit 0167857
Author: Gibbs Cullen <gibbs@chronosphere.io>
Date:   Fri May 8 11:01:58 2020 -0400

    initial formatting done

commit 21dc3af
Author: Gibbs Cullen <gibbs@chronosphere.io>
Date:   Mon Apr 27 12:55:20 2020 -0400

    content trasferred to hugo site

commit ff7d60c
Author: Gibbs Cullen <gibbs@chronosphere.io>
Date:   Tue Apr 21 21:07:00 2020 -0400

    updated outline

commit 642464c
Author: Gibbs Cullen <gibbs@chronosphere.io>
Date:   Thu Apr 2 12:50:30 2020 -0400

    updated docs format in hugo

commit c431580
Author: Gibbs Cullen <gibbs@chronosphere.io>
Date:   Mon Mar 30 18:51:17 2020 -0400

    hugo updates

commit 0249640
Author: Gibbs Cullen <gibbs@chronosphere.io>
Date:   Fri Mar 27 19:28:34 2020 -0400

    new format

commit bb39315
Author: Benjamin Raskin <benraskin92@gmail.com>
Date:   Mon Mar 2 15:22:36 2020 -0500

    [WIP][docs] Add hugo theme

Signed-off-by: ChrisChinchilla <chris@chronosphere.io>
  • Loading branch information
ChrisChinchilla committed Sep 18, 2020
1 parent adf89b8 commit 7d00195
Show file tree
Hide file tree
Showing 9 changed files with 242 additions and 15 deletions.
2 changes: 1 addition & 1 deletion .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -61,4 +61,4 @@ m3db.io/openapi
.vagrant

#docs ignores
docs/public
docs/public
4 changes: 2 additions & 2 deletions netlify.toml
View file
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
[build]
# Directory to change to before starting a build.
# This is where we will look for package.json/.nvmrc/etc.
base = "docs-beta/"
base = "docs/"

# Directory that contains the deploy-ready HTML files and assets generated by
# the build. This is relative to the base directory if one has been set, or the
# root directory if a base has not been set. This sample publishes the
# directory located at the absolute path "root/project/build-output"
publish = "docs-beta/public"
publish = "docs/public"

[context."chrischinch/quickstart"]
command = "hugo"
Expand Down
3 changes: 3 additions & 0 deletions src/cmd/services/m3coordinator/server/m3msg/config.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
package m3msg

import (
"github.com/m3db/m3/src/metrics/policy"
"github.com/m3db/m3/src/msg/consumer"
"github.com/m3db/m3/src/x/instrument"
xio "github.com/m3db/m3/src/x/io"
Expand Down Expand Up @@ -67,6 +68,7 @@ func (c Configuration) NewServer(
type handlerConfiguration struct {
// ProtobufDecoderPool configs the protobuf decoder pool.
ProtobufDecoderPool pool.ObjectPoolConfiguration `yaml:"protobufDecoderPool"`
BlackholePolicies []policy.StoragePolicy `yaml:"blackholePolicies"`
}

func (c handlerConfiguration) newHandler(
Expand All @@ -82,6 +84,7 @@ func (c handlerConfiguration) newHandler(
}),
),
ProtobufDecoderPoolOptions: c.ProtobufDecoderPool.NewObjectPoolOptions(iOpts),
BlockholePolicies: c.BlackholePolicies,
})
return consumer.NewMessageHandler(p, cOpts), nil
}
Expand Down
47 changes: 40 additions & 7 deletions src/cmd/services/m3coordinator/server/m3msg/protobuf_handler.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import (
"sync"

"github.com/m3db/m3/src/metrics/encoding/protobuf"
"github.com/m3db/m3/src/metrics/policy"
"github.com/m3db/m3/src/msg/consumer"
"github.com/m3db/m3/src/x/instrument"
"github.com/m3db/m3/src/x/pool"
Expand All @@ -38,11 +39,13 @@ type Options struct {
InstrumentOptions instrument.Options
WriteFn WriteFn
ProtobufDecoderPoolOptions pool.ObjectPoolOptions
BlockholePolicies []policy.StoragePolicy
}

type handlerMetrics struct {
messageReadError tally.Counter
metricAccepted tally.Counter
droppedMetricBlackholePolicy tally.Counter
droppedMetricDecodeError tally.Counter
droppedMetricDecodeMalformed tally.Counter
}
Expand All @@ -55,6 +58,9 @@ func newHandlerMetrics(scope tally.Scope) handlerMetrics {
droppedMetricDecodeError: messageScope.Tagged(map[string]string{
"reason": "decode-error",
}).Counter("dropped"),
droppedMetricBlackholePolicy: messageScope.Tagged(map[string]string{
"reason": "blackhole-policy",
}).Counter("dropped"),
droppedMetricDecodeMalformed: messageScope.Tagged(map[string]string{
"reason": "decode-malformed",
}).Counter("dropped"),
Expand All @@ -68,19 +74,34 @@ type pbHandler struct {
wg *sync.WaitGroup
logger *zap.Logger
m handlerMetrics

// Set of policies for which when we see a metric we drop it on the floor.
blackholePolicies []policy.StoragePolicy
}

func newProtobufProcessor(opts Options) consumer.MessageProcessor {
p := protobuf.NewAggregatedDecoderPool(opts.ProtobufDecoderPoolOptions)
p.Init()
return &pbHandler{
ctx: context.Background(),
writeFn: opts.WriteFn,
pool: p,
wg: &sync.WaitGroup{},
logger: opts.InstrumentOptions.Logger(),
m: newHandlerMetrics(opts.InstrumentOptions.MetricsScope()),

h := &pbHandler{
ctx: context.Background(),
writeFn: opts.WriteFn,
pool: p,
wg: &sync.WaitGroup{},
logger: opts.InstrumentOptions.Logger(),
m: newHandlerMetrics(opts.InstrumentOptions.MetricsScope()),
blackholePolicies: opts.BlockholePolicies,
}

if len(opts.BlockholePolicies) > 0 {
policyNames := make([]string, 0, len(opts.BlockholePolicies))
for _, sp := range h.blackholePolicies {
policyNames = append(policyNames, sp.String())
}
h.logger.Info("m3msg handler blackholing metrics for configured policies", zap.Strings("policyNames", policyNames))
}

return h
}

func (h *pbHandler) Process(msg consumer.Message) {
Expand All @@ -96,10 +117,22 @@ func (h *pbHandler) Process(msg consumer.Message) {
h.m.droppedMetricDecodeMalformed.Inc(1)
return
}

h.m.metricAccepted.Inc(1)

h.wg.Add(1)
r := NewProtobufCallback(msg, dec, h.wg)

// If storage policy is blackholed, ack the message immediately and don't
// bother passing down the write path.
for _, blackholeSp := range h.blackholePolicies {
if sp.Equivalent(blackholeSp) {
h.m.droppedMetricBlackholePolicy.Inc(1)
r.Callback(OnSuccess)
return
}
}

h.writeFn(h.ctx, dec.ID(), dec.TimeNanos(), dec.EncodeNanos(), dec.Value(), sp, r)
}

Expand Down
104 changes: 100 additions & 4 deletions src/cmd/services/m3coordinator/server/m3msg/protobuf_handler_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import (
"net"
"sync"
"testing"
"time"

"github.com/m3db/m3/src/metrics/encoding/protobuf"
"github.com/m3db/m3/src/metrics/metric"
Expand All @@ -36,13 +37,19 @@ import (
"github.com/m3db/m3/src/msg/protocol/proto"
"github.com/m3db/m3/src/x/instrument"
"github.com/m3db/m3/src/x/server"
xtime "github.com/m3db/m3/src/x/time"

"github.com/stretchr/testify/require"
)

var (
testID = "stats.sjc1.gauges.m3+some-name+dc=sjc1,env=production,service=foo,type=gauge"
validStoragePolicy = policy.MustParseStoragePolicy("1m:40d")
testID = "stats.foo1.gauges.m3+some-name+dc=foo1,env=production,service=foo,type=gauge"

// baseStoragePolicy represents what we typically define in config for SP.
// precisionStoragePolicy is the same retention/resolution, but includes the
// precision (which is often included with incoming writes).
baseStoragePolicy = policy.MustParseStoragePolicy("1m:40d")
precisionStoragePolicy = policy.NewStoragePolicy(time.Minute, xtime.Second, 40*24*time.Hour)
)

func TestM3MsgServerWithProtobufHandler(t *testing.T) {
Expand Down Expand Up @@ -74,7 +81,7 @@ func TestM3MsgServerWithProtobufHandler(t *testing.T) {
Value: 1,
Type: metric.GaugeType,
},
StoragePolicy: validStoragePolicy,
StoragePolicy: precisionStoragePolicy,
}

encoder := protobuf.NewAggregatedEncoder(nil)
Expand All @@ -98,7 +105,7 @@ func TestM3MsgServerWithProtobufHandler(t *testing.T) {
Value: 0,
Type: metric.UnknownType,
},
StoragePolicy: validStoragePolicy,
StoragePolicy: precisionStoragePolicy,
}
require.NoError(t, encoder.Encode(m2, 3000))
enc = proto.NewEncoder(opts.EncoderOptions())
Expand Down Expand Up @@ -127,6 +134,95 @@ func TestM3MsgServerWithProtobufHandler(t *testing.T) {
require.Equal(t, m2.StoragePolicy, payload.sp)
}

func TestM3MsgServerWithProtobufHandler_Blackhole(t *testing.T) {
l, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)

w := &mockWriter{m: make(map[string]payload)}
hOpts := Options{
WriteFn: w.write,
InstrumentOptions: instrument.NewOptions(),
BlockholePolicies: []policy.StoragePolicy{baseStoragePolicy},
}
opts := consumer.NewOptions().
SetAckBufferSize(1).
SetConnectionWriteBufferSize(1)

s := server.NewServer(
"a",
consumer.NewMessageHandler(newProtobufProcessor(hOpts), opts),
server.NewOptions(),
)
s.Serve(l)

conn, err := net.Dial("tcp", l.Addr().String())
require.NoError(t, err)
m1 := aggregated.MetricWithStoragePolicy{
Metric: aggregated.Metric{
ID: []byte(testID),
TimeNanos: 1000,
Value: 1,
Type: metric.GaugeType,
},
StoragePolicy: precisionStoragePolicy,
}

encoder := protobuf.NewAggregatedEncoder(nil)
require.NoError(t, encoder.Encode(m1, 2000))
enc := proto.NewEncoder(opts.EncoderOptions())
require.NoError(t, enc.Encode(&msgpb.Message{
Value: encoder.Buffer().Bytes(),
}))
_, err = conn.Write(enc.Bytes())
require.NoError(t, err)

var a msgpb.Ack
dec := proto.NewDecoder(conn, opts.DecoderOptions(), 10)
require.NoError(t, dec.Decode(&a))
require.Equal(t, 0, w.ingested())

// Ensure a metric with a different policy still gets ingested.
m2 := aggregated.MetricWithStoragePolicy{
Metric: aggregated.Metric{
ID: []byte{},
TimeNanos: 0,
Value: 0,
Type: metric.UnknownType,
},
StoragePolicy: policy.MustParseStoragePolicy("5m:180d"),
}
require.NoError(t, encoder.Encode(m2, 3000))
enc = proto.NewEncoder(opts.EncoderOptions())
require.NoError(t, enc.Encode(&msgpb.Message{
Value: encoder.Buffer().Bytes(),
}))
_, err = conn.Write(enc.Bytes())
require.NoError(t, err)
require.NoError(t, dec.Decode(&a))
require.Equal(t, 1, w.ingested())

// Ensure a metric with base policy (equivalent but default precision) is
// still ignored.
m3 := aggregated.MetricWithStoragePolicy{
Metric: aggregated.Metric{
ID: []byte(testID),
TimeNanos: 1000,
Value: 1,
Type: metric.GaugeType,
},
StoragePolicy: baseStoragePolicy,
}
require.NoError(t, encoder.Encode(m3, 3000))
enc = proto.NewEncoder(opts.EncoderOptions())
require.NoError(t, enc.Encode(&msgpb.Message{
Value: encoder.Buffer().Bytes(),
}))
_, err = conn.Write(enc.Bytes())
require.NoError(t, err)
require.NoError(t, dec.Decode(&a))
require.Equal(t, 1, w.ingested())
}

type mockWriter struct {
sync.Mutex

Expand Down
56 changes: 56 additions & 0 deletions src/dbnode/client/client_mock.go
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 7d00195

Please sign in to comment.