Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add POC for measuring adoption #859

Merged
merged 102 commits into from
Mar 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
102 commits
Select commit Hold shift + click to select a range
9952df2
feat: add POC for measuring adoption
smoya Oct 22, 2023
07c7866
Update convert command
peter-rr Nov 6, 2023
e9e79f6
Add metrics recording to some commands
peter-rr Oct 31, 2023
4c7839d
feat: send metrics to NR only when installed as NPM binary (#2)
smoya Nov 23, 2023
94d6e03
feat: add new metric recording for action invocation (#3)
peter-rr Nov 23, 2023
17e9a5f
chore: refactor metrics recording so it gets unified (#4)
smoya Nov 23, 2023
ee6f3be
Merge branch 'master' into feat/adoptionMetrics
smoya Nov 23, 2023
0bb0791
remove unused hook
smoya Nov 23, 2023
3ce1520
add missing execute metrics
smoya Nov 23, 2023
2fd9762
fix metrics func callable
smoya Nov 23, 2023
03eaf22
chore: unify logic for metrics collection when any command is invoked…
peter-rr Dec 12, 2023
b99c19e
Merge branch 'master' into feat/adoptionMetrics
smoya Dec 12, 2023
4f81da5
Merge branch 'master' into feat/adoptionMetrics
smoya Dec 12, 2023
ed74a95
update @smoya/asyncapi-adoption-metrics
smoya Dec 13, 2023
ac30d00
Merge branch 'master' into feat/adoptionMetrics
smoya Dec 13, 2023
8f3e48a
fix: update @smoya/multi-parser to v5.0.1
smoya Dec 13, 2023
b10f763
fix: update @asyncapi/parser
smoya Dec 13, 2023
cbd1c51
chore: remove unused imports
smoya Dec 13, 2023
32c822c
Solve some bugs reported by SonarCloud (#7)
peter-rr Dec 13, 2023
e855ab3
Merge branch 'master' into feat/adoptionMetrics
smoya Dec 13, 2023
e6f4917
feat: add message to warn users about metrics collection (#6)
peter-rr Dec 18, 2023
60d149d
Merge remote-tracking branch 'origin/master' into feat/adoptionMetrics
smoya Jan 12, 2024
7a22c5f
update @smoya/asyncapi-adoption-metrics to latest version
smoya Jan 12, 2024
d57655b
chore: unify logic for metrics collection when any command is execute…
peter-rr Jan 12, 2024
16a38a5
Add METRICS_COLLECTION markdown file
peter-rr Jan 18, 2024
7c08679
Update document
peter-rr Jan 19, 2024
67801cf
Add remaining info to the document
peter-rr Jan 22, 2024
463911d
Add relative link to disable tracking section
peter-rr Jan 22, 2024
d53c0ec
Update link to section in the same document
peter-rr Jan 24, 2024
afd053c
Move markdown document to 'docs' folder
peter-rr Jan 24, 2024
33759d2
feat: stop sending metrics when CLI command is executed on CI/CD pipe…
peter-rr Jan 30, 2024
188bc72
Update document with requested changes
peter-rr Jan 30, 2024
64120fd
Rename one of the metrics on the document
peter-rr Jan 30, 2024
4e22ebf
feat: rename one of the metrics (#12)
peter-rr Jan 31, 2024
85efb45
Fix issue coming from 'bundle' command (#11)
peter-rr Jan 31, 2024
be26e67
Convert 'optimizations' array into an object
peter-rr Jan 31, 2024
e0f6949
Apply JSON format to 'optimizations' array
peter-rr Feb 2, 2024
75e0a2d
Adapt metadata to New Relic required format for attributes
peter-rr Feb 5, 2024
8d227f3
Clean some code
peter-rr Feb 5, 2024
e6c25fa
Reduce cognitive complexity
peter-rr Feb 5, 2024
d096c54
Refactor to reach the cognitive complexity allowed
peter-rr Feb 6, 2024
5fe9bc4
Clean some repeated code
peter-rr Feb 6, 2024
22e98ae
Apply suggested changes
peter-rr Feb 7, 2024
ab57855
Change some metadata to camel case
peter-rr Feb 7, 2024
737f3ee
Solve cognitive complexity issue and fix failing test
peter-rr Feb 7, 2024
397efd5
Reduce the complexity of metrics collection
peter-rr Feb 8, 2024
2bd4077
Change warning message to informational message about metrics usage
peter-rr Feb 13, 2024
131c52a
Update 'How to disable tracking' section
peter-rr Feb 13, 2024
e3fc3c7
Add new command to disable analytics
peter-rr Feb 16, 2024
42aa925
Update md document and warning message
peter-rr Feb 19, 2024
68e2488
Merge branch 'feat/adoptionMetrics' into feat/adoptMetrics-metrics-docs
peter-rr Feb 19, 2024
18059c9
Merge pull request #13 from peter-rr/fix/adoptMetrics-optimize-issue
peter-rr Feb 19, 2024
edb7f38
Merge pull request #9 from peter-rr/feat/adoptMetrics-metrics-docs
peter-rr Feb 19, 2024
e1a315c
Solve some sonarcloud issues in base.ts
peter-rr Feb 19, 2024
4b26d58
Merge branch 'master' into feat/adoptionMetrics
peter-rr Feb 19, 2024
a9b7feb
Add '--enable' flag to analytics command
peter-rr Feb 19, 2024
aaf7731
Get rid of ASYNCAPI_METRICS env variable
peter-rr Feb 20, 2024
1b5a62b
Update MD document and info message about metrics
peter-rr Feb 20, 2024
e16f70c
Get rid of readFileSync, using fs promises instead
peter-rr Feb 20, 2024
a5d1e95
Update 'catch' with more specific errors
peter-rr Feb 20, 2024
783f77a
Refactor code for enabling/disabling analytics
peter-rr Feb 20, 2024
8a22fab
update @smoya/asyncapi-adoption-metrics to latest version
smoya Feb 20, 2024
9b0abd3
update @smoya/asyncapi-adoption-metrics and @asyncapi/parser to lates…
smoya Feb 20, 2024
671f4fd
Solve linter issues
peter-rr Feb 20, 2024
3436ccf
Merge branch 'master' into feat/adoptionMetrics
peter-rr Feb 22, 2024
135cdff
Update @smoya/asyncapi-adoption-metrics to latest version
peter-rr Feb 23, 2024
b5928e7
Add user_id metadata
peter-rr Feb 23, 2024
c9945ee
Solve linter issue
peter-rr Feb 23, 2024
95f755d
Solve sonarcloud issues
peter-rr Feb 27, 2024
a8d52a1
Merge branch 'master' into feat/adoptionMetrics
peter-rr Feb 27, 2024
9382563
Merge branch 'master' into feat/adoptionMetrics
peter-rr Feb 27, 2024
2f468d0
Add integration tests for analytics command
peter-rr Feb 28, 2024
e51dcc3
Merge branch 'master' into feat/adoptionMetrics
peter-rr Feb 28, 2024
0f3da30
Solve linter issues
peter-rr Feb 28, 2024
49a2be1
Add 'source' metadata from hash generated
peter-rr Mar 1, 2024
a21c8d6
Modify generateSHA1() function to be async
peter-rr Mar 1, 2024
2c9331a
Solve failing tests
peter-rr Mar 5, 2024
b9ee8da
Merge branch 'master' into feat/adoptionMetrics
peter-rr Mar 5, 2024
5d7e726
Solve sonarcloud security issue
peter-rr Mar 5, 2024
4452b44
Solve another sonarcloud issue
peter-rr Mar 5, 2024
0d1059c
Add analytics metadata to 'new' command
peter-rr Mar 5, 2024
9527057
Fix issue related to generateSHA256()
peter-rr Mar 6, 2024
ed03700
Merge remote-tracking branch 'upstream/master' into fix/adoptMetrics-…
peter-rr Mar 7, 2024
e5c3081
Solve more conflicts in package-lock.json
peter-rr Mar 7, 2024
de5b49f
Merge branch 'master' into feat/adoptionMetrics
peter-rr Mar 8, 2024
48ba1a9
Merge branch 'master' into feat/adoptionMetrics
peter-rr Mar 8, 2024
9ad9d87
Solve new conflicts
peter-rr Mar 8, 2024
eba927a
Merge branch 'master' into feat/adoptionMetrics
peter-rr Mar 11, 2024
a8e6621
Merge branch 'master' into feat/adoptionMetrics
smoya Mar 11, 2024
9f1be17
Merge branch 'master' into feat/adoptionMetrics
peter-rr Mar 12, 2024
fcfe8ee
Disable 'source' metadata collection
peter-rr Mar 12, 2024
43f3b0b
Merge branch 'master' into feat/adoptionMetrics
peter-rr Mar 13, 2024
dd4362f
Merge branch 'master' into feat/adoptionMetrics
peter-rr Mar 14, 2024
5d0cd26
Merge branch 'master' into feat/adoptionMetrics
peter-rr Mar 14, 2024
eacbf73
Merge branch 'master' into feat/adoptionMetrics
peter-rr Mar 14, 2024
1ce7cda
Merge branch 'master' into feat/adoptionMetrics
peter-rr Mar 14, 2024
70aad0d
Merge branch 'master' into feat/adoptionMetrics
peter-rr Mar 15, 2024
9c371ce
Merge branch 'master' into feat/adoptionMetrics
peter-rr Mar 18, 2024
ee9f626
Merge branch 'master' into feat/adoptionMetrics
peter-rr Mar 19, 2024
9cd11e8
Update asyncapi-adoption-metrics to latest version
peter-rr Mar 19, 2024
eb0111b
Update multi-parser to latest version
peter-rr Mar 19, 2024
2305ab4
Merge branch 'master' into feat/adoptionMetrics
peter-rr Mar 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions bin/run
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
#!/usr/bin/env node

process.env.NODE_ENV = 'development';

const oclif = require('@oclif/core');

oclif.run()
Expand Down
14 changes: 14 additions & 0 deletions bin/run_bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
#!/usr/bin/env node

// Only the binary installed through NPM is considered production environment. See "bin" in package.json.
process.env.NODE_ENV = 'production';

const oclif = require('@oclif/core');

oclif.run()
.then(require('@oclif/core/flush'))
.catch((err) => {
const oclifHandler = require('@oclif/core/handle');
return oclifHandler(err.message);
});

3 changes: 3 additions & 0 deletions bin/run_bin.cmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
@echo off

node "%~dp0\run_bin" %*
49 changes: 49 additions & 0 deletions docs/metrics_collection.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Metrics collection guideline

AsyncAPI **anonymously** tracks command executions to improve the specification and tools, ensuring no sensitive data reaches our servers. It aids in comprehending how AsyncAPI tools are used and adopted, facilitating ongoing improvements to our specifications and tools.

Even though metrics collection is enabled by default, you can always [disable tracking](#how-to-disable-tracking) if you want to.

## What we collect
We are collecting the following metrics:

- `asyncapi_adoption.action.invoked`:
With this metric we are tracking the command executed on the CLI as soon as the command is invoked, so it has already been executed but not finished yet. We just want to know which commands are used, regardless they have failed or succeeded.

Example of the data collected by this metric when the `validate` command has been executed:
```
asyncapi_adoption.action.invoked COUNTER { action: 'validate' } 1
```

- `asyncapi_adoption.action.finished`:
This metric tracks the command executed once it has already finished, carrying the result of the execution and some metadata based on the AsyncAPI document in place.

Example for `validate` command successfully executed and finished:
```
asyncapi_adoption.action.finished COUNTER {
validation_result: 'valid',
success: true,
asyncapi_version: '2.6.0',
asyncapi_servers: 2,
asyncapi_channels: 4,
asyncapi_messages: 3,
asyncapi_operations_send: 3,
asyncapi_operations_receive: 1,
asyncapi_schemas: 52,
action: 'validate'
} 1
```

## Where the data is stored
We are making use of [New Relic API](https://docs.newrelic.com/docs/apis/intro-apis/introduction-new-relic-apis/#rest-api) to send the metrics collected to _New Relic_ servers, where they are stored, and finally visualized on the AsyncAPI website.

Metrics won't be collected in CI environments, or when the "CI" env variable is set up to "true".

## How to disable tracking
To disable tracking, please run the following command:
`asyncapi config analytics --disable`

Once disabled, if you want to enable tracking back again then run:
`asyncapi config analytics --enable`

Remember that keeping this tracking enabled will help AsyncAPI community to provide better specifications and tools in the future.
Loading
Loading