Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add telemetry to companion #1117

Merged
merged 32 commits into from
Jan 27, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
dbde5bd
feat: add types for state
SgtPooki Dec 16, 2022
3e3dc16
tmp
SgtPooki Dec 16, 2022
829283e
feat: ipfs-companion tracks views and sessions
SgtPooki Dec 17, 2022
2ab0bbe
Update add-on/src/lib/telemetry.js
SgtPooki Jan 12, 2023
1934425
Update add-on/src/lib/telemetry.js
SgtPooki Jan 12, 2023
faf72a8
Update add-on/src/lib/telemetry.js
SgtPooki Jan 12, 2023
3b83c17
Update add-on/_locales/en/messages.json
SgtPooki Jan 12, 2023
f6b6081
Update add-on/_locales/en/messages.json
SgtPooki Jan 12, 2023
e372e74
Update add-on/_locales/en/messages.json
SgtPooki Jan 12, 2023
11d3b9b
Update add-on/_locales/en/messages.json
SgtPooki Jan 12, 2023
4a0af8f
chore: fix options and state typings
SgtPooki Jan 12, 2023
a256e05
chore: use debug logger
SgtPooki Jan 12, 2023
798c5fe
fix(lint): remove unused method
SgtPooki Jan 12, 2023
8d4d6c8
fix(lint): run 'npm run fix:lint'
SgtPooki Jan 12, 2023
4058da2
chore: build and lint success
SgtPooki Jan 12, 2023
0e66ea9
chore(types): fix type errors
SgtPooki Jan 12, 2023
090d700
chore: add docs/telemetry/COLLECTED_DATA.md
SgtPooki Jan 12, 2023
d83de1a
chore: update old metric group names in logConsent
SgtPooki Jan 12, 2023
ceb65b6
chore: clean up UI
SgtPooki Jan 13, 2023
9db9b6e
chore: use ignite-metrics from npm
SgtPooki Jan 13, 2023
15ab207
chore: update ignite-metrics and some types
SgtPooki Jan 14, 2023
fa915c3
fix(tests): tests dont fail on countly-sdk-web import
SgtPooki Jan 18, 2023
c19ba88
fix: build
SgtPooki Jan 18, 2023
d334ec0
Merge branch 'main' into 1115-feat-getting-basic-metrics-in-ipfs-comp…
SgtPooki Jan 18, 2023
162aaa8
Merge branch 'main' into 1115-feat-getting-basic-metrics-in-ipfs-comp…
SgtPooki Jan 25, 2023
089b0d0
chore: temporarily use updated ignite-metrics
SgtPooki Jan 20, 2023
ee060c2
chore: use deployed ignite-metrics version
SgtPooki Jan 20, 2023
ea798df
chore: address PR comments
SgtPooki Jan 27, 2023
dad1bfa
chore: pin ignite-metrics dependency
SgtPooki Jan 27, 2023
d2f9cab
chore(lint): fix lint errors
SgtPooki Jan 27, 2023
0810499
fix: use browser.runtime.sendMessage
SgtPooki Jan 27, 2023
7160856
Merge branch 'main' into 1115-feat-getting-basic-metrics-in-ipfs-comp…
SgtPooki Jan 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,6 @@
/coverage
/.nyc_output
/add-on/manifest.json

.DS_Store
.vscode
48 changes: 48 additions & 0 deletions add-on/_locales/en/messages.json
Original file line number Diff line number Diff line change
Expand Up @@ -726,5 +726,53 @@
"page_landingWelcome_projects_title": {
"message": "Related Projects",
"description": "Projects section title (page_landingWelcome_projects_title)"
},
"option_header_telemetry": {
"message": "Telemetry",
"description": "A section header on the Preferences screen (option_header_telemetry)"
},
"option_telemetry_disclaimer": {
"message": "We're collecting minimal telemetry data to improve and prioritize our work. Please consent to the collection of these metrics to assist in our efforts!",
"description": "Disclaimer about telemetry collection in the telemetry section on the Preferences screen (option_telemetry_disclaimer)"
},
"option_telemetryGroupMinimal_title": {
"message": "Minimal metrics: session & companion views",
SgtPooki marked this conversation as resolved.
Show resolved Hide resolved
"description": "A title for the 'minimal' grouping of metrics we collect (option_telemetryGroupMinimal_title)"
},
"option_telemetryGroupMinimal_description": {
"message": "We send `session` and `view` events only if ipfs-companion is enabled and your browser is not idle.",
SgtPooki marked this conversation as resolved.
Show resolved Hide resolved
"description": "A description for the 'minimal' grouping of metrics we collect (option_telemetryGroupMinimal_description)"
},
"option_telemetryGroupMinimal_session_description": {
"message": "A `session` event - every 60 seconds.",
SgtPooki marked this conversation as resolved.
Show resolved Hide resolved
"description": "An explanation of the `session` telemetry we collect (option_telemetryGroupMinimal_session_description)"
},
"option_telemetryGroupMinimal_view_description": {
"message": "A `view` event - only for ipfs-companion pages.",
SgtPooki marked this conversation as resolved.
Show resolved Hide resolved
"description": "An explanation of the `session` telemetry we collect (option_telemetryGroupMinimal_session_description)"
},
"option_telemetryGroupMarketing_title": {
"message": "Marketing title",
"description": "A title for the 'marketing' grouping of metrics we collect (option_telemetryGroupMarketing_title)"
},
"option_telemetryGroupMarketing_description": {
"message": "Marketing description",
"description": "A description for the 'marketing' grouping of metrics we collect (option_telemetryGroupMarketing_description)"
},
"option_telemetryGroupPerformance_title": {
"message": "Performance title",
"description": "A title for the 'performance' grouping of metrics we collect (option_telemetryGroupPerformance_title)"
},
"option_telemetryGroupPerformance_description": {
"message": "Performance description",
"description": "A description for the 'performance' grouping of metrics we collect (option_telemetryGroupPerformance_description)"
},
"option_telemetryGroupTracking_title": {
"message": "Tracking title",
"description": "A title for the 'tracking' grouping of metrics we collect (option_telemetryGroupTracking_title)"
},
"option_telemetryGroupTracking_description": {
"message": "Tracking description",
"description": "A description for the 'tracking' grouping of metrics we collect (option_telemetryGroupTracking_description)"
Comment on lines +730 to +752
Copy link
Member Author

@SgtPooki SgtPooki Jan 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SgtPooki to address

Need to remove these since we wont be using them for now

}
}
1 change: 1 addition & 0 deletions add-on/src/background/background.html
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@
<meta charset="utf-8">
<script src="/dist/bundles/ipfs.bundle.js"></script>
<script src="/dist/bundles/backgroundPage.bundle.js"></script>
<body></body>
lidel marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 2 additions & 0 deletions add-on/src/background/background.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,15 @@ import { onInstalled } from '../lib/on-installed.js'
import { getUninstallURL } from '../lib/on-uninstalled.js'
import { optionDefaults } from '../lib/options.js'
import createIpfsCompanion from '../lib/ipfs-companion.js'
import { trackView } from '../lib/telemetry.js'

// register lifecycle hooks early, otherwise we miss first install event
browser.runtime.onInstalled.addListener(onInstalled)
browser.runtime.setUninstallURL(getUninstallURL(browser))

// init add-on after all libs are loaded
document.addEventListener('DOMContentLoaded', async () => {
trackView('background')
// setting debug namespaces require page reload to get applied
const debugNs = (await browser.storage.local.get({ logNamespaces: optionDefaults.logNamespaces })).logNamespaces
if (debugNs !== localStorage.debug) {
Expand Down
2 changes: 2 additions & 0 deletions add-on/src/landing-pages/welcome/store.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
'use strict'
/* eslint-env browser, webextensions */
import browser from 'webextension-polyfill'
import { trackView } from '../../lib/telemetry.js'

export default function createWelcomePageStore (i18n, runtime) {
return function welcomePageStore (state, emitter) {
Expand All @@ -9,6 +10,7 @@ export default function createWelcomePageStore (i18n, runtime) {
state.webuiRootUrl = null
let port
emitter.on('DOMContentLoaded', async () => {
trackView('welcome')
emitter.emit('render')
port = runtime.connect({ name: 'browser-action-port' })
port.onMessage.addListener(async (message) => {
Expand Down
5 changes: 5 additions & 0 deletions add-on/src/lib/ipfs-companion.js
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import createRuntimeChecks from './runtime-checks.js'
import { createContextMenus, findValueForContext, contextMenuCopyAddressAtPublicGw, contextMenuCopyRawCid, contextMenuCopyCanonicalAddress, contextMenuViewOnGateway, contextMenuCopyPermalink, contextMenuCopyCidAddress } from './context-menus.js'
import { registerSubdomainProxy } from './http-proxy.js'
import { runPendingOnInstallTasks } from './on-installed.js'
import { handleConsentFromState, startSession, endSession } from './telemetry.js'
const log = debug('ipfs-companion:main')
log.error = debug('ipfs-companion:main:error')

Expand All @@ -33,6 +34,7 @@ export default async function init () {
// INIT
// ===================================================================
let ipfs // ipfs-api instance
/** @type {ReturnType<initState>} */
let state // avoid redundant API reads by utilizing local cache of various states
let dnslinkResolver
let ipfsPathValidator
Expand All @@ -57,6 +59,7 @@ export default async function init () {
notify = createNotifier(getState)

if (state.active) {
startSession()
// It's ok for this to fail, node might be unavailable or mis-configured
try {
ipfs = await initIpfsClient(browser, state)
Expand Down Expand Up @@ -556,6 +559,7 @@ export default async function init () {
await registerSubdomainProxy(getState, runtime)
shouldRestartIpfsClient = true
shouldStopIpfsClient = !state.active
state.active ? startSession() : endSession()
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If state.active is changed, and state is true-ish, start session. Otherwise, end session.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@whizzzkid @lidel is there a better place to do this

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for now this should be ok, in the future we might wanna have a message being broadcast and such listeners can subscribe to those.

break
case 'ipfsNodeType':
if (change.oldValue !== braveNodeType && change.newValue === braveNodeType) {
Expand Down Expand Up @@ -622,6 +626,7 @@ export default async function init () {
break
}
}
handleConsentFromState(state)

if ((state.active && shouldRestartIpfsClient) || shouldStopIpfsClient) {
try {
Expand Down
8 changes: 7 additions & 1 deletion add-on/src/lib/options.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
import { isIPv4, isIPv6 } from 'is-ip'
import isFQDN from 'is-fqdn'

// console.log('igniteMetrics: ', igniteMetrics)

export const optionDefaults = Object.freeze({
active: true, // global ON/OFF switch, overrides everything else
ipfsNodeType: 'external',
Expand Down Expand Up @@ -31,7 +33,11 @@ export const optionDefaults = Object.freeze({
importDir: '/ipfs-companion-imports/%Y-%M-%D_%h%m%s/',
useLatestWebUI: false,
dismissedUpdate: null,
openViaWebUI: true
openViaWebUI: true,
telemetryGroupMinimal: true,
telemetryGroupMarketing: false,
telemetryGroupPerformance: false,
telemetryGroupTracking: false
SgtPooki marked this conversation as resolved.
Show resolved Hide resolved
})

function buildDefaultIpfsNodeConfig () {
Expand Down
27 changes: 26 additions & 1 deletion add-on/src/lib/state.js
Original file line number Diff line number Diff line change
@@ -1,9 +1,34 @@
'use strict'
/* eslint-env browser, webextensions */

// @ts-check
import { safeURL, isHostname } from './options.js'

/**
* @typedef {object} CompanionState
* @extends {typeof import('./options').optionDefaults}
* @property {number} peerCount
* @property {URL} pubGwURL
* @property {string} pubGwURLString
* @property {URL} pubSubdomainGwURL
* @property {string} pubSubdomainGwURLString
* @property {boolean} redirect
* @property {URL} apiURL
* @property {string} apiURLString
* @property {URL} gwURL
* @property {string} gwURLString
* @property {boolean|string} dnslinkPolicy
* @property {(url: string|URL) => boolean} activeIntegrations
* @property {boolean} localGwAvailable
* @property {string} webuiRootUrl
*/

export const offlinePeerCount = -1
/**
*
* @param {typeof import('./options').optionDefaults} options
* @param {Partial<typeof import('./options').optionDefaults>} overrides
* @returns {CompanionState}
*/
export function initState (options, overrides) {
// we store options and some pregenerated values to avoid async storage
// reads and minimize performance impact on overall browsing experience
Expand Down
60 changes: 60 additions & 0 deletions add-on/src/lib/telemetry.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@

import { MetricsProvider } from '@ipfs-shipyard/ignite-metrics/vanilla'
Copy link
Contributor

@whizzzkid whizzzkid Dec 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ipfs-shipyard/ignite-metrics/vanilla

This needs to be mapped correctly, this breaks the build.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be working properly with latest release of ignite-metrics


let metricsProvider = null
export function getMetricsProviderInstance () {
if (metricsProvider != null) {
return metricsProvider
}
metricsProvider = new MetricsProvider({ appKey: '393f72eb264c28a1b59973da1e0a3938d60dc38a', autoTrack: false })

return metricsProvider
SgtPooki marked this conversation as resolved.
Show resolved Hide resolved
}

/**
* @param {ReturnType<import('./state').initState>['options']} state
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we import types form the declaration from the lib?

* @returns {string[]}
*/
function mapStateToConsent (stateOptions) {
const obj = {
minimal: stateOptions?.telemetryGroupMinimal || false,
marketing: stateOptions?.telemetryGroupMarketing || false,
performance: stateOptions?.telemetryGroupPerformance || false,
tracking: stateOptions?.telemetryGroupTracking || false
}

const enabledConsentGroups = Object.keys(obj).filter(key => obj[key] === true)
console.log('enabledConsentGroups: ', enabledConsentGroups)
return enabledConsentGroups
}
function logConsent () {
SgtPooki marked this conversation as resolved.
Show resolved Hide resolved
console.log('checkConsent(\'minimal\'): ', getMetricsProviderInstance().checkConsent('minimal'))
console.log('checkConsent(\'marketing\'): ', getMetricsProviderInstance().checkConsent('marketing'))
console.log('checkConsent(\'performance\'): ', getMetricsProviderInstance().checkConsent('performance'))
console.log('checkConsent(\'tracking\'): ', getMetricsProviderInstance().checkConsent('tracking'))
SgtPooki marked this conversation as resolved.
Show resolved Hide resolved
}
/**
SgtPooki marked this conversation as resolved.
Show resolved Hide resolved
*
* @param {ReturnType<import('./state')['initState']>} state
* @returns {void}
*/
export function handleConsentFromState (state) {
console.log('handleConsentFromState', state)
getMetricsProviderInstance().updateConsent(mapStateToConsent(state))
logConsent()
}

export function handleConsentUpdate (consent) {
console.log('handleConsentUpdate', consent)
getMetricsProviderInstance().updateConsent(consent)
}

// const ignoredViewsRegex = [/^ipfs:\/\/.*/]
const ignoredViewsRegex = []
export function trackView (view) {
console.log('trackView called for view: ', view)
getMetricsProviderInstance().metricsService.track_pageview(view, ignoredViewsRegex)
}

export const startSession = (...args) => getMetricsProviderInstance().startSession(...args)
export const endSession = (...args) => getMetricsProviderInstance().endSession(...args)
43 changes: 43 additions & 0 deletions add-on/src/options/forms/telemetry-form.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
'use strict'
/* eslint-env browser, webextensions */

import browser from 'webextension-polyfill'
import html from 'choo/html/index.js'
import switchToggle from '../../pages/components/switch-toggle.js'

export default function telemetryForm ({
onOptionChange,
...stateOptions
}) {
// const onTelemetryChange = (key) => {

// }
return html`
<form>
<fieldset class="mb3 pa1 pa4-ns pa3 bg-snow-muted charcoal">
<h2 class="ttu tracked f6 fw4 teal mt0-ns mb3-ns mb1 mt2 ">${browser.i18n.getMessage('option_header_telemetry')}</h2>
<div class="mb2">
<p>${browser.i18n.getMessage('option_telemetry_disclaimer')}</p>
<p>
<a class="link underline hover-aqua" href="https://github.com/ipfs/ipfs-gui/issues/125" target="_blank">
${browser.i18n.getMessage('option_legend_readMore')}
</a>
</p>
</div>
<div class="flex-row-ns pb0-ns">
<label for="telemetryGroupMinimal">
<dl>
<dt>${browser.i18n.getMessage('option_telemetryGroupMinimal_title')}</dt>
<dd>
<p>${browser.i18n.getMessage('option_telemetryGroupMinimal_description')}</p>
<p>${browser.i18n.getMessage('option_telemetryGroupMinimal_session_description')}</p>
<p>${browser.i18n.getMessage('option_telemetryGroupMinimal_view_description')}</p>
</dd>
</dl>
</label>
<div class="self-center-ns">${switchToggle({ id: 'telemetryGroupMinimal', checked: stateOptions.telemetryGroupMinimal, onchange: onOptionChange('telemetryGroupMinimal') })}</div>
</div>
</fieldset>
</form>
`
}
9 changes: 9 additions & 0 deletions add-on/src/options/page.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ import dnslinkForm from './forms/dnslink-form.js'
import gatewaysForm from './forms/gateways-form.js'
import apiForm from './forms/api-form.js'
import experimentsForm from './forms/experiments-form.js'
import telemetryForm from './forms/telemetry-form.js'
import resetForm from './forms/reset-form.js'
import { trackView } from '../lib/telemetry.js'

// Render the options page:
// Passed current app `state` from the store and `emit`, a function to create
Expand Down Expand Up @@ -102,6 +104,13 @@ export default function optionsPage (state, emit) {
logNamespaces: state.options.logNamespaces,
onOptionChange
})}
${telemetryForm({
telemetryGroupMinimal: state.options.telemetryGroupMinimal,
telemetryGroupMarketing: state.options.telemetryGroupMarketing,
telemetryGroupPerformance: state.options.telemetryGroupPerformance,
telemetryGroupTracking: state.options.telemetryGroupTracking,
onOptionChange
})}
Comment on lines +106 to +112
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SgtPooki to update

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SgtPooki was this updated?

${resetForm({
onOptionsReset
})}
Expand Down
5 changes: 5 additions & 0 deletions add-on/src/options/store.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
import browser from 'webextension-polyfill'
import { optionDefaults } from '../lib/options.js'
import createRuntimeChecks from '../lib/runtime-checks.js'
import { trackView } from '../lib/telemetry.js'

// The store contains and mutates the state for the app
export default function optionStore (state, emitter) {
Expand All @@ -12,11 +13,15 @@ export default function optionStore (state, emitter) {
const updateStateOptions = async () => {
const runtime = await createRuntimeChecks(browser)
state.withNodeFromBrave = runtime.brave && await runtime.brave.getIPFSEnabled()
/**
* FIXME: Why are we setting `state.options` when state is supposed to extend options?
*/
state.options = await getOptions()
Comment on lines +15 to 18
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lidel this threw me off when first making changea. Do you know what this is supposed to be used for? Seems like an artifact-of-old.

emitter.emit('render')
}

emitter.on('DOMContentLoaded', async () => {
trackView('options')
updateStateOptions()
browser.storage.onChanged.addListener(updateStateOptions)
})
Expand Down
2 changes: 2 additions & 0 deletions add-on/src/popup/browser-action/store.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import { browserActionFilesCpImportCurrentTab } from '../../lib/ipfs-import.js'
import { ipfsContentPath } from '../../lib/ipfs-path.js'
import { welcomePage, optionsPage } from '../../lib/constants.js'
import { contextMenuViewOnGateway, contextMenuCopyAddressAtPublicGw, contextMenuCopyPermalink, contextMenuCopyRawCid, contextMenuCopyCanonicalAddress, contextMenuCopyCidAddress } from '../../lib/context-menus.js'
import { trackView } from '../../lib/telemetry.js'

// The store contains and mutates the state for the app
export default (state, emitter) => {
Expand Down Expand Up @@ -38,6 +39,7 @@ export default (state, emitter) => {
let port

emitter.on('DOMContentLoaded', async () => {
trackView('browser-action')
// initial render with status stub
emitter.emit('render')
// initialize connection to the background script which will trigger UI updates
Expand Down
2 changes: 2 additions & 0 deletions add-on/src/popup/quick-import.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import { formatImportDirectory } from '../lib/ipfs-import.js'
import all from 'it-all'
import drop from 'drag-and-drop-files'
import { filesize } from 'filesize'
import { trackView } from '../lib/telemetry.js'

document.title = browser.i18n.getMessage('quickImport_page_title')

Expand Down Expand Up @@ -48,6 +49,7 @@ function quickImportStore (state, emitter) {
let port

emitter.on('DOMContentLoaded', async () => {
trackView('quick-import')
// initialize connection to the background script which will trigger UI updates
port = browser.runtime.connect({ name: 'browser-action-port' })
port.onMessage.addListener(async (message) => {
Expand Down
Loading