Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add keystone telemetry inform command and update telemetry policy #9292

Merged
merged 19 commits into from
Aug 20, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/add-telemetry-inform.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@keystone-6/core": minor
---

Adds `keystone telemetry inform` command to show an informed consent notice
9 changes: 5 additions & 4 deletions docs/content/docs/reference/telemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ Keystone collects telemetry information in the form of two different types of da
We refer to these two different reports, as “device telemetry” and “project telemetry” respectively.

These reports are forwarded to [https://telemetry.keystonejs.com/](https://telemetry.keystonejs.com/), and are reported separately to minimize any correlation between them insofar as the timing and grouping of that data, that an otherwise combined report may have. We are collecting these two reports for different reasons, and thus have no need to associate them.
We differentiate and record the type and version of reports from the URL used by Keystone.
Copy link
Member Author

@dcousens dcousens Aug 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving forward, we'll use the /3 version in the endpoint to differentiate report schema versions in the same way we differentiate report types using /project and /device.

This makes that abundantly transparent.


We additionally record a timestamp of the time that the report is received by the server at [https://telemetry.keystonejs.com](https://telemetry.keystonejs.com/).

Expand All @@ -89,7 +90,7 @@ A device telemetry report is formatted as JSON and currently looks like:
The type of information contained within a project telemetry report is currently:

- The last date you used `keystone dev` for this project, and
- The resolved versions of any `@keystone-6` packages used by this project, and
- The resolved package versions of any `@keystone-6` packages used by this project, and
- The number of lists for this project, and
- The name and number of field types that you are using

Expand All @@ -98,7 +99,7 @@ A project telemetry report is formatted as JSON and currently looks like:
```json
{
"previous": "2022-11-23",
"versions": {
"packages": {
"@keystone-6/auth": "5.0.1",
"@keystone-6/core": "3.1.2",
"@keystone-6/document-renderer": "1.1.2",
Expand Down Expand Up @@ -185,8 +186,8 @@ If you wish to see how telemetry is currently configured for your device or proj

## What if I have a complaint or question

If you have any questions or concerns about the information that is gathered please contact us by logging a GitHub Issue [https://github.com/keystonejs/keystone](https://github.com/keystonejs/keystone).
If you have any questions or concerns about the information that is gathered please contact us by logging a GitHub Issue [https://github.com/keystonejs/keystone](https://github.com/keystonejs/keystone).

Alternatively please contact our Privacy Officer by email to [privacy@keystonejs.com](mailto:privacy@keystonejs.com), or by mail to Level 10, 191 Clarence Street, Sydney NSW 2000.
Alternatively please contact our Privacy Officer by email to [privacy@keystonejs.com](mailto:privacy@keystonejs.com), or by mail to Level 10, 191 Clarence Street, Sydney NSW 2000.

For further information about Keystone’s security policy please see [https://github.com/keystonejs/keystone/security/policy](https://github.com/keystonejs/keystone/security/policy)
135 changes: 74 additions & 61 deletions packages/core/src/lib/telemetry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,17 +5,18 @@ import ci from 'ci-info'
import Conf from 'conf'
import {
bold,
blue as b,
yellow as y,
red as r,
green as g
green as g,
grey,
} from 'chalk'
import {
type Configuration,
type Device,
type PackageName,
type Project,
type TelemetryVersion1,
type TelemetryVersion2and3,
type TelemetryVersion2,
} from '../types/telemetry'
import { type DatabaseProvider } from '../types'
import { type InitialisedList } from './core/initialise-lists'
Expand All @@ -37,8 +38,14 @@ function log (message: unknown) {
}
}

type Telemetry = TelemetryVersion2
type TelemetryOK = Exclude<Telemetry, false | undefined>
type Configuration = ReturnType<typeof getTelemetryConfig>['userConfig']

function getTelemetryConfig () {
const userConfig = new Conf<Configuration>({
const userConfig = new Conf<{
telemetry?: TelemetryVersion2
}>({
projectName: 'keystonejs',
projectSuffix: '',
projectVersion: '3.0.0',
Expand All @@ -47,15 +54,15 @@ function getTelemetryConfig () {
const existing = store.get('telemetry') as TelemetryVersion1
if (!existing) return // skip non-configured or known opt-outs

const replacement: TelemetryVersion2and3 = {
const replacement: TelemetryVersion2 = {
informedAt: null, // re-inform
device: {
lastSentDate: existing.device.lastSentDate ?? null,
},
projects: {}, // see below
}

// copy existing project lastSentDate's
// copy existing project.lastSentDate's
for (const [projectPath, project] of Object.entries(existing.projects)) {
if (projectPath === 'default') continue // informedAt moved to device.lastSentDate

Expand All @@ -70,16 +77,16 @@ function getTelemetryConfig () {
}
}

store.set('telemetry', replacement)
store.set('telemetry', replacement satisfies TelemetryVersion2)
},
'^3.0.0': (store) => {
const existing = store.get('telemetry') as TelemetryVersion2and3
const existing = store.get('telemetry') as TelemetryVersion2
if (!existing) return // skip non-configured or known opt-outs

store.set('telemetry', {
...existing,
informedAt: null, // re-inform
} satisfies TelemetryVersion2and3)
} satisfies Telemetry)
},
},
})
Expand All @@ -90,23 +97,15 @@ function getTelemetryConfig () {
}
}

function getDefaultedTelemetryConfig () {
const { telemetry, userConfig } = getTelemetryConfig()

if (telemetry === undefined) {
return {
telemetry: {
informedAt: null,
device: {
lastSentDate: null,
},
projects: {},
} as TelemetryVersion2and3, // help Typescript infer the type
userConfig,
}
}

return { telemetry, userConfig }
function getDefault (telemetry: Telemetry) {
if (telemetry) return telemetry
return {
informedAt: null,
device: {
lastSentDate: null,
},
projects: {},
} satisfies Telemetry // help Typescript infer the type
}

const todaysDate = new Date().toISOString().slice(0, 10)
Expand All @@ -133,42 +132,49 @@ function collectFieldCount (lists: Record<string, InitialisedList>) {
}

function collectPackageVersions () {
const versions: Project['versions'] = {
const packages: Project['packages'] = {
'@keystone-6/core': '0.0.0', // effectively unknown
}

for (const packageName of packageNames) {
try {
const packageJson = require(`${packageName}/package.json`)
versions[packageName] = packageJson.version
packages[packageName] = packageJson.version
} catch {
// do nothing, most likely because the package is not installed
}
}

return versions
return packages
}

function printAbout () {
console.log(`${y`Keystone collects anonymous data when you run`} ${g`"keystone dev"`}`)
console.log()
console.log(`For more information, including how to opt-out see https://keystonejs.com/telemetry`)
}

export function printTelemetryStatus () {
function printNext (telemetry: Telemetry) {
if (!telemetry) {
console.log(`Telemetry data will ${r`not`} be sent by this system user`)
return
}
console.log(`Telemetry data will be sent the next time you run ${g`"keystone dev"`}`)
dcousens marked this conversation as resolved.
Show resolved Hide resolved
}

function printTelemetryStatus () {
const { telemetry } = getTelemetryConfig()

if (telemetry === undefined) {
console.log(`Keystone telemetry has been reset to ${y`uninitialized`}`)
dcousens marked this conversation as resolved.
Show resolved Hide resolved
console.log()
console.log(`Telemetry will be sent the next time you run ${g`"keystone dev"`}, unless you opt-out`)
printNext(telemetry)
return
}

if (telemetry === false) {
console.log(`Keystone telemetry is ${r`disabled`}`)
console.log()
console.log(`Telemetry will ${r`not`} be sent by this system user`)
printNext(telemetry)
return
}

Expand All @@ -181,22 +187,24 @@ export function printTelemetryStatus () {
}

console.log()
console.log(`Telemetry will be sent the next time you run ${g`"keystone dev"`}, unless you opt-out`)
printNext(telemetry)
}

function inform () {
const { telemetry, userConfig } = getDefaultedTelemetryConfig()

// no telemetry? somehow we missed something, do nothing
if (!telemetry) return

function inform (
telemetry: TelemetryOK,
userConfig: Configuration
) {
console.log() // gap to help visiblity
console.log(`${bold('Keystone Telemetry')}`)
printAbout()
console.log(`You can use ${g`"keystone telemetry --help"`} to update your preferences at any time`)
console.log()
console.log(`No telemetry data has been sent, but telemetry will be sent the next time you run ${g`"keystone dev"`}, unless you opt-out`)
if (telemetry.informedAt === null) {
console.log(`No telemetry data has been sent as part of this notice`)
}
printNext(telemetry)
console.log() // gap to help visiblity
console.log(`For more information, including how to opt-out see ${grey`https://keystonejs.com/telemetry`} (updated ${b`2024-08-15`})`)
Copy link
Member Author

@dcousens dcousens Aug 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As everyone will be re-informed as of #9290, this date helps to show that the information has been updated (however minor that change is)


// update the informedAt
telemetry.informedAt = new Date().toJSON()
Expand All @@ -221,13 +229,10 @@ async function sendEvent (eventType: 'project' | 'device', eventData: Project |
async function sendProjectTelemetryEvent (
cwd: string,
lists: Record<string, InitialisedList>,
dbProviderName: DatabaseProvider
dbProviderName: DatabaseProvider,
telemetry: TelemetryOK,
userConfig: Configuration
) {
const { telemetry, userConfig } = getDefaultedTelemetryConfig()

// no telemetry? somehow we missed something, do nothing
if (!telemetry) return

const project = telemetry.projects[cwd] ?? { lastSentDate: null }
const { lastSentDate } = project
if (lastSentDate && lastSentDate >= todaysDate) {
Expand All @@ -239,7 +244,7 @@ async function sendProjectTelemetryEvent (
previous: lastSentDate,
fields: collectFieldCount(lists),
lists: Object.keys(lists).length,
versions: collectPackageVersions(),
packages: collectPackageVersions(),
database: dbProviderName,
})

Expand All @@ -248,12 +253,10 @@ async function sendProjectTelemetryEvent (
userConfig.set('telemetry', telemetry)
}

async function sendDeviceTelemetryEvent () {
const { telemetry, userConfig } = getDefaultedTelemetryConfig()

// no telemetry? somehow we missed something, do nothing
if (!telemetry) return

async function sendDeviceTelemetryEvent (
telemetry: TelemetryOK,
userConfig: Configuration
) {
const { lastSentDate } = telemetry.device
if (lastSentDate && lastSentDate >= todaysDate) {
log('device telemetry already sent today')
Expand Down Expand Up @@ -285,26 +288,36 @@ export async function runTelemetry (
return
}

const { telemetry } = getDefaultedTelemetryConfig()
const { telemetry, userConfig } = getTelemetryConfig()

// don't run if the user has opted out
// or if somehow our defaults are problematic, do nothing
if (!telemetry) return
if (telemetry === false) return

// don't send telemetry before we inform the user, allowing opt-out
if (!telemetry.informedAt) return inform()
const telemetryDefaulted = getDefault(telemetry)
if (!telemetryDefaulted.informedAt) return inform(telemetryDefaulted, userConfig)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest adding an explicit log here to remind that telemetry is active:

    console.log('🛎️  Telemetry is enabled')

image

Copy link
Member Author

@dcousens dcousens Aug 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pretty neat, what about showing it is disabled too?

    console.log('🛎️  Telemetry is enabled')
    // or
    console.log('✅  Telemetry is enabled')

and

    console.log('❎  Telemetry is disabled')

Copy link
Member Author

@dcousens dcousens Aug 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At some stage I want to switch to declarative wording here rather than procedural, but we won't do that in this pull request

Example is Database unchanged versus Connecting to the database

Copy link
Member Author

@dcousens dcousens Aug 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kennedybaird can we do this in a different pull request?
I want to merge and release this and #9290 as soon as possible

The suggested improvement might require a new set of wiring and functions, depending on how we do this

Copy link
Member Author

@dcousens dcousens Aug 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And, like in #8118 (see #8118 (comment)), the phrasing of this might resurface whether we want to try and manage our dependencies telemetry configuration too

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


await sendProjectTelemetryEvent(cwd, lists, dbProviderName)
await sendDeviceTelemetryEvent()
await sendProjectTelemetryEvent(cwd, lists, dbProviderName, telemetryDefaulted, userConfig)
await sendDeviceTelemetryEvent(telemetryDefaulted, userConfig)
} catch (err) {
log(err)
}
}

export function statusTelemetry () {
printTelemetryStatus()
}

export function informTelemetry () {
const { userConfig } = getTelemetryConfig()
inform(getDefault(false), userConfig)
}

export function enableTelemetry () {
const { telemetry, userConfig } = getTelemetryConfig()
if (telemetry === false) {
userConfig.delete('telemetry')
if (!telemetry) {
userConfig.set('telemetry', getDefault(telemetry))
}
printTelemetryStatus()
}
Expand Down
13 changes: 8 additions & 5 deletions packages/core/src/scripts/telemetry.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
import chalk from 'chalk'
import { bold } from 'chalk'
import {
printTelemetryStatus,
enableTelemetry,
disableTelemetry,
enableTelemetry,
resetTelemetry,
statusTelemetry,
informTelemetry,
} from '../lib/telemetry'

export async function telemetry (cwd: string, command?: string) {
Expand All @@ -15,16 +16,18 @@ export async function telemetry (cwd: string, command?: string) {
enable opt-in to telemetry
reset resets your telemetry configuration (if any)
status show if telemetry is enabled, disabled or uninitialised
inform show an informed consent notice

For more details visit: https://keystonejs.com/telemetry
`

if (command === 'disable') return disableTelemetry()
if (command === 'enable') return enableTelemetry()
if (command === 'reset') return resetTelemetry()
if (command === 'status') return printTelemetryStatus()
if (command === 'status') return statusTelemetry()
if (command === 'inform') return informTelemetry()
if (command === '--help') {
console.log(`${chalk.bold('Keystone Telemetry')}`)
console.log(`${bold('Keystone Telemetry')}`)
console.log(usageText)
return
}
Expand Down
8 changes: 2 additions & 6 deletions packages/core/src/types/telemetry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ export type TelemetryVersion1 =
}
}

export type TelemetryVersion2and3 =
export type TelemetryVersion2 =
| undefined
| false
| {
Expand All @@ -26,10 +26,6 @@ export type TelemetryVersion2and3 =
}>
}

export type Configuration = {
telemetry?: undefined | false | TelemetryVersion2and3
}

export type Device = {
previous: string | null // new Date().toISOString().slice(0, 10)
os: string // `linux` | `darwin` | `windows` | ... // os.platform()
Expand All @@ -53,7 +49,7 @@ export type Project = {
// - `@keystone-6`
// - `@opensaas`
// - ...
versions: Partial<Record<PackageName, string>>
packages: Partial<Record<PackageName, string>>
lists: number
database: DatabaseProvider
// uses a new `field.__ksTelemetryFieldTypeName` for the key, defaults to `unknown`
Expand Down
Loading
Loading