Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

run node tests in github actions #1375

Closed
wants to merge 103 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
103 commits
Select commit Hold shift + click to select a range
97e046d
run node tests in github actions
rwblair Oct 26, 2021
0470055
fix yml syntax error
rwblair Oct 26, 2021
9c4a1a7
fix yml syntax error
rwblair Oct 26, 2021
3bcc8be
use node action to actually setup node
rwblair Oct 26, 2021
77f1cea
try __dirname based join to find testdata instead of relying on proce…
rwblair Oct 26, 2021
3e0a391
shut eslint up
rwblair Oct 26, 2021
677c9f9
Revert "shut eslint up"
rwblair Oct 26, 2021
21fde2d
Revert "try __dirname based join to find testdata instead of relying …
rwblair Oct 26, 2021
3792f1f
Merge branch 'master' of github.com:bids-standard/bids-validator into…
rwblair Jan 7, 2022
0a48622
add --colors to jest call
rwblair Jan 7, 2022
56614cd
Revert "add --colors to jest call"
rwblair Jan 7, 2022
50a11d9
Revert "Revert "add --colors to jest call""
rwblair Jan 7, 2022
24150f4
will --colors work for windows gitlab?
rwblair Jan 7, 2022
59a93d8
Revert "will --colors work for windows gitlab?"
rwblair Jan 7, 2022
93bcebb
putting command after ./ in package.json lint call might allow it to …
rwblair Jan 7, 2022
b512edc
Revert "putting command after ./ in package.json lint call might allo…
rwblair Jan 7, 2022
703f336
remove relative paths from npm scripts
rwblair Jan 7, 2022
a8add55
add eslint ignore and update eslint call to use config file
rwblair Jan 7, 2022
bb83792
eslint/prettier fixes
rwblair Jan 7, 2022
0d67e6a
move eslintrc to root of project
rwblair Jan 7, 2022
d1d6ad8
set no auto clrf for git/actions to fix failing windows test
rwblair Jan 7, 2022
5ca5d73
add missing lf..
rwblair Jan 7, 2022
e7bd0df
Added the microscopy Regex parts in the rules and added the function …
etiennebergeron Jul 28, 2021
a8d5eca
Added the different microscopy modality suffixes
etiennebergeron Jul 28, 2021
c3ff2fb
Add back the enter removed by accident in phenotypic_rules.json
etiennebergeron Jul 28, 2021
c9ea85c
Created the file with the required metadata fields microscopy.json an…
etiennebergeron Jul 29, 2021
23f3c1c
JSON field 'AcquisitionParameters' is now 'OtherAcquisitionParameters'
etiennebergeron Jul 30, 2021
67fb84a
Added error codes 214 and 215 for missing samples files. Added the fi…
etiennebergeron Aug 3, 2021
5f1e2ee
Corrected a typo between the 'pet' and 'microscopy' paths.
etiennebergeron Aug 4, 2021
2132b88
Codes '216' and '217' added for 'PixelSizeUnits' and a missing recomm…
etiennebergeron Aug 4, 2021
a82a847
Removed code 214 since 'samples.json' is only recommended if samples.…
etiennebergeron Aug 4, 2021
28c9d44
Modified 'checkSamples.js' to check 'samples.tsv' instead of JSON. Mi…
etiennebergeron Aug 4, 2021
8fdd70b
Modified again to check 'samples.tsv'.
etiennebergeron Aug 5, 2021
9db3466
The checkSamples method was working with 'samples.json', but not with…
etiennebergeron Aug 5, 2021
cb2c396
Didn't succeed to figure out how to detect samples.tsv. I restored th…
etiennebergeron Aug 5, 2021
ac1922c
Removed enters in fullTest.js.
etiennebergeron Aug 5, 2021
770f8c9
Removed the warning for RECOMMENDED JSON fields missing. Doesn't seem…
etiennebergeron Aug 5, 2021
a6be30d
Update bids-validator/utils/issues/list.js
etiennebergeron Aug 6, 2021
73f76ca
Update bids-validator/validators/json/json.js
etiennebergeron Aug 6, 2021
51fa457
Changed the suffixes to uppercase in the rules and 'json.js'.
etiennebergeron Aug 6, 2021
78d2d59
The entity 'sample' is REQUIRED in microscopy, so modified the regex …
etiennebergeron Aug 6, 2021
9dcafaa
Added a new 'checkSamples.js' file, but this time based on 'checkRead…
etiennebergeron Aug 9, 2021
9e13c3f
Removed unnecessary import.
etiennebergeron Aug 9, 2021
c10ab22
Added the rules related to the 'samples.tsv' file in 'tsv.js'. Added …
etiennebergeron Aug 11, 2021
0c1414e
The PixelSizeUnits now needs to be in um. Currently using the existin…
etiennebergeron Aug 12, 2021
d1fe83c
Added a dictionary 'microscopy_json' in the 'file_level_rules.json' t…
etiennebergeron Sep 16, 2021
4a864dd
Added the Regex rule for a file named _photo.jpg. Added the JSON fiel…
etiennebergeron Sep 19, 2021
43d9a8b
Entities order modified; <chunk> at the end.
etiennebergeron Sep 28, 2021
23215c8
Modified tsv.js for samples.tsv ; now two different subjects can have a
etiennebergeron Oct 13, 2021
69c0950
Completed the JSON fields ('TissueDeformationScaling' and 'SampleEnvi…
etiennebergeron Oct 21, 2021
61c98ac
Resolve some issues that mentioned in BEP031 pr changes document
konantian Dec 5, 2021
da45e7b
Resolve comments from PR
konantian Dec 5, 2021
39543b7
Fix microscopy regexp
konantian Dec 6, 2021
e48f955
Remove "ChunkTransformationMatrix" from the recommended list
konantian Dec 7, 2021
fbc6fc7
Add regexp for _photo ext in microscopy
konantian Dec 8, 2021
6c27e34
Revert changes
konantian Dec 8, 2021
270906e
1. Add Check for consistency for PixelSize between JSON and OME-TIFF …
konantian Dec 20, 2021
85b9ef9
Resolve comments
konantian Dec 21, 2021
439f5c1
Resolve comments
konantian Dec 23, 2021
831e941
Resolve comments
konantian Dec 23, 2021
91d52a5
Resolve comments
konantian Dec 24, 2021
40d01e1
Rename folder ometiff to microscopy
konantian Dec 24, 2021
725a7f0
1. Update regex rules for microscopy
konantian Dec 27, 2021
4ee0199
Update regex for matrix field check
konantian Dec 27, 2021
37de6df
Bug fix
konantian Dec 28, 2021
8198449
Update the error message when sample type is not valid
konantian Dec 29, 2021
5933078
1. Add validation for tiff and big tiff
konantian Jan 2, 2022
2ce47e2
Change an error to warning
konantian Jan 3, 2022
b7d18dc
add tests for checkJSONAndField
rwblair Jan 6, 2022
c9712e1
add tests for checkSample.js
rwblair Jan 6, 2022
4fc3c1a
add tests for microscopy validate.js and test data
rwblair Jan 6, 2022
9ce48d7
add param docs to readFile function
rwblair Jan 6, 2022
87ec79e
generate error if magic number is incorrect on tiff file
rwblair Jan 6, 2022
3a88e3e
move new dependencies into validator package.json
rwblair Jan 7, 2022
6f906e9
have micro json check use old sidecar generation function
rwblair Jan 7, 2022
8c9e1e1
Update validation for OME namespace
konantian Jan 6, 2022
af3c759
Fix code style
konantian Jan 7, 2022
7f4b5b2
rearrange logic for samples.tsv validation. Add tests.
rwblair Jan 7, 2022
a0e2e31
updated package-lock to reflect package.json changes.
rwblair Jan 7, 2022
c725d7c
have microscopy json check return empty list if no other issues found
rwblair Jan 11, 2022
8f4663c
set esbuild buffer size env in bin/bids-validator when we are using e…
rwblair Jan 12, 2022
b4c8972
added babel runtime to web package.json, maybe it was just my setup b…
rwblair Jan 12, 2022
1918f80
Add DatasetLinks to JSON schema (#1404)
sappelhoff Jan 20, 2022
cd627bb
Fix NO_AUTHORS evidence zero length array passing array type as evidence
nellh Feb 10, 2022
47a3bcd
add option to throw an error if specified modalities are detected
rwblair Feb 10, 2022
e0f2f92
eslint --fix
rwblair Feb 10, 2022
7ad713a
update consoleFormat spec to not check for undefined file in output.
rwblair Feb 10, 2022
a8e582e
treat ignored files in node readDir like we do in browser. Call modal…
rwblair Feb 11, 2022
cf26b68
revert changes to gitTree ignore check in readDir.js
rwblair Feb 11, 2022
b48b1ab
v1.9.0
rwblair Feb 15, 2022
c828499
v1.9.1-dev.0
rwblair Feb 15, 2022
2f46cb7
Bump nanoid from 3.1.25 to 3.3.0 (#1421)
dependabot[bot] Feb 16, 2022
f804f49
add prepublish step to root package.json
rwblair Feb 16, 2022
c07b264
switch prepublish scripts to prepublishOnly, prepublish deprectaed in…
rwblair Feb 16, 2022
2a9d30f
v1.9.1
rwblair Feb 16, 2022
c1aa9f6
v1.9.2
rwblair Feb 16, 2022
87c2402
v1.9.3-dev.0
rwblair Feb 16, 2022
98917df
remove relative paths from npm scripts
rwblair Jan 7, 2022
1e9936b
fix errant import from rebase?
rwblair Mar 15, 2022
1f08ad9
Merge remote-tracking branch 'origin/master' into ci/github_actions
rwblair Mar 15, 2022
0b43f96
fix lint issues. fix call to linter for linux in package.json
rwblair Mar 15, 2022
b83c04d
porperty -> property
rwblair Mar 15, 2022
e5e9250
does windows actually care about single vs double quotes?
rwblair Mar 15, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .eslintignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
node_modules/
dist/
File renamed without changes.
48 changes: 48 additions & 0 deletions .github/workflows/node_tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: Node Tests

on:
push:
branches: [master]
pull_request:
branches: [master]

jobs:
build:
strategy:
fail-fast: false
matrix:
platform: [ubuntu-latest, macos-latest, windows-latest]
node: [14]

runs-on: ${{ matrix.platform }}

steps:
- name: Set git to use LF
run: |
git config --global core.autocrlf false
git config --global core.eol lf
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: "14"
cache: "npm"
- name: Set git name/email
run: |
git config --global user.email "bids.maintenance@gmail.com"
git config --global user.name "bids-maintenance"
- name: Set up Node
run: |
npm install -g npm@^7
npm install
- name: Eslint
run: npm run lint
- name: Get bids-examples data
run: git submodule update --init
- name: Jest tests
run: npm run coverage -- --maxWorkers=2 --testTimeout=10000 --colors
- name: Upload to codecov
run: npm run codecov
- name: Smoke tests
run: |
bids-validator/bin/bids-validator bids-validator/tests/data/valid_headers/ --ignoreNiftiHeaders
bids-validator/bin/bids-validator bids-validator/tests/data/valid_headers/ --ignoreNiftiHeaders --json
4 changes: 1 addition & 3 deletions bids-validator/bids_validator/rules/top_level_rules.json
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,7 @@
"pet_top": {
"regexp": "^[\\/\\\\](?:ses-[a-zA-Z0-9]+_)?(?:trc-[a-zA-Z0-9]+_)?(?:rec-[a-zA-Z0-9]+_)?(?:task-[a-zA-Z0-9]+_)?(?:run-[0-9]+_)?(@@@_pet_suffixes_@@@)\\.json$",
"tokens": {
"@@@_pet_suffixes_@@@": [
"pet"
]
"@@@_pet_suffixes_@@@": ["pet"]
}
},

Expand Down
6 changes: 3 additions & 3 deletions bids-validator/cli.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import colors from 'colors/safe'
import fs from 'fs'
import { filenamesOnly } from './utils/filenamesOnly.js'

const errorToString = (err) => {
const errorToString = err => {
if (err instanceof Error) return err.stack
else if (typeof err === 'object') return JSON.parse(err)
else return err
Expand Down Expand Up @@ -39,7 +39,7 @@ export function cli(argumentOverride) {
const argv = parseOptions(argumentOverride)
const dir = argv._[0]
const options = argv
process.on('unhandledRejection', (err) => {
process.on('unhandledRejection', err => {
console.log(
format.unexpectedError(
// eslint-disable-next-line
Expand All @@ -61,7 +61,7 @@ export function cli(argumentOverride) {
reject(2)
}

validate.BIDS(dir, options, function (issues, summary) {
validate.BIDS(dir, options, function(issues, summary) {
function resolveOrReject() {
if (
issues === 'Invalid' ||
Expand Down
1 change: 1 addition & 0 deletions bids-validator/utils/__tests__/filenamesOnly.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { validateFilenames } from '../filenamesOnly.js'

describe('test filenames mode', () => {
beforeEach(() => {
// eslint-disable-next-line
console.log = jest.fn()
})
it('throws an error when obviously non-BIDS input', async () => {
Expand Down
59 changes: 32 additions & 27 deletions bids-validator/utils/__tests__/gitTreeMode.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -87,36 +87,41 @@ describe('gitTreeMode functions', () => {
size: 1,
},
]
const expected = [
{
path: '/path/to/dataset/path/to/a',
size: 100,
relativePath: '/path/to/a',
name: 'a',
},
{
path: '/path/to/dataset/path/to/b',
size: 99,
relativePath: '/path/to/b',
name: 'b',
},
{
path: '/path/to/dataset/path/to/c',
size: 98,
relativePath: '/path/to/c',
name: 'c',
},
{
path: '/path/to/dataset/path/to/d',
size: 1,
relativePath: '/path/to/d',
name: 'd',
},
]
/* Not currently in use.
const expected = [
{
path: '/path/to/dataset/path/to/a',
size: 100,
relativePath: '/path/to/a',
name: 'a',
},
{
path: '/path/to/dataset/path/to/b',
size: 99,
relativePath: '/path/to/b',
name: 'b',
},
{
path: '/path/to/dataset/path/to/c',
size: 98,
relativePath: '/path/to/c',
name: 'c',
},
{
path: '/path/to/dataset/path/to/d',
size: 1,
relativePath: '/path/to/d',
name: 'd',
},
]
*/
const output = processFiles('/path/to/dataset', ig, filesA, filesB)
const fileNames = output.map(file => file.name)
assert(!fileNames.includes('.DS_Store'), 'filters out ignored files')
assert(!fileNames.includes('derivative_file'), 'filters out ignored directories',)
assert(
!fileNames.includes('derivative_file'),
'filters out ignored directories',
)
assert.deepEqual(fileNames, ['a', 'b', 'c', 'd'], 'aggregates files')
assert.isString(output[0].relativePath, 'adds relativePath to files')
assert.isString(output[1].relativePath, 'adds name to files')
Expand Down
15 changes: 13 additions & 2 deletions bids-validator/utils/__tests__/unit.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,11 @@ describe('unit validator', () => {
const goodOutput = unit.validate(validRoot)
expect(goodOutput.isValid).toBe(true)
})
const invalidRoots = ['definitielynotavalidroot', `%/${validRoot}`, `n/a*${validRoot}`]
const invalidRoots = [
'definitielynotavalidroot',
`%/${validRoot}`,
`n/a*${validRoot}`,
]
invalidRoots.forEach(invalidRoot => {
const badOutput = unit.validate(invalidRoot)
expect(badOutput.isValid).toBe(false)
Expand All @@ -25,7 +29,14 @@ describe('unit validator', () => {
expect(badOutput.isValid).toBe(false)
})

const validExponents = ['^2', '^543', '¹²³', ...unit.superscriptNumbers.slice(0, 3), '^-2', '⁻³']
const validExponents = [
'^2',
'^543',
'¹²³',
...unit.superscriptNumbers.slice(0, 3),
'^-2',
'⁻³',
]
it('handles simple units with exponents', () => {
validExponents.forEach(exp => {
const goodOutput = unit.validate(validRoot + exp)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ describe('readDir.js - examples integration', () => {
'sub-02_task-mixedeventrelatedprobe_run-01_bold.nii.gz',
)
expect(filenames[200]).toBe(
'sub-14_task-probabilisticclassification_run-02_events.tsv'
'sub-14_task-probabilisticclassification_run-02_events.tsv',
)
done()
})
Expand Down
4 changes: 3 additions & 1 deletion bids-validator/utils/files/generateMergedSidecarDict.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,9 @@ function generateMergedSidecarDict(potentialSidecars, jsonContents) {
const jsonObject = jsonContents[sidecarName]
if (jsonObject) {
for (var key in jsonObject) {
mergedDictionary[key] = jsonObject[key]
if (jsonObject.hasOwnProperty(key)) {
mergedDictionary[key] = jsonObject[key]
}
}
} else if (jsonObject === null) {
mergedDictionary.invalid = true
Expand Down
14 changes: 9 additions & 5 deletions bids-validator/utils/files/readDir.js
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ async function preprocessNode(dir, ig, options) {
* @returns {string[]}
*/
const getGitLsTree = (cwd, gitRef) =>
new Promise((resolve, reject) => {
new Promise(resolve => {
let output = ''
const gitProcess = child_proccess.spawn(
'git',
Expand Down Expand Up @@ -181,10 +181,14 @@ const readLsTreeLines = gitTreeLines =>
const getGitCatFile = (cwd, input) =>
new Promise(resolve => {
let output = ''
const gitProcess = spawn('git', ['cat-file', '--batch', '--buffer'], {
cwd,
encoding: 'utf-8',
})
const gitProcess = child_proccess.spawn(
'git',
['cat-file', '--batch', '--buffer'],
{
cwd,
encoding: 'utf-8',
},
)

// pass in symlink objects
gitProcess.stdin.write(input)
Expand Down
3 changes: 3 additions & 0 deletions bids-validator/utils/issues/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,9 @@ var issues = {

// organize by severity
for (const codePropertyName in categorized) {
if (!categorized.hasOwnProperty(codePropertyName)) {
continue
}
// Properties are always strings but error codes are always integers
const code = parseInt(codePropertyName)
issue = categorized[code]
Expand Down
3 changes: 2 additions & 1 deletion bids-validator/utils/issues/list.js
Original file line number Diff line number Diff line change
Expand Up @@ -712,7 +712,8 @@ export default {
139: {
key: 'BLACKLISTED_MODALITY',
severity: 'error',
reason: 'Found a modality that has been blacklisted through validator configuration.',
reason:
'Found a modality that has been blacklisted through validator configuration.',
},
140: {
key: '140_EMPTY',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,12 @@ const CRLFParticipantsTsv = 'participant_id\tsex\tage\r\nsub-01\tM\t25\r\n'

describe('collectSubjectMetadata()', () => {
it('handles Windows newline characters in column row', () => {
expect(collectSubjectMetadata(CRLFParticipantsTsv)).toEqual([{
age: 25,
participantId: '01',
sex: 'M',
}])
expect(collectSubjectMetadata(CRLFParticipantsTsv)).toEqual([
{
age: 25,
participantId: '01',
sex: 'M',
},
])
})
})
2 changes: 1 addition & 1 deletion bids-validator/utils/summary/collectSessions.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import type from '../type'

const collectSessions = (fileList, options) => {
const collectSessions = fileList => {
const sessions = []
Object.keys(fileList).forEach(key => {
const file = fileList[key]
Expand Down
2 changes: 1 addition & 1 deletion bids-validator/utils/summary/collectSubjects.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import type from '../type'

const collectSubjects = (fileList, options) => {
const collectSubjects = fileList => {
const subjects = []
const fileKeys = Object.keys(fileList)
fileKeys.forEach(key => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,9 @@ describe('checkDatasetDescription', () => {
})
describe('checkGeneticDatabaseField', () => {
it('returns code 128 when there is no Genetics.Dataset with a genetic_info.json present', () => {
const invalidJsonContentsDict = {
'/dataset_description.json': { },
'/genetic_info.json': { }
const invalidJsonContentsDict = {
'/dataset_description.json': {},
'/genetic_info.json': {},
}
let issues = checkDatasetDescription(invalidJsonContentsDict)
assert(
Expand All @@ -71,19 +71,18 @@ describe('checkDatasetDescription', () => {
)
})
it('does not return code 128 when GeneticDataset field and genetic_info.json present', () => {
const validJsonContentsDict = {
const validJsonContentsDict = {
'/dataset_description.json': {
Authors: ['Benny', 'the Jets'],
Genetics: {Dataset: 'GeneticGeneticDataset'},
Genetics: { Dataset: 'GeneticGeneticDataset' },
},
'/genetic_info.json': { }
'/genetic_info.json': {},
}
let issues = checkDatasetDescription(validJsonContentsDict)
assert(
issues.findIndex(issue => issue.code === 128) === -1,
'issues does not include a code 128'
'issues does not include a code 128',
)
})

})
})
4 changes: 2 additions & 2 deletions bids-validator/validators/bids/fullTest.js
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,8 @@ const fullTest = (fileList, options, annexed, dir, schema, callback) => {
new Issue({
file: mod,
evidence: `found ${mod} files`,
code: 139
})
code: 139,
}),
)
}
})
Expand Down
2 changes: 1 addition & 1 deletion bids-validator/validators/bids/start.js
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ const start = (dir, options, callback) => {
console.log(`bids-specification@${options.schema}`)
}

utils.options.parse(dir, options, async function (issues, options) {
utils.options.parse(dir, options, async function(issues, options) {
if (issues && issues.length > 0) {
// option parsing issues
callback({ config: issues })
Expand Down
2 changes: 1 addition & 1 deletion bids-validator/validators/json/json.js
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ const selectSchema = file => {
file.name.endsWith('NLO.json') ||
file.name.endsWith('OCT.json') ||
file.name.endsWith('SPIM.json')
) {
) {
schema = require('./schemas/microscopy.json')
} else if (
file.relativePath.includes('/meg/') &&
Expand Down
Loading