Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade runner for type-tests #73

Merged
merged 12 commits into from
Jul 31, 2024
Merged

Conversation

SleeplessByte
Copy link
Member

@SleeplessByte SleeplessByte commented Jul 31, 2024

This upgrades the test runner output to look like this:

╔═════════════════════════════════════════════════════════════╗
  🔧 Process input arguments for run
╚═════════════════════════════════════════════════════════════╝

✔️  Using reporter : C:/Users/Derk-Jan/Documents/GitHub/exercism/typescript-test-runner/dist/reporter.js
✔️  Using test-root: C:/Users/Derk-Jan/Documents/GitHub/exercism/typescript-test-runner/test/fixtures/pythagorean-triplet/chroth7/
✔️  Using base-root: C:/Users/Derk-Jan/Documents/GitHub/exercism/typescript-test-runner
✔️  Using setup-env: C:/Users/Derk-Jan/Documents/GitHub/exercism/typescript-test-runner/dist/jest/setup.js

╔═════════════════════════════════════════════════════════════╗
  🔧 Preparing run
╚═════════════════════════════════════════════════════════════╝

Input does not match output directory.
👁️  C:/Users/Derk-Jan/AppData/Local/Temp/foo-5qmJv9/
✔️  Copying C:/Users/Derk-Jan/Documents/GitHub/exercism/typescript-test-runner/test/fixtures/pythagorean-triplet/chroth7/ to output.

If the solution contains babel.config.js, package.json,
or tsconfig.json at the root, these configuration files will
be used during the test-runner process which we do not want.
The test-runner will therefore temporarily rename these files.

✔️  renaming babel.config.js in output so it can be replaced.
✔️  renaming package.json in output so it can be replaced.
✔️  renaming tsconfig.json in output so it can be replaced.

The output directory is likely not placed inside the test
runner root. This means the CLI tools need configuration
files as given and understood by the test-runner for running
the tests. Will now turn the output directory into a
standalone package.

✔️  .yarn cache from root to output
✔️  .yarnrc.yml from root to output
✔️  .yarn.lock from root to output
✔️  .pnp.cjs from root to output
✔️  .pnp.loader.mjs from root to output
✔️  babel.config.js from root to output
✔️  package.json from root to output
✔️  tsconfig.json from root to output

The results of this run will be written to 'results.json'.
👁️  C:/Users/Derk-Jan/AppData/Local/Temp/foo-5qmJv9/results.json

╔═════════════════════════════════════════════════════════════╗
  🔧 Preparing test suite file(s)
╚═════════════════════════════════════════════════════════════╝

There is a configuration file in the expected .meta location
which will now be used to determine which test files to prep.
👁️  C:/Users/Derk-Jan/Documents/GitHub/exercism/typescript-test-runner/test/fixtures/pythagorean-triplet/chroth7/.meta/config.json

Enabling tests in C:/Users/Derk-Jan/AppData/Local/Temp/foo-5qmJv9/pythagorean-triplet.test.ts

╔═════════════════════════════════════════════════════════════╗
  🔧 Preparing test project
╚═════════════════════════════════════════════════════════════╝

✔️  enabling corepack
✔️  yarn version now: 4.3.1

✔️  standalone package found installing packages from cache

total 3776
drwxr-xr-x 1 197612 197612      0 Jul 31 20:30 .
drwxr-xr-x 1 197612 197612      0 Jul 31 20:30 ..
drwxr-xr-x 1 197612 197612      0 Jul 31 20:30 .docs
-rw-r--r-- 1 197612 197612    132 Jul 31 20:30 .eslintignore
-rw-r--r-- 1 197612 197612    518 Jul 31 20:30 .eslintrc
drwxr-xr-x 1 197612 197612      0 Jul 31 20:30 .meta
-rwxr-xr-x 1 197612 197612 884757 Jul 31 20:30 .pnp.cjs
-rw-r--r-- 1 197612 197612  73628 Jul 31 20:30 .pnp.loader.mjs
drwxr-xr-x 1 197612 197612      0 Jul 31 20:30 .yarn
-rw-r--r-- 1 197612 197612    235 Jul 31 20:30 .yarnrc.yml
-rw-r--r-- 1 197612 197612    257 Jul 31 20:30 babel.config.js
-rw-r--r-- 1 197612 197612    374 Jul 31 20:30 babel.config.js.💥.bak
-rw-r--r-- 1 197612 197612   2332 Jul 31 20:30 expected_results.json
-rw-r--r-- 1 197612 197612    407 Jul 31 20:30 jest.config.js
-rw-r--r-- 1 197612 197612   2140 Jul 31 20:30 package.json
-rw-r--r-- 1 197612 197612    992 Jul 31 20:30 package.json.💥.bak
-rw-r--r-- 1 197612 197612   1902 Jul 31 20:30 pythagorean-triplet.test.ts
-rw-r--r-- 1 197612 197612   1461 Jul 31 20:30 pythagorean-triplet.ts
-rw-r--r-- 1 197612 197612   2332 Jul 31 20:30 results.json
-rw-r--r-- 1 197612 197612    111 Jul 31 20:30 tsconfig.json
-rw-r--r-- 1 197612 197612    487 Jul 31 20:30 tsconfig.json.💥.bak
-rw-r--r-- 1 197612 197612 249470 Jul 31 20:30 yarn.lock

➤ YN0000: · Yarn 4.3.1
➤ YN0000: ┌ Project validation
➤ YN0090: │ Offline work is enabled; Yarn won't fetch packages from the remote registry if it can avoid it
➤ YN0000: └ Completed
➤ YN0000: ┌ Resolution step
➤ YN0000: └ Completed in 0s 496ms
➤ YN0000: ┌ Post-resolution validation
➤ YN0060: │ eslint is listed by your project with version 9.8.0 (pc52f3), which doesn't satisfy what @exercism/eslint-config-tooling and other dependencies request (but they have non-overlapping ranges!).
➤ YN0002: │ @exercism/typescript-test-runner@workspace:. doesn't provide @babel/core (p9b671), requested by babel-jest.
➤ YN0086: │ Some peer dependencies are incorrectly met by your project; run yarn explain peer-requirements <hash> for details, where <hash> is the six-letter p-prefixed code.
➤ YN0086: │ Some peer dependencies are incorrectly met by dependencies; run yarn explain peer-requirements for details.
➤ YN0000: └ Completed
➤ YN0000: ┌ Fetch step
➤ YN0000: └ Completed
➤ YN0000: ┌ Link step
➤ YN0000: │ ESM support for PnP uses the experimental loader API and is therefore experimental
➤ YN0008: │ core-js@npm:3.37.1 must be rebuilt because its dependency tree changed
➤ YN0000: └ Completed in 0s 219ms
➤ YN0000: · Done with warnings in 0s 927ms

╔═════════════════════════════════════════════════════════════╗
  ➤  Step 1/3: Build (tests: does it compile?)                
╚═════════════════════════════════════════════════════════════╝

✔️  found a tsconfig.json (as expected). Re-configuring.
👁️  C:/Users/Derk-Jan/AppData/Local/Temp/foo-5qmJv9/tsconfig.json
{
  "extends": "@tsconfig/node20/tsconfig.json",
  "// include": ["src"],
  "exclude": ["test", "node_modules", ".meta/*", "__typetests__/*", "*.test.ts", "*.tst.ts"]
}

⚙️  corepack yarn run tsc

✔️  found a tsconfig.json (as expected). Restoring.

✅ tsc compilation success

╔═════════════════════════════════════════════════════════════╗
  ➤  Step 2/3: Type tests (tests: are the types as expected?) 
╚═════════════════════════════════════════════════════════════╝

✅ no type tests (*.tst.ts) discovered.

╔═════════════════════════════════════════════════════════════╗
  ➤  Step 3/3: Execution (tests: does the solution work?)     
╚═════════════════════════════════════════════════════════════╝

✔️  jest tests (*.test.ts) discovered.
C:\Users\Derk-Jan\AppData\Local\Temp\foo-5qmJv9\pythagorean-triplet.test.ts

⚙️  corepack yarn jest <...>

✅ all tests (*.test.ts) passed.

If the solution previously contained configuration files,
they were disabled and now need to be restored.

✔️  restoring babel.config.js in output
✔️  restoring package.json in output
✔️  restoring tsconfig.json in output

---------------------------------------------------------------
The results of this run have been written to 'results.json'.
👁️  C:/Users/Derk-Jan/AppData/Local/Temp/foo-5qmJv9/results.json
  • Dedicated steps for each thing we test for
  • Dedicated sections for restoring solutions
  • Standardised formatting for the log
  • Standardised handling of overwriting config
  • Initial support for type tests
  • Support for optional tests (explicit definition)

Example tstyche output for now (version 1):

{
  "version": 1,
  "status": "error",
  "message": "The submitted code did compile but at least one of the type-tests failed. We have collected the failing test encountered. At this moment the error messages are not very read-friendly, but it's a start. We are working on a more helpful output.\n-------------------------------\n\u001b[31mError: \u001b[0mExpression type did not raise a type error. \u001b[90m 9\u001b[0m\u001b[90m | \u001b[0m \u001b[90m 10\u001b[0m\u001b[90m | \u001b[0m // @ts-expect-error \u001b[31m 11\u001b[0m\u001b[90m | \u001b[0m expect(firstItem()).type.toRaiseError(\"Expected 1 argument\"); \u001b[90m | \u001b[0m \u001b[31m~~~~~~~~~~~~\u001b[0m \u001b[90m 12\u001b[0m\u001b[90m | \u001b[0m}); \u001b[90m 13\u001b[0m\u001b[90m | \u001b[0m \u001b[90m 14\u001b[0m\u001b[90m | \u001b[0mfunction secondItem<T>(target: Array<T>): T | undefined { \u001b[90m at \u001b[0m\u001b[36m./__typetests__/docs.tst.ts\u001b[0m\u001b[90m:11:28\u001b[0m\u001b[90m ❭ first item requires a parameter\u001b[0m\n"
}

Which should render as:

The submitted code did compile but at least one of the type-tests failed. We have collected the failing test encountered. At this moment the error messages are not very read-friendly, but it's a start. We are working on a more helpful output.
-------------------------------
�[31mError: �[0mExpression type did not raise a type error.

�[90m   9�[0m�[90m | �[0m
�[90m  10�[0m�[90m | �[0m  // @ts-expect-error
�[31m  11�[0m�[90m | �[0m  expect(firstItem()).type.toRaiseError("Expected 1 argument");
    �[90m | �[0m                           �[31m~~~~~~~~~~~~�[0m
�[90m  12�[0m�[90m | �[0m});
�[90m  13�[0m�[90m | �[0m
�[90m  14�[0m�[90m | �[0mfunction secondItem<T>(target: Array<T>): T | undefined {

      �[90m at �[0m�[36m./__typetests__/docs.tst.ts�[0m�[90m:11:28�[0m�[90m ❭ first item requires a parameter�[0m

Or rather:

Render of the ansi color codes of the previous codeblock

@SleeplessByte SleeplessByte changed the title Upgrade runner for type-tests wip Upgrade runner for type-tests Jul 31, 2024
@SleeplessByte SleeplessByte marked this pull request as ready for review July 31, 2024 20:47
@SleeplessByte SleeplessByte requested a review from a team as a code owner July 31, 2024 20:47
Copy link
Member Author

@SleeplessByte SleeplessByte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh fun I can self-merge

@SleeplessByte SleeplessByte merged commit bb4d160 into main Jul 31, 2024
4 checks passed
@SleeplessByte SleeplessByte deleted the feature/type-tests-and-more branch July 31, 2024 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant