Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: filter out index files temporarily #197

Closed
wants to merge 2 commits into from
Closed

Conversation

KSXGitHub
Copy link
Contributor

Changes in the npm registry cause unnecessary snapshot failures in our tests. The ideal solution is to implement a fake registry that we can control, but until then, filtering them out is good enough.

Copy link

github-actions bot commented Nov 13, 2023

Micro-Benchmark Results

Linux

group                          main                                   pr
-----                          ----                                   --
tarball/download_dependency    1.00      6.7±0.20ms   644.2 KB/sec    1.03      7.0±1.73ms   622.9 KB/sec

Copy link

codecov bot commented Nov 13, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (6b00165) 87.11% compared to head (d031d26) 87.04%.
Report is 1 commits behind head on main.

❗ Current head d031d26 differs from pull request most recent head 0e8bfe6. Consider uploading reports for the commit 0e8bfe6 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #197      +/-   ##
==========================================
- Coverage   87.11%   87.04%   -0.07%     
==========================================
  Files          56       56              
  Lines        2824     2825       +1     
==========================================
- Hits         2460     2459       -1     
- Misses        364      366       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

github-actions bot commented Nov 13, 2023

Integrated-Benchmark Report (Linux)

Scenario: Frozen Lockfile

Command Mean [ms] Min [ms] Max [ms] Relative
pacquet@HEAD 138.4 ± 8.4 123.3 147.8 1.05 ± 0.09
pacquet@main 132.3 ± 7.2 123.4 145.5 1.00
BENCHMARK_REPORT.json
{
  "results": [
    {
      "command": "pacquet@HEAD",
      "mean": 0.1384241537542857,
      "stddev": 0.008414157122340154,
      "median": 0.13771208354,
      "user": 0.05704961714285715,
      "system": 0.20423031999999997,
      "min": 0.12327296254000002,
      "max": 0.14780967154,
      "times": [
        0.14737982154,
        0.14552210354,
        0.13390590354,
        0.14518258154,
        0.13540762154,
        0.13246162254,
        0.14723337954,
        0.13342041354,
        0.14634528454,
        0.13896461154,
        0.14780967154,
        0.12327296254000002,
        0.13645955554,
        0.12457261954
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0
      ]
    },
    {
      "command": "pacquet@main",
      "mean": 0.13227419422750003,
      "stddev": 0.0071780807654990625,
      "median": 0.13035875654,
      "user": 0.06165876,
      "system": 0.20150538250000002,
      "min": 0.12335731054000001,
      "max": 0.14548627454,
      "times": [
        0.12544221254,
        0.13664241354,
        0.12335731054000001,
        0.14035412653999998,
        0.12445625754000002,
        0.12903680554,
        0.12357289854000002,
        0.14548627454,
        0.12660023154,
        0.13906317354,
        0.13065172953999998,
        0.13006578354,
        0.14343481353999998,
        0.13496635854,
        0.12831084154,
        0.13494587654
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0
      ]
    }
  ]
}

@zkochan
Copy link
Member

zkochan commented Nov 13, 2023

How does this mitigate the issue? The snapshot will be invalid if any of the files from the package will change. Or a package will get a new file.

@KSXGitHub
Copy link
Contributor Author

How does this mitigate the issue?

Create an actual mock registry that we have full control over. Or maybe just a verdaccio proxy is sufficient?

The snapshot will be invalid if any of the files from the package will change.

Right now, the package versions are pinned. This does mean that we cannot test something involving version ranges (such as pacquet add foo@latest), so eventually, we would have to properly solve the problem.

@zkochan
Copy link
Member

zkochan commented Nov 14, 2023

For pnpm we use a verdaccio proxy with prepublished packages for testing: https://github.com/pnpm/registry-mock

I think we should use the same in this repo.

package.json Outdated
@@ -0,0 +1,5 @@
{
"devDependencies": {
"@pnpm/registry-mock": "^3.16.0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see it being used anywhere. Should it be started by the user?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed it to wrong branch by mistake.

@KSXGitHub KSXGitHub marked this pull request as draft November 15, 2023 00:54
@KSXGitHub KSXGitHub mentioned this pull request Nov 15, 2023
@zkochan zkochan closed this in #198 Nov 17, 2023
@KSXGitHub KSXGitHub deleted the test-fix-snapshot branch November 17, 2023 13:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants