fix: ttl and caching for ipns urls #34

2color · 2024-03-26T14:00:23Z

Description

This PR mainly fixes the TLRU cache behaviour by ensuring that when setting DNSLink and IPNS resolution answers to the cache, we correctly convert the TTL from seconds to milliseconds as expected by the TLRU cache

In addition:

Adds tests to ensure that the correct TTL is returned by the parseUrlString function.
Converts the IPNS record TTL from nanoseconds to seconds correctly.

Notes & open questions

This should fix some of the duplicate resolutions reported in Bug: DoH client is missing in-memory cache service-worker-gateway#59. However, it should
We still need to follow up in js-ipns regarding why nanoseconds are incorrectly calculated when converting from ms.

Change checklist

I have performed a self-review of my own code
I have made corresponding changes to the documentation if necessary (this includes comments as well)
I have added tests that prove my fix is effective or that my feature works

also add trace logging

packages/verified-fetch/test/utils/parse-url-string.spec.ts

packages/verified-fetch/src/utils/parse-url-string.ts

packages/verified-fetch/test/cache-control-header.spec.ts

packages/verified-fetch/test/utils/parse-url-string.spec.ts

Co-authored-by: Russell Dempsey <1173416+SgtPooki@users.noreply.github.com>

## @helia/verified-fetch [1.3.3](https://github.com/ipfs/helia-verified-fetch/compare/@helia/verified-fetch-1.3.2...@helia/verified-fetch-1.3.3) (2024-03-28) ### Bug Fixes * ttl and caching for ipns urls ([#34](#34)) ([44ac5a1](44ac5a1))

github-actions · 2024-03-28T12:55:29Z

🎉 This PR is included in version 1.3.3 🎉

The release is available on npm package (@latest dist-tag)

Your semantic-release bot 📦🚀

github-actions · 2024-03-28T12:55:43Z

🎉 This PR is included in version 1.11.0 🎉

The release is available on npm package (@latest dist-tag)

Your semantic-release bot 📦🚀

github-actions · 2024-05-09T21:51:20Z

🎉 This PR is included in version 1.0.0 🎉

The release is available on npm package (@latest dist-tag)

Your semantic-release bot 📦🚀

## 1.0.0 (2024-10-14) ### ⚠ BREAKING CHANGES * upgrade to helia v5 (#107) ### Features * abort signals are respected ([#26](#26)) ([30148fe](30148fe)) * allow use as a singleton ([#413](https://github.com/ipfs/helia-verified-fetch/issues/413)) ([7cbeed0](7cbeed0)) * create @helia/verified-fetch ([#392](https://github.com/ipfs/helia-verified-fetch/issues/392)) ([06d36fe](06d36fe)) * customize ipns dnsResolvers ([#445](https://github.com/ipfs/helia-verified-fetch/issues/445)) ([8f60822](8f60822)) * download tars from @helia/verified-fetch ([#442](https://github.com/ipfs/helia-verified-fetch/issues/442)) ([70ddd00](70ddd00)) * require content-type parser to set content-type ([#423](https://github.com/ipfs/helia-verified-fetch/issues/423)) ([b39d07c](b39d07c)) * support `Accept` header in `@helia/verified-fetch` ([#438](https://github.com/ipfs/helia-verified-fetch/issues/438)) ([54c4383](54c4383)) * support downloading car files from @helia/verified-fetch ([#441](https://github.com/ipfs/helia-verified-fetch/issues/441)) ([703980c](703980c)) * support http range header ([#10](#10)) ([9f5078a](9f5078a)) * support IPFS/IPNS paths, Gateways, etc ([#4](#4)) ([e7f1816](e7f1816)) * support redirects for UnixFS directories ([#5](#5)) ([4601d46](4601d46)) * support requesting raw IPNS records in @helia/verified-fetch ([#443](https://github.com/ipfs/helia-verified-fetch/issues/443)) ([e92086a](e92086a)) * upgrade to helia v5 ([#107](#107)) ([91a6473](91a6473)) * use blockstore sessions ([#50](#50)) ([541dd64](541dd64)) * use the waterworks trustless gateway by default ([#94](#94)) ([2db8d1e](2db8d1e)) ### Bug Fixes * @helia/verified-fetch init args are optional ([#412](https://github.com/ipfs/helia-verified-fetch/issues/412)) ([e6d41a0](e6d41a0)) * aborted signals throw libp2p AbortError ([#30](#30)) ([4575791](4575791)) * add interop tests and update project config ([fdc83b8](fdc83b8)) * allow contentTypeParser with Helia instance ([#427](https://github.com/ipfs/helia-verified-fetch/issues/427)) ([b5c95e6](b5c95e6)) * append query path to path resolved from IPNS name ([#3](#3)) ([fd86e6a](fd86e6a)) * byte range request end should never equal file size ([#24](#24)) ([aafc567](aafc567)) * content-type response header hints how to process response ([#426](https://github.com/ipfs/helia-verified-fetch/issues/426)) ([8b78e79](8b78e79)) * decodeURIComponent on path parts ([#40](#40)) ([f628cf6](f628cf6)) * do not coerce `undefined` to `null` for JSON serialization ([#2](#2)) ([d36ce29](d36ce29)) * gateway conformance improvements ([#85](#85)) ([7281078](7281078)) * gateway conformance tests ([#81](#81)) ([d0a3b6b](d0a3b6b)) * identity CIDs use contentTypeParser ([#49](#49)) ([3014498](3014498)) * implicit accept header can be overridden by format query ([#36](#36)) ([75c0b75](75c0b75)) * missing paths returns 404 instead of 502 ([#59](#59)) ([291a054](291a054)), closes [#53](#53) * pass url and body to badRequestResponse ([#44](#44)) ([cc228e6](cc228e6)) * prevent interop timeouts with fast fixture loading ([#73](#73)) ([a43d994](a43d994)) * reduce dagPb and dagCbor handler complexity ([#45](#45)) ([3b41752](3b41752)) * remove redundant abort controller ([#41](#41)) ([04b220d](04b220d)) * remove stubs from verified-fetch tests ([#431](https://github.com/ipfs/helia-verified-fetch/issues/431)) ([cbab8b0](cbab8b0)) * set cache-control header correctly ([#19](#19)) ([114f3a4](114f3a4)), closes [#17](#17) [#23](#23) [#10](#10) * support https?://<dnsLink>.ipns.<gateway> urls ([#16](#16)) ([0ece19a](0ece19a)) * ttl and caching for ipns urls ([#34](#34)) ([44ac5a1](44ac5a1)) * unixfs dir redirect ([#33](#33)) ([32ca87f](32ca87f)) * update @helia/ipns and dns config ([#18](#18)) ([9f88c54](9f88c54)) * use CID object for verified-fetch progress events ([#425](https://github.com/ipfs/helia-verified-fetch/issues/425)) ([b010184](b010184)) * use ipfs-unixfs-exporter directly ([#42](#42)) ([4532bf1](4532bf1)) * verified-fetch etag header ([#434](https://github.com/ipfs/helia-verified-fetch/issues/434)) ([f9b1ffe](f9b1ffe)) * verified-fetch header cache-control typo ([#433](https://github.com/ipfs/helia-verified-fetch/issues/433)) ([ea39b48](ea39b48)) * walking dag-cbor paths ([#39](#39)) ([99668ce](99668ce)) * X-Ipfs-Path is set correctly ([#46](#46)) ([5bb6685](5bb6685)) ### Trivial Changes * add tests for directory redirects for gateways ([#15](#15)) ([269609d](269609d)) * fix build ([#22](#22)) ([01261fe](01261fe)) * initial import ([4b4fb8c](4b4fb8c)) * **release:** 1.0.0 [skip ci] ([ef5b6a5](ef5b6a5)), closes [#68](#68) * **release:** 1.0.0 [skip ci] ([a584a1a](a584a1a)) * **release:** 1.0.0 [skip ci] ([5207a64](5207a64)) * **release:** 1.0.1 [skip ci] ([5499724](5499724)), closes [#81](#81) * **release:** 1.0.1 [skip ci] ([8d2fdda](8d2fdda)), closes [#2](#2) * **release:** 1.0.2 [skip ci] ([104af2d](104af2d)), closes [#3](#3) * **release:** 1.1.0 [skip ci] ([e300af2](e300af2)) * **release:** 1.1.0 [skip ci] ([f4fcb4d](f4fcb4d)), closes [#4](#4) [#5](#5) [#6](#6) * **release:** 1.1.0 [skip ci] ([b9b959b](b9b959b)) * **release:** 1.1.1 [skip ci] ([8ff0b8b](8ff0b8b)), closes [#83](#83) * **release:** 1.1.1 [skip ci] ([6f2c6bd](6f2c6bd)), closes [#15](#15) [#14](#14) * **release:** 1.1.2 [skip ci] ([6161aba](6161aba)), closes [#85](#85) * **release:** 1.1.2 [skip ci] ([4e1abc0](4e1abc0)), closes [#16](#16) * **release:** 1.1.3 [skip ci] ([14c2302](14c2302)), closes [#18](#18) * **release:** 1.10.0 [skip ci] ([4589c26](4589c26)) * **release:** 1.10.1 [skip ci] ([140bab3](140bab3)), closes [#33](#33) * **release:** 1.11.0 [skip ci] ([39bb14e](39bb14e)) * **release:** 1.12.0 [skip ci] ([50682bf](50682bf)) * **release:** 1.13.0 [skip ci] ([e00d41c](e00d41c)) * **release:** 1.14.0 [skip ci] ([2521838](2521838)) * **release:** 1.15.0 [skip ci] ([a38a0c6](a38a0c6)) * **release:** 1.15.1 [skip ci] ([f59e862](f59e862)), closes [#47](#47) * **release:** 1.16.0 [skip ci] ([0a18c44](0a18c44)) * **release:** 1.17.0 [skip ci] ([f5299cf](f5299cf)) * **release:** 1.18.0 [skip ci] ([7771af2](7771af2)) * **release:** 1.19.0 [skip ci] ([089635d](089635d)) * **release:** 1.2.0 [skip ci] ([befd270](befd270)) * **release:** 1.2.0 [skip ci] ([e836abf](e836abf)), closes [#10](#10) [#22](#22) * **release:** 1.2.0 [skip ci] ([77d5e9e](77d5e9e)) * **release:** 1.2.1 [skip ci] ([718644b](718644b)), closes [#24](#24) * **release:** 1.20.0 [skip ci] ([abaaeab](abaaeab)) * **release:** 1.21.0 [skip ci] ([aeb5a89](aeb5a89)) * **release:** 1.21.1 [skip ci] ([f7ac2e7](f7ac2e7)), closes [#73](#73) * **release:** 1.22.0 [skip ci] ([6d4b26c](6d4b26c)), closes [#50](#50) * **release:** 1.23.0 [skip ci] ([754e219](754e219)) * **release:** 1.24.0 [skip ci] ([c1a795f](c1a795f)) * **release:** 1.24.1 [skip ci] ([9f8fb86](9f8fb86)), closes [#85](#85) [#54](#54) * **release:** 1.25.0 [skip ci] ([3d8bb78](3d8bb78)) * **release:** 1.3.0 [skip ci] ([43c41e4](43c41e4)), closes [#26](#26) [#17](#17) [#23](#23) [#10](#10) * **release:** 1.3.0 [skip ci] ([bf8af38](bf8af38)) * **release:** 1.3.1 [skip ci] ([0bf5847](0bf5847)), closes [#30](#30) * **release:** 1.3.10 [skip ci] ([6768fbd](6768fbd)), closes [#51](#51) * **release:** 1.3.11 [skip ci] ([8606b15](8606b15)) * **release:** 1.3.12 [skip ci] ([969a2e0](969a2e0)), closes [#42](#42) * **release:** 1.3.13 [skip ci] ([a180547](a180547)), closes [#53](#53) [#46](#46) * **release:** 1.3.14 [skip ci] ([31f238a](31f238a)), closes [#63](#63) * **release:** 1.3.2 [skip ci] ([2e141ca](2e141ca)), closes [#33](#33) * **release:** 1.3.3 [skip ci] ([19574cf](19574cf)), closes [#34](#34) * **release:** 1.3.4 [skip ci] ([7b06a56](7b06a56)), closes [#40](#40) * **release:** 1.3.5 [skip ci] ([554bb3e](554bb3e)), closes [#41](#41) * **release:** 1.3.6 [skip ci] ([489a1d6](489a1d6)), closes [#36](#36) * **release:** 1.3.7 [skip ci] ([d0114c1](d0114c1)), closes [#39](#39) * **release:** 1.3.8 [skip ci] ([0a37856](0a37856)), closes [#44](#44) [#47](#47) * **release:** 1.3.9 [skip ci] ([768e3ff](768e3ff)), closes [#49](#49) * **release:** 1.4.0 [skip ci] ([dab598a](dab598a)), closes [#50](#50) * **release:** 1.4.0 [skip ci] ([e008b20](e008b20)) * **release:** 1.4.1 [skip ci] ([e3c0a68](e3c0a68)), closes [#68](#68) [#72](#72) [#72](#72) * **release:** 1.4.2 [skip ci] ([2633d31](2633d31)), closes [#45](#45) * **release:** 1.4.3 [skip ci] ([924a6df](924a6df)), closes [#85](#85) * **release:** 1.5.0 [skip ci] ([4885fde](4885fde)), closes [#94](#94) * **release:** 1.5.0 [skip ci] ([71d613d](71d613d)) * **release:** 1.5.1 [skip ci] ([4e6775c](4e6775c)), closes [#12](#12) * **release:** 1.6.0 [skip ci] ([03d0409](03d0409)) * **release:** 1.7.0 [skip ci] ([8bf9c9f](8bf9c9f)) * **release:** 1.8.0 [skip ci] ([a240056](a240056)) * **release:** 1.9.0 [skip ci] ([8715740](8715740)) * start running CI again ([#64](#64)) ([9399f95](9399f95)) * stub blocks interface in tests ([#418](https://github.com/ipfs/helia-verified-fetch/issues/418)) ([a06f64b](a06f64b)) * unskip IPNS test ([#6](#6)) ([76485a4](76485a4)) * Update .github/pull_request_template.md [skip ci] ([5b825e6](5b825e6)) * Update .github/workflows/stale.yml [skip ci] ([3e40ad0](3e40ad0)) * update project config ([#419](https://github.com/ipfs/helia-verified-fetch/issues/419)) ([cb9dc90](cb9dc90)) ### Documentation * add link to blog post and ready-to-run example ([#63](#63)) ([696ed57](696ed57)) * add link to the blog post and example ([#87](#87)) ([f30f979](f30f979)) * change cidv0 to cidv1 in the readme ([#105](#105)) ([060e726](060e726)) * fix readme link ([#51](#51)) ([8a41c57](8a41c57)) * generate readme ([#70](#70)) ([a75567a](a75567a)), closes [#68](#68) [#72](#72) [#72](#72) * move the package docs to root of repo ([#1](#1)) ([8fb4a76](8fb4a76)) * update api docs link in readme ([#14](#14)) ([d615633](d615633)) * update documented default value ([161a470](161a470)), closes [/github.com/multiformats/js-dns/blob/a56c9e0b953d644392cf10fd0792757da0d61c32/src/resolvers/default.browser.ts#L6-L7](https://github.com/ipfs//github.com/multiformats/js-dns/blob/a56c9e0b953d644392cf10fd0792757da0d61c32/src/resolvers/default.browser.ts/issues/L6-L7) ### Dependencies * bump aegir from 42.2.11 to 44.1.4 ([#108](#108)) ([e36fbff](e36fbff)) * bump kubo from 0.26.0 to 0.27.0 ([#12](#12)) ([92cad49](92cad49)) * bump kubo from 0.27.0 to 0.28.0 ([#54](#54)) ([3579844](3579844)) * updating all deps ([#47](#47)) ([6d0ffd8](6d0ffd8)) ### Tests * add gateway conformance tests ([#67](#67)) ([30958fb](30958fb)), closes [#68](#68) * use successRate for gateway conformance tests ([#83](#83)) ([5f71a33](5f71a33))

github-actions · 2024-10-14T10:03:53Z

🎉 This PR is included in version 1.0.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

2color added 3 commits March 26, 2024 14:52

fix: calculate ipns ttl correctly

25ca433

fix: set the cache ttl as seconds consistently

e1f02f4

also add trace logging

chore: suffix argument with unit to avoid confusion

569a076

2color requested a review from a team as a code owner March 26, 2024 14:00

add tests to ensure the ttl is correctly computed

5e0f4cd

2color force-pushed the fix/fix-ipns-ttl branch from 88c4ba7 to 5e0f4cd Compare March 26, 2024 14:01

2color mentioned this pull request Mar 26, 2024

Bug: DoH client is missing in-memory cache ipfs/service-worker-gateway#59

Open

2color changed the title ~~Fix ttl and caching for ipns urls~~ fix: ttl and caching for ipns urls Mar 26, 2024

chore: disable failing test due to bug in js-ipns

9e5c62a

lidel requested changes Mar 27, 2024

View reviewed changes

packages/verified-fetch/test/utils/parse-url-string.spec.ts Outdated Show resolved Hide resolved

SgtPooki approved these changes Mar 27, 2024

View reviewed changes

packages/verified-fetch/src/utils/parse-url-string.ts Show resolved Hide resolved

packages/verified-fetch/test/cache-control-header.spec.ts Show resolved Hide resolved

packages/verified-fetch/test/utils/parse-url-string.spec.ts Outdated Show resolved Hide resolved

Update packages/verified-fetch/test/utils/parse-url-string.spec.ts

d583bcb

Co-authored-by: Russell Dempsey <1173416+SgtPooki@users.noreply.github.com>

2color mentioned this pull request Mar 27, 2024

test: test ipns ttl interop ipfs/helia#476

Merged

2 tasks

lidel approved these changes Mar 28, 2024

View reviewed changes

2color merged commit 44ac5a1 into main Mar 28, 2024
19 checks passed

2color deleted the fix/fix-ipns-ttl branch March 28, 2024 12:48

github-actions bot added the released label Mar 28, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: ttl and caching for ipns urls #34

fix: ttl and caching for ipns urls #34

2color commented Mar 26, 2024 •

edited

Loading

github-actions bot commented Mar 28, 2024

github-actions bot commented Mar 28, 2024

github-actions bot commented May 9, 2024

github-actions bot commented Oct 14, 2024

fix: ttl and caching for ipns urls #34

fix: ttl and caching for ipns urls #34

Conversation

2color commented Mar 26, 2024 • edited Loading

Description

Notes & open questions

Change checklist

github-actions bot commented Mar 28, 2024

github-actions bot commented Mar 28, 2024

github-actions bot commented May 9, 2024

github-actions bot commented Oct 14, 2024

2color commented Mar 26, 2024 •

edited

Loading