Enhance the cache option

[szmarczak]

The cache mechanism:

• should be a promise
• should be replaceable with any other module*
• should have something hook-like to allow storing different properties e.g. ip addresses
• should accept different caching logic
• should make a new request when the cached response cannot be updated (Do not use invalid cache #963)
• beforeCache hook

* Through the request option. Accepting ClientRequest and IncomingMessage as the return type. Return undefined to fallback. You can override the request option also by returning a ClientRequest or an IncomingMessage instance in a beforeRequest hook.

[sithmel]

About #746:
What about the approach of adding an hook that can override the headers (including the caching ones), before they are evaluated by cacheable-request ?
This way you can set different caching headers, tweaking the cache key (etag or even the url) etc.

[szmarczak]

@sithmel beforeCache hook? Sounds good to me.

[sithmel]

Happy to implement it as soon as this is done and merged

[sithmel]

@szmarczak I guess I should wait for #1051 to be merged ... do you agree ?

[szmarczak]

@sithmel You don't have to. Feel free to send a PR anytime, I'll merge with master later. No worries! :)

[MrSwitch]

Can i add another suggest feature which is kind of related to being a promise #1078

[rifler]

i've just read ky docs https://www.npmjs.com/package/ky#hooksbeforerequest

The hook can return a Request to replace the outgoing request, or return a Response to completely avoid making an HTTP request. This can be used to mock a request, check an internal cache, etc

sounds quite similar for #746 needs

[szmarczak]

@rifler It's already done. It hasn't been documented yet.

[szmarczak]

@rifler 779062a

[adityapatadia]

It would be great if you can also write how can we give custom request. It's still not clear from Readme.

[szmarczak]

@adityapatadia http.request returns a ClientRequest instance for example. I will write a more detailed tutorial soon.

[adityapatadia]

@adityapatadia http.request returns a ClientRequest instance for example. I will write a more detailed tutorial soon.

I understand the return type but I am not sure how can I return custom request function.

Do you mean something like this?

beforeRequest: [
			async options => {
				return new CacheableRequest(http.request);
			}
		]

[szmarczak]

No. I mean ClientRequest, not CacheableRequest:

const got = require('got');
const https = require('https');

got('https://example.com', {
    hooks: {
        beforeRequest: [
            () => {
				// `https.request` returns a ClientRequest instance
                return https.request('https://httpbin.org/anything');
            }
        ]
    }
}).json();

[szmarczak]

You can pass an IncomingMessage instance as well. To generate one you can use for example https://github.com/lukechilds/responselike

[adityapatadia]

Understood. Thanks for help.

[szmarczak]

No problem, if you got any other questions, feel free to ask :)

[hk7math]

May I know if #746 is possible yet? When remote host returns headers with {"cache-control": "no-cache"}, is there a way to override it without adding beforeRequest and afterResponse hooks to implement custom caching? Thank you very much!

[adityapatadia]

No problem, if you got any other questions, feel free to ask :)

Sorry to get back after long time but I tried this and it throws many errors related to agent and timeout.

beforeRequest: [async options => {
        return new Promise((resolve, reject) => {
          let cr;
          if(options.url.protocol == "https:"){
            cr = new cacheableRequest(https.request);
          } else {
            cr = new cacheableRequest(http.request);
          }
          cr(options).on("request", resolve);
        });
      }],

Basically there is no easy way to just edit some part of cacheableRequest module and then plug it in this library.

[szmarczak]

@adityapatadia You're creating a new CacheableRequest instance on every got call. You can do this instead:

const runCachedHttps = new CacheableRequest(https.request);
const runCachedHttp = new CacheableRequest(http.request);

...

beforeRequest: [async options => {
        return new Promise((resolve, reject) => {
          const cr = options.url.protocol == "https:" ? runCachedHttps : runCachedHttp;
          const instance = cr(options);
		  instance.on("request", resolve);
		  instance.on("error", reject);
        });
      }],

...

[StarpTech]

Anything we can do to push this feature further?

[KyleAMathews]

I had an issue where I couldn't trust the cache-control headers so edited them before saving the cached response gatsbyjs/gatsby#32012

[szmarczak]

I have started working on a new cache implementation. I will let you know when I have a working example.

[szmarczak]

Update: more than 50% RFC-compliant things finished.

[szmarczak]

Update: I just need to finish revalidation, conditional requests, tests and we're all set ✨

[chris-schra]

any news on this? :)

[szmarczak]

Sorry, been busy w/ work and uni. Will work on this asap.

[vjpr]

Any progress on this?

I just want to be able to always cache a response...is there any hook I can use to do this?

[wilsonssss]

Any progress on this?

I just want to be able to always cache a response...is there any hook I can use to do this?

The same requirement.