-
-
Notifications
You must be signed in to change notification settings - Fork 6.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[build] importing from hashed chunks makes caching terribly ineffective #6773
Comments
Additional example 1 (with vue 3)I've created a new branch in my reproduction so that maybe the problem is clearer: https://github.com/jacekkarczmarczyk/vite-chunks-very-very-bad/tree/vue3 Run Additional example 2 (without vue)https://github.com/jacekkarczmarczyk/vite-chunks-very-very-bad/tree/no-vue Update body of Webpack examplehttps://github.com/jacekkarczmarczyk/vite-chunks-very-very-bad/tree/webpack-async Change the text in |
I had a very similar problem and came up with this hacky solution: import { defineConfig } from 'vite'
import { createHash } from 'crypto'
import vue from '@vitejs/plugin-vue'
export default defineConfig({
plugins: [vue()],
build: {
rollupOptions: {
output: {
entryFileNames: 'assets/[name].js',
chunkFileNames: chunkInfo => {
if (chunkInfo.isDynamicEntry) {
const hash = createHash('md5')
.update(Object.values(chunkInfo.modules).map(m => m.code).join())
.digest('hex')
.substr(0, 6)
return 'assets/[name].' + hash + '.js'
} else {
return 'assets/[name].[hash].js'
}
}
}
}
}
}) When adding Every kind of feedback is very appreciated! :) |
@jacekkarczmarczyk I have very much the same problem due to some (react) lazy component loading. Did you manage to work-around the issue? |
Yes, by using Vue Cli |
now ,How to configure Vite long-term cache? |
There is an additional problem with circular dependencies causing all components to import the main index because Vite ejects a helper function there (a virtual module). Here's how I prevent that helper function (and Vite polyfills) from being added to main index chunk (and keep it lean): import { defineConfig } from 'vite'
import vue from '@vitejs/plugin-vue'
// https://vitejs.dev/config/
export default defineConfig({
build: {
target: 'ES2020',
rollupOptions: {
output: {
// Workaround: Vite is bundling its plugins to the main index chunk,
// causing circular dependencies and cascading hash changes.
manualChunks(id) {
if (id.startsWith('vite/') || id.startsWith('\0vite/')) {
// Put the Vite modules and virtual modules (beginning with \0) into a vite chunk.
return 'vite';
}
},
},
},
},
plugins: [vue()]
}); |
@patak-dev sadly, #9870 didn't change much Update: no improvement with Results with Vite 4.0.0-alpha.0Example 1 (vue 2)Steps: run Vite 2.9 (
|
I don't think not much can be done here on Vite side (except using another bundler which I don't believe is going to happen soon), I've added the issue to rollup repo |
For anyone interested - here's a proof of concept based on @lukastaegert's comment in rollup repo: https://github.com/jacekkarczmarczyk/importmap-plugin (UPDATE: now supports SystemJS) Feel free to steal the code and create a proper vite/rollup plugin. Any comments welcome Plugin applied to the original reproduction (https://github.com/jacekkarczmarczyk/vite-chunks-very-very-bad/tree/main-stable-hash-plugin - |
vite is fast and great, but this one feature really causes a lot of issues. For example on every update of our application - the dynamic loads all give errors because the bundle they need (even if it did not change) has a hash change - so the file does not exist. @bluwy - is there plan to fix this caching issue - unsure if the suggestion above is the right fix. Just trying to get some eyes on this to maybe be fixed in 4.1? |
There really isn't a way to simply fix this as explained in #6773 (comment). The way bundlers work without a runtime proxy module loader means it would always have this hash cascading issue. Unless Vite implements it by default, which could be a possibility, but your best bet for now is to use the plugin: https://github.com/jacekkarczmarczyk/importmap-plugin. |
This sounds like you need to fix your deployment strategy. My recommendation is to not delete the files from your previous deployment but just put the changed files next to it, that fixes this issue easily. Otherwise, there is no safe way to do a deployment without compromising long-running sessions. Another approach is not to use hashes but separate folders per deployment. |
@lukastaegert We do an automated deployment (OctopusDeploy) that contains the package of the site (client and server). This creates a brand new deployment, folder, etc with the files from the deployment (which would not contain old files by definition). This has the positive thing of being a brand new deployment with all new configs and such - but has the negative of old files not being there. With create-react-app (what we used before), this type of error occurred very infrequently (like once per month type of thing) instead of every deployment. I agree we would compromise long-running sessions when the code they were needing changed. However, we bundle large dependencies separately - so rarely do the actually change - but with vite - appears they always change. @bluwy thank you for the comment and consideration. I was just 'waiting for official fix', but will try above. Perhaps consider adding this side-effect to the docs in the meantime? maybe https://vitejs.dev/guide/features.html#async-chunk-loading-optimization and https://vitejs.dev/guide/build.html#chunking-strategy letting people that ViteJs will generate a new hash each time for dynamically loaded modules even if they did not change (resulting in a new filename each time) - eliminating the browser cache feature of generating 'stable chunks'. Thanks again for a great product - our build times have never been faster! |
@bluwy can I ask how high this possibility is? Is it something closer to "yeah, maybe, after we solve 1000 of other more important issues" or to "that's indeed a big problem we want to solve asap" (regardless of whether it's easy or hard to implement it). @waynebrantley while my plugin seems to solve the problem in simple cases and I do use it in production with smaller projects, it wasn't tested (and I'm pretty sure it wouldn't work) in more complicated setups (like PWA or anything that relies on chunk names), so I'd prefer to see it as a built-in solution instead of 3rd party plugin. Not sure if it needs some changes in rollup to make the change in Vite easier though, there doesn't seem to be much discussion on it in rollup repo (some of other discussions are linked in this issue), which surprises me a lot tbh as I don't believe that most of the projects are "release and forget" or that devs don't case about caching... |
It's more of "if someone's interested and creates a PR, we can discuss and review it" kinda situation. Since Vite's development comprises of community contributions, and sometimes sponsored work from metaframeworks, we don't have a list of things we have to do. For me, I usually implement features or bug fixes if I find them interesting. |
I meet same question and update vite@4.0.4,but not work,How did you solve it? |
I like the approach that parcel uses to resolve this issue https://parceljs.org/features/production/#cascading-invalidation |
This is not straightforward as it requires you to have an intermediate loading handler that maps imports to their hashed counterparts when something is imported. That being said, this is possible by using import maps. While those are not yet widely adopted, this makes them work for at least 95% of browsers: https://github.com/guybedford/es-module-shims Then you can basically completely remove hashes from file names and instead e.g. put them into import maps like this: <script type="importmap">
{
"imports": {
"./my-chunk.js": "./my-chunk.js?ab34fa7"
}
}
</script> Here, It should be easy to put this into a plugin, but I would not be surprised if someone has not done that already. It will completely avoid any hash cascades. Note that it will not completely fix the original issue because long-running sessions through a re-deploy will still run into chunks that e.g. may be missing imports, or chunks that do not exist at all. I.e. chunk To fix that while preventing the hashing cascade, one could spin it differently: All files have content hashes, but internally, they reference other chunks without hashes. To do that, one would need to create a bundle without hashes and in a post-processing step, calculate the content hashes of each chunk and change the file names and generate the import map. <script type="importmap">
{
"imports": {
"./my-chunk.js": "./my-chunk-ab34fa7.js"
}
}
</script> |
@lukastaegert Someone did, and it's already mentioned few times in this issue |
This comment was marked as off-topic.
This comment was marked as off-topic.
This problem also affects us, webpack uses |
Pretty sure this would cut my build time down by half due to Monaco Editor being included everytime. |
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
@anatolsommer Yes, your screenshot above is awesome. I also very much hope that Vite 4 can also achieve this effect. |
@dukexie This is Vite 4 (it says |
@anatolsommer Oh I see, thank you. |
@dukexie I completely agree! Just as long as I can keep the entry file small and with ETags/304 I can live quite well with this workaround until there is an actual solution and tbh I think most people can. :) |
Implementing a solution for the Cascading Cache Invalidation problem using importmap as a feature of Vite might not be as complicated as it seems. As a PoC, I implemented the functionality to rewrite source code and generate importmap, and confirmed that it could pass the existing test cases. Therefore, I have opened a Discussion proposing the implementation of a feature in Vite to solve the Cascading Cache Invalidation problem using importmap. |
@jacekkarczmarczyk I solved the hash change problem with importmap-plugin, but the ImportmapPlugin base is set to the same as' base 'option in Vite config and I get an Error: 'Error: Missing import map entry for entry file: /app/index.js', how to solve this? |
@zhuchenglong Not sure why it's not working, but the plugin was just a proof of concept, and given that afair it's not compatible with Vite 4/5 and that there's a PR in Vite that seems to resolve the issue I'm not going to investigate. |
@jacekkarczmarczyk Which version of vite are you using? I'm using vite4, but I haven't solved the hash change issue yet. Do you know which PR in Vite resolve the issue? |
I'm not using Vite at the moment, PR is here #15373 |
@jacekkarczmarczyk Thank you for your answer. I have read #15373 , but I did not understand how to use chunkMap in the vite4/5 project. Can you give me a demo? |
This comment has been minimized.
This comment has been minimized.
An working importmap plugin solution for Cascading Cache Invalidation problem with vite, vite-plugin-long-term-cache-import-map,it ensures the long term cache for almost all assets using sourcemap. |
New hack
|
想请教一下你现在知道怎么在最近的vite里解决这个问题了吗,需要添加什么配置 |
I would like to ask if I want to solve the problem of hash changes after modifying the packaging of a single file. Can this configuration be used to solve the problem?
set the entry file: Cache-Control: must-revalidate, max-age=0 |
I think the core problem is not the cache for the single hashed file but the referenced file. e.g. something like https://www.webpackjs.com/configuration/optimization/#optimizationruntimechunk is needed |
I tried like this, but it didn't work as expected.
|
Also, vite or rollup seems to have no concept of |
All hashes are actually content hashes. |
Update
I've published a plugin that solves the issue for me: https://github.com/jacekkarczmarczyk/importmap-plugin
See #6773 (comment) for example usage in the issue reproduction repository
Describe the bug
Built files import other files which names contain content hash. So if the chunk
A
changes its contents then the output file changes its hash (A.123.js
becomesA.234.js
). So if there's other file that imports fromA
chunk then it also changes its contents and hash becauseimport {...} from 'A.123.js'
becomesimport {...} from 'A.234.js'
.Imagine now that I'm defining an env variable with build time. Main chunk imports this file to show the build time in
App.vue
. However main chunk exports the following vue related function:that is being imported by all component chunks (note that this is just example, depending on the project there might be other user defined functions that are exported from main chunk, also in my actual project I've extracted vendors to separate chunks but still vue related functions were exported from main chunk).
So now when I build again env chunk will change its contents and hash/name, therefore main chunk will change it's contents and hash/name, therefor ALL other chunks will change their names. That makes caching very innefective. Also worth to mention that that technique worked fine in vue-cli - when build time changed only single small env.hash.js chunk was changed, all others remained unchanged (EDIT: there's also relatively small runtime chunk that also was changed)
Reproduction
https://github.com/jacekkarczmarczyk/vite-chunks-very-very-bad
Run
yarn && yarn build && git add . && yarn build && git status
It behaves the same when displaying build time is moved to an async Home.vue route component, in this case hash change of main chunk is definitely unjustified
System Info
Used Package Manager
yarn
Logs
No response
Validations
Additional notes
I believe that this more like rollup issue (or whatever is used for generating chunks) but if it's possible to fix it by using some rollup's settings then vite should use these settings by default. And if it's not possible to fix it by using different settings then I think that chosing rollup was not the best choice. This page https://bundlers.tooling.report/hashing/js-import-cascade/
and this https://bundlers.tooling.report/hashing/avoid-cascade/ however suggest that this should not be an issue for rollup
The text was updated successfully, but these errors were encountered: