-
-
Notifications
You must be signed in to change notification settings - Fork 865
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Varnish cache tags with a big number of entities #3168
Comments
Follow up of a discussion at SymfonyCon:
I'm all in favor of a strategy to reduce cache tags sent to Varnish, to be able to work with default nginx / varnish config. |
We're facing this issue currently with HAProxy rejecting our Nginx (with bumped buffer size) response because of |
I stumbled upon the same need of "disabling" varnish for a specific collection, because of the huge number of IRIs retrieved, not to mention IRIs of entities nested in each one of them. The Cache-Tags header was so long that it was crashing (error 500). The built-in features of API Platform apparently do not provide an option to disable a particular collection. The fix proposed by @bastnic was a bit complicated for my needs, but inspired me to create a very simple EventSubscriber. I thought interesting to share it if someone encounters the same issue. You just have to copy-paste the code in a new file src/EventSubscriber/CacheTagsSubscriber.php, then replace your_collection_name by the entity you need to disable, and it works : <?php
namespace App\EventSubscriber;
use Symfony\Component\EventDispatcher\EventSubscriberInterface;
use Symfony\Component\HttpKernel\KernelEvents;
use Symfony\Component\HttpKernel\Event\ResponseEvent;
final class CacheTagsSubscriber implements EventSubscriberInterface
{
public static function getSubscribedEvents()
{
return [
KernelEvents::RESPONSE => ['removeCacheTags', -2],
];
}
public function removeCacheTags(ResponseEvent $event)
{
$request = $event->getRequest();
$response = $event->getResponse();
if($request->getPathInfo() === '/your_collection_name) {
$response->headers->remove('Cache-Tags');
}
}
} |
Thanks @arthurpietruch for your feedback! I agree this is too complicated, but it maintains the "exact" nature of the cache while still being somewhat cached (it's purged too much but I prefer this ratio for now). I'm waiting (searching) for a better fix ;). |
Hi @bastnic Did you find a cleaner way to handle this ? Example calling my categories endpoint leads to 8K Cache-tags headers with 500 error. Regards |
Hi @lwillems, in my todo list, it's right in the bottom :p. It works for now so and looking at your cardinality, it should work for you too. |
I'm facing issues with large cache-tags header too, at the moment. In my case, the main cause is the length of my IRI's: using slugs makes them really long. I think it might be a solution to create an option to configure custom cache tags, next to the IRI of an API resource. If that is possible, I can shorten my cache tag from |
we're working on this at #5758 |
How does this PR reduce the headers size of Cache-tags ? I'm not sure what the collector is doing |
On a biiig api platform website with lots and lots of nested entities, we need varnish to be fully working.
I talked with @alanpoulain last night and we concurred that I don't have an usual setup but the experience must be shared as it can helps some people.
I already have fixed all my issues on my side, but cool if we can fix the official release.
There is multiples troubles:
Cache-Tags
headers are too long. More on that later.Too much cache tags to add on the response
On the biggest response i found, the Cache-Tags header weight 100k chars. It's quite long.
I tried multiples approaches:
http_resp_hdr_len
,http_resp_size
): doesn't scale as much as I wanthttp_max_hdr
): it's seems to work at first but in fact when the purge happens, it only use the Cache tags of the first header line. Maybe a bug on my sideBrace yourself, awful code:
with this patch, I' finish with only 4-5 collections on some big list, and I'm ok with that.
One issue I may have is that I'm using
ORMBehaviors\Translatable
and when I update a translation only and nothing on the entity, the entity itself is not updated (only the translation), but the translation is not an api platform resource. InPurgeHttpCacheListener
, the main entity is seen as a "relationTag" and so the collection iri is not added to the list of iri to purge. I will fix that with another doctrine listener that update the main entity updated at field and so the entity will be seen as updated.Too much cache tags to add on the response
When I insert a lot of entities at once, the purge cannot works as the regexp is waaaaaaaaay too long.
Multiple approaches too:
The first approach can add some safety and fix #1856.
WDYT?
cc @teohhanhui
The text was updated successfully, but these errors were encountered: