Changed IP then CSS files 404

[benjaminadk]

I had to rebuild a server and changed IPs but same domain. When I fire up new server everything works except 404 for all pagespeed CSS files.

Other file types appear to be working as expected, and when I try ?ModPagespeed=off it fetches the correct file directly and everything looks fine.

Running the same CSS related stuff as old configuration

 #CSS handling
 pagespeed EnableFilters combine_css,fallback_rewrite_css_urls,flatten_css_imports,inline_css,rewrite_css;

I did not set this up initially, and this is my intro to pagespeed. Any ideas of what I am missing here?

[oschaaf]

Maybe the dns resolution is being cached for the domain on the server. The module would then attempt to fetch from the old server.

[Lofesa]

| I did not set this up initially, and this is my intro to pagespeed. Any ideas of what I am missing here?

Before this, you do nothing with css files?

Can you try https;//mydomain.com/?PageSpeedFilters=+debug and then look at the html code to see debug messages?

[benjaminadk]

It seemed to be working for a while tonight. CSS files were all being delivered. All I did was purge the cache a few times. Then I was switching between Google users (chrome instances) and then all CSS 404ed again.

Could it have something to do with this session cookie?

#this next part requires the more_clear_headers module - if nginx fails, remove this
if ($skip_cache = 0 ) {
    more_clear_headers "Set-Cookie*";
    set $rt_session "";
}

This is the only thing I didn't install from the previous setup. Based on the comment I found I just skipped it.

[Lofesa]

When you cleared the cache, css files are served rewrited by pagespeed or are the original css files?

My bet is that some thing has changed and pagespeed is unable to fech the resources.
I you try https;//mydomain.com/?PageSpeedFilters=+debug maybe the debug messages (as html comments in the code) have some clue.

How pagespeed fecht https resources (What config have)?
What domains have you authorized?

Have you tried the LoadFromFile?

[benjaminadk]

I am back trying to get our server up again.

This is a gist with the output from ?PageSpeedFilters=+debug

The configuration

PageSpeed appears to be partially working, with a couple CSS files not being found

When I purge the cache:
That initial reload looks fine as it is fetching the actual server resources.
Then subsequent reloads w/o purging is almost like a crap shoot - sometimes all the CSS files are available, and other times some are 404 - and it is different files each time. Once in a while it is the main CSS file, which is catastrophic.

It also looked like a couple of the 404s were erroring around the Google Analytics script, even though they were totally unrelated style files.

The configuration is exactly the same on this new server as the current one. This new one has new versions of Ubuntu, PHP, Nginx, etc.

[Lofesa]

This is a gist with the output from ?PageSpeedFilters=+debug

No, this gist contains only javascript, in not the output from the filter.

Some server config is wrong, when I try to PageSpeedFilters=+debug,-inline_css this come in an infinyte loop, the plus sign is decoded to %20....

The outpu from the debug filter is some like:

You can see in green the debug messages.

An other question is that some css don´t be manipulated by pagespeed.
Ths one is:

But this other not:

See the diff betwen the Etag headers, and the second image don´t have the pagespeed header. Both are from your domain, so are authorized.

[benjaminadk]

I still haven't figured this out...granted I just now put the new IP live for the first time since my last post. I seems to randomly produce 404 errors for CSS files - but only some of them.

[oschaaf]

I still haven't figured this out...granted I just now put the new IP live for the first time since my last post. I seems to randomly produce 404 errors for CSS files - but only some of them.

I would recommend putting the module in OptimizeForBandwidth mode to avoid the need for http(s) fetching while figuring this out. You may also want to check https://www.modpagespeed.com/doc/https_support#configuring_ssl_certificates as a potential culprit.

[benjaminadk]

The issue persists. When I purge the cache everything runs fine for a couple days, then I start to get 404s again on some CSS files. If it happens to hit the main CSS it becomes a critical situation. I purge the cache a couple times and everything works again.

Here is our output from ?PageSpeedFilters=+debug:
https://gist.github.com/benjaminadk/75cbcaf8c0ab21541ade94de3120be79

Here is our current config:

# ...

# PageSpeed Admin
pagespeed StatisticsPath /ngx_pagespeed_statistics;
pagespeed GlobalStatisticsPath /ngx_pagespeed_global_statistics;
pagespeed MessagesPath /ngx_pagespeed_message;
pagespeed ConsolePath /pagespeed_console;
pagespeed AdminPath /pagespeed_admin;
pagespeed GlobalAdminPath /pagespeed_global_admin;
pagespeed Statistics on;
pagespeed StatisticsLogging on;

# PageSpeed Cache Purge
pagespeed EnableCachePurge on;
pagespeed PurgeMethod PURGE;

pagespeed MessageBufferSize 300000;
pagespeed FileCachePath /sites/ladesignconcepts.com/pagespeed-cache;
pagespeed ForceCaching on;
pagespeed FileCacheSizeKb            1024000000;
pagespeed FileCacheCleanIntervalMs   3600000;
pagespeed FileCacheInodeLimit        500000;
pagespeed SslCertDirectory /etc/ssl/certs;

server {
  # ...
  
    # Pagespeed Config
    pagespeed on;

    location ~ "\.pagespeed\.([a-z]\.)?[a-z]{2}\.[^.]{10}\.[^.]+" {
        add_header "" "";
    }

    location ~ "^/pagespeed_static/" { }
    
    location ~ "^/ngx_pagespeed_beacon$" { }

    # Disable pagespeed WP Admin, Woo cart, etc
    pagespeed Disallow */wp-admin/*;
    pagespeed Disallow */cart/*;
    pagespeed Disallow */ideas/*;
    pagespeed Disallow */my-account/*;

    # PageSpeed Filters
    pagespeed RewriteLevel PassThrough;

    pagespeed EnableFilters add_head;
    pagespeed EnableFilters convert_meta_tags;
    pagespeed EnableFilters extend_cache;
    pagespeed EnableFilters rewrite_style_attributes_with_url;
    pagespeed EnableFilters inline_import_to_link;

    # HTML
    pagespeed EnableFilters insert_dns_prefetch;
    pagespeed EnableFilters collapse_whitespace,remove_comments;
    pagespeed EnableFilters canonicalize_javascript_libraries;
    pagespeed PreserveUrlRelativity on;
    pagespeed ImagePreserveURLs on;

    # CSS
    pagespeed EnableFilters combine_css;
    pagespeed EnableFilters fallback_rewrite_css_urls;
    pagespeed EnableFilters flatten_css_imports;
    pagespeed EnableFilters inline_css;
    pagespeed EnableFilters rewrite_css;

    # JS (Disabled)
    #pagespeed EnableFilters combine_javascript;
    #pagespeed EnableFilters inline_javascript;
    #pagespeed EnableFilters rewrite_javascript;

    # Images
    pagespeed EnableFilters sprite_images;
    pagespeed EnableFilters lazyload_images;

    # Images (Disabled)
    #inline_images,recompress_images,convert_to_webp_lossless,resize_images.
    #pagespeed EnableFilters rewrite_images;
    #pagespeed DisableFilters convert_jpeg_to_webp,convert_to_webp_lossless,convert_to_webp_animated,recompress_webp;
    #pagespeed EnableFilters responsive_images,resize_rendered_image_dimensions,resize_images,responsive_images_zoom;
    #pagespeed EnableFilters strip_image_meta_data,strip_image_color_profile;
    #pagespeed EnableFilters jpeg_subsampling,recompress_jpeg,recompress_png,;
    #pagespeed EnableFilters convert_png_to_jpeg,convert_gif_to_png;
    #pagespeed EnableFilters convert_jpeg_to_progressive;

    # Lazy Loading (Disabled)
    #pagespeed LazyloadImagesBlankUrl "https://www.gstatic.com/psa/static/1.gif";
    
    # Prioritize (Disabled)
    #pagespeed EnableFilters defer_javascript;
    #pagespeed EnableFilters prioritize_critical_css;

    # Google Analytics Async
    pagespeed EnableFilters make_google_analytics_async;

    # Google Fonts
    pagespeed FetchHttps enable;
    pagespeed EnableFilters inline_google_font_css;

    # Domains
    pagespeed Domain https://ladesignconcepts.com;
    pagespeed Domain http*://*.ladesignconcepts.com;
    pagespeed MapRewriteDomain ladesignconcepts.com images.ladesignconcepts.com;
    pagespeed MapOriginDomain localhost ladesignconcepts.com;
    pagespeed ShardDomain https://images.ladesignconcepts.com https://ladesignconcepts.com;

    pagespeed MaxSegmentLength 250;

    # Memcached 
    pagespeed MemcachedThreads 1;
    pagespeed MemcachedServers "127.0.0.1:11211";
}

# ...

[Lofesa]

1.- pagespeed FetchHttps enable need

pagespeed SslCertDirectory directory;
pagespeed SslCertFile file;

See https://www.modpagespeed.com/doc/https_support#configuring_ssl_certificates as stated by @oschaaf

2.- pagespeed Domain https://ladesignconcepts.com; and pagespeed Domain http*://*.ladesignconcepts.com; are redundant, the 1st is included in the 2nd.

3- Why you need MapRewriteDomain, MapOriginDomain and ShardDomain?

Whit pagespeed MapRewriteDomain ladesignconcepts.com images.ladesignconcepts.com; any images.ladesignconcepts.com in the html code is converted to ladesignconcepts.com but I can´t see any images.ladesignconcepts.com in the html code when pagespeed is turned off.

Whit pagespeed ShardDomain https://images.ladesignconcepts.com https://ladesignconcepts.com; you try to download request to https://images.ladesignconcepts.com from https://ladesignconcepts.com but you use http2 so no shard works here. Http2 creates 1 connection and download all assets from this connection.

Whit pagespeed MapOriginDomain localhost ladesignconcepts.com; you make a request to localhost when the client request http://ladesignconcepts.com. This need 2 servers blocks in nginx config, 1 for localhost (http) and 1 for ladesignconcepts.com (http).
If ladesignconcepts.com is https and you have this config, then the directive must be set as:
pagespeed MapOriginDomain localhost https://ladesignconcepts.com;

[benjaminadk]

@Lofesa, thanks for bringing these issues to my attention. I inherited this whole project and have no idea who wrote the code or set it all up. These issues aren't surprising, there have been many.

I attempted to address the config issues:

1 - I added SslCertDirectory /etc/ssl/certs and I read the follow to mean I would not set SslCertFile file; on my VM.

The default directory for Debian-based systems is /etc/ssl/certs, and there is no certificate file setting.

2 & 3 - I commented out MapRewriteDomain, MapOriginDomain and ShardDomain.

I will continue to monitor this issue.

[Lofesa]

I have tested it now, and don´t have any more 404 error.
But some css files are "out of radar" for pagespeed module. You may compare the http headers of css files from your own domain (from others domains not to be rewrited by pagespeed is normal) and you can see diffs betwwen these rewrited by pagespeed and those not rewrited.
Maybe there is a Disallow directive for those css files not rewrited?

[benjaminadk]

yes. i put a Disallow on a couple css files for plugins.

[benjaminadk]

So I finally got a 404 on the main CSS file today. One week since the last time it happened. I purged the cache and all is well.

[bigretromike]

@benjaminadk did you end up resolving the issue or is this something that come back link boomerang ?