Does component streaming and suspense hurt SEO?

[SongRongLee]

This page uses new App router with dynamic rendering. The children is a server component which requires 1 second to render.

If we disable javascript in the browser and refresh the page, we can see the actual pre-rendered HTML -- Yes, it shows a loading state indefinitely.

I understand this is what component streaming mechanism offers us, but if a crawler tries to get this page, does next.js give it the loading state as well? Doesn't this behavior hurt SEO?

In my opinion, the best behavior is like fallback: true of getStaticPaths in page router, where normal users are served loading screens, while crawlers are blocked and served a fully rendered page with data.

It would be great if someone can help clarify this or describe it in the documentation.
Thank you very much!

[ciruz]

Did you test it yourself?

https://pagespeed.web.dev/analysis/https-app-dir-vercel-app-loading-books/ndeij4yca3

And what google has in the cache?
https://webcache.googleusercontent.com/search?q=cache:Y6KgMCTS43sJ:https://app-dir.vercel.app/loading/books
view-source:https://webcache.googleusercontent.com/search?q=cache:Y6KgMCTS43sJ:https://app-dir.vercel.app/loading/books

[SongRongLee]

Thanks for the reply. I did use the recommended tool written in the document to test the "crawler view" of the website. It seems the full page is loaded correctly. The result is fully rendered because the crawler they use accepts limited script execution, but the execution of scripts seems to have negative effect on SEO due to resource limitation. According to the official document:

3. Rendering – executing any resources on the page such as JavaScript that might enhance the features and enrich content on the site. This process doesn't happen for all pages that are crawled and sometimes it happens before the content is actually indexed. Rendering might happen after indexing if there are no available resources to perform the task at the time.

Besides, according to the article comparing websites w/ and w/o pre-rendering, a fully client-side app can be actually rendered in the tools above... so in my opinion, disabling script execution might be a better way to see the result of pre-rendering. That's why I think app router component streaming is not actually doing better than page router's fallback: blocking regarding SEO. Please correct me if I have misunderstood. Thanks!

[kcrwfrd]

Great question, I was wondering the same. FWIW lighthouse gives your example page a 100 for SEO, but I don't know if it's properly assessing your question. I'm also seeing the loaded content in the Google cache...but the question still remains.

[Ryuurock]

Great question

[oliviertassinari]

Streaming can hurt SEO but it depends on your use case, at least, it's how I read: https://webmasters.stackexchange.com/questions/140250/do-search-engines-perform-js-rendering-while-crawling. Google also explains in https://developers.google.com/search/docs/crawling-indexing/javascript/javascript-seo-basics how its indexing engine works.

1. Crawlers without JavaScript support. Google can index pages with component streaming but Yandex and Baidu are not indexing client-side rendered pages. Google has a 90%+ share of the search market and supports client-side rendered pages so it's probably OK in most use cases.

2. The render queue. Googlebot has separate queues for regular crawling and rendering. It does a first pass to grab the server-supplied HTML then it comes back later to do the rendering. The reason is that running JavaScript is much more expensive than analyzing a plain HTML page. See Rendering Queue: Google Needs 9X More Time To Crawl JS Than HTML | Onely. If you have millions of pages with content that changes frequently, streaming looks like a deal breaker to me. If you have a thousand pages with content that is stable in time, it's OK.

3. The semantics of the page. Deferred components are rendered in a hidden div, appended to the .html output, so they are rendered outside of context. When a bot crawls the page without its JS engine enabled, it won't see all the DOM nodes with the correct parent relationship.

So the claim in https://nextjs.org/docs/app/building-your-application/routing/loading-ui-and-streaming#seo looks false:

Since streaming is server-rendered, it does not impact SEO.

streaming has an impact on SEO, but it largely depends on your use cases. I did a quick fix along the way #61647.

[jynzo94]

If you disable javascript in the browser, tha data is not loaded(streamed) so I guess it hurts SEO but I might be wrong.