Sitecore jss + proxy has degrading performance in amount of requests served per second

[NickiSibbern]

Description

Using Sitecore jss react and proxy the amount of requests per second the proxy handles will degrade over time

Expected behaviour

Requests per second should stay the same or be near the same over time

Steps To Reproduce

1. setup the sample jss site
2. deploy the site to Sitecore, and connect it
3. build the site and copy the output to the proxy server
4. run proxy server
5. perform load test, I did it with ApacheBench using 20 connection for 1000 requests (regardless of variation the degradation still happens)
6. see Requests Per Second and compare the previous run

Your Environment

Sitecore 9.3
JSS 13

test was performed locally and on a kubernetes cluster

Results from own testing

this issue (#287) mentions there is a memory leak, so I did the test with and without the fix to rule out the memory leak
this is the result

# run	no patch	patched
1	440	416
2	341	313
3	268	249
4	218	207
5	187	181
6	162	154
7	139	134
8	109	118

[anastasiya29]

Closing this ticket as this is expected behavior.

The behavior you describe is not an issue with JSS, as the Layout Service scales better than MVC as traffic increases. It's an issue with Node - this is a a side effect of using the server to render virtual-DOM-based frameworks (React, Angular, Vue), which are optimized to run in browsers.

For SSR, Node has to create and re-create a virtual DOM server-side, which is extra work that's normally done by the browser. This is a very intensive operation, which can require several updates as async data is fetched to construct the page. As traffic increases, the Node server becomes your bottleneck as it tries to keep up with the demands of SSR.

For Sitecore's recommendations on how to tackle performance issues in JSS applications, please watch these videos:
Catching up with JSS - Caching in JSS
Catching up with JSS - Deployment Topologies and Performance

[anastasiya29]

@NickiSibbern - I may have misunderstood your description of the "load test".
Can you clarify whether the load was increased over time or whether it was kept the same between runs? If it's the latter, then this is indeed an issue that needs to be looked at.

[NickiSibbern]

@anastasiya29 it is the latter, I ran the test with the same parameters each time, eg. X concurrent connection for X amount of time. I should mention that we tested this with the following load test tools the result was seemingly the same, degrading performance:
ApacheBench, Autocannon, Hey, Lotus

SSR is ofcourse CPU intensive and we would not expect that we could uphold the same amount of RPS over time as the CPU load increases, we would however expect that after a stabilization period where we stop the load test to let the CPU go back to 1-2% we would achieve higher RPS again, this is not the case, for each consecutive run the RPS just degraded further and only way to achieve the RPS of the first run is to "recycle" the NodeJS instance.

[sc-illiakovalenko]

@NickiSibbern Fixed in commit
And released in v13.0.3. Feel free to reopen issue, if you will have this problem.

[NickiSibbern]

Hi @sc-illiakovalenko I did a new test with 13.0.3 and it does not seem fixed to me. Below is the test results, Could you please share your results, maybe I am doing something wrong when running the tests.

jss --version
JSS CLI is running in global mode because it was not installed in the local node_modules folder.
13.0.3

proxy server taken from dev branch, latest commit is:
15240aa (HEAD -> dev, origin/dev, origin/HEAD) Set i18n initial language based on SSR language or static config defaultLanguage to prevent re-renders in multi-lingual apps (#330)

• Created a new VM from Windows 10 PRO build 18363
• Created a new sitecore instance from 9.3 XP Single developer on the vm
• Created a new sample app with the JSS CLI on the vm
• Deployed the sample app to the new sitecore instance
• did a build of the sample app (jss build)
• cloned the JSS repo and followed: https://jss.sitecore.com/docs/techniques/ssr/headless-mode-ssr to inject the sample into the proxy and then ran the following test on the proxy:

autocannon -c 50 -d 120 http://localhost:3000/

#1 | Req/Sec: 123 Requests: 15k Latency: 313ms Bytes/sec: 3.47 MB
#2 | Req/Sec: 58 Requests: 7k Latency: 615ms Bytes/sec: 1.66 MB
#3 | Req/Sec: 45 Requests: 5k Latency: 776ms Bytes/sec: 1.29 MB
#4 | Req/Sec: 39 Requests: 5k Latency: 920ms Bytes/sec: 1.1 MB
#5 | Req/Sec: 34 Requests: 4k Latency: 994ms Bytes/sec: 979 kB
#6 | Req/Sec: 30 Requests: 4k Latency: 1103ms Bytes/sec: 855 kB
#7 | Req/Sec: 27 Requests: 3k Latency: 1249ms Bytes/sec: 779 kB
#8 | Req/Sec: 26 Requests: 3k Latency: 1307ms Bytes/sec: 738 kB
#9 | Req/Sec: 26 Requests: 3k Latency: 1404ms Bytes/sec: 695 kB
#10 | Req/Sec: 23 Requests: 3k Latency: 1442ms Bytes/sec: 672 kB
-> restart nodejs process
#11 | Req/Sec: 125 Requests: 15k Latency: 309ms Bytes/sec: 3.53 MB
#12 | Req/Sec: 58 Requests: 7k Latency: 616ms Bytes/sec: 1.66 MB
#13 | Req/Sec: 45 Requests: 5k Latency: 769ms Bytes/sec: 1.29 MB
#14 | Req/Sec: 38 Requests: 5k Latency: 907ms Bytes/sec: 1.1 MB
#14 | Req/Sec: 34 Requests: 4k Latency: 1002ms Bytes/sec: 965 kb

the numbers are the taken from the avg. output from autocannon

Observations:

• on first run, Doesn't build #1 and Use the RichText helper for the basic app #11 the cpu spikes to 50% and then normalizes on 20% the remainder of the runs used around ~20%.
• No memory leak detected when running the load test, its around ~65 - ~70 MB during load test.

[sc-illiakovalenko]

@NickiSibbern Yeah, thank you. I found another possible problem. I'll continue to investigate it

[sc-illiakovalenko]

@NickiSibbern Hi! Please install latest version and try again to run load test, i pushed fix.
I'll wait for your response. Will reopen if issue will happened

[NickiSibbern]

Hi @sc-illiakovalenko I will have to do some more testing but a quick test indicated that the problem seems fixed.