Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to launch host processes #2461

Merged
merged 21 commits into from
May 28, 2021

Conversation

letmaik
Copy link
Member

@letmaik letmaik commented Apr 13, 2021

This PR allows to trigger host processes from an app endpoint. The app endpoint does not block on it.

  • What if the host process sends a response back to CCF (via HTTP) and the relevant app endpoint depends on the transaction that triggered the host process being finished/committed? Does it make sense to investigate an after-commit behaviour?

    • This is ok, assumption is that things are idempotent, so retrying a failed request in this situation would be fine.
  • To expose it to JS I temporarily expose the node state as a second argument during ccf population. That's a work-around. Related to Make ccf global injection in JS Context modular #2448

@ccf-bot
Copy link
Collaborator

ccf-bot commented Apr 13, 2021

letmaik/trigger-host-process-launch@26231 aka 20210528.25 vs main ewma over 20 builds from 25716 to 26210

Click to see table
build_id build_number sb_sgx_cft^ sb_sgx_cft_mem sb_sgx_bft^ sb_sgx_bft_mem sb_ws_sgx_cft^ sb_ws_sgx_cft_mem sb_sig_sgx_cft^ sb_sig_sgx_cft_mem tpcc_sgx_cft^ tpcc_sgx_cft_mem tpcc_sgx_bft^ tpcc_sgx_bft_mem ls_sgx_cft^ ls_sgx_cft_mem ls_ws_sgx_cft^ ls_ws_sgx_cft_mem ls_jwt_sgx_cft^ ls_jwt_sgx_cft_mem ls_js_sgx_cft^ ls_js_sgx_cft_mem ls_full_js_sgx_cft^ ls_full_js_sgx_cft_mem ls_js_jwt_sgx_cft^ ls_js_jwt_sgx_cft_mem CHAMP put (/s)^ CHAMP get (/s)^
25716 20210519.7 22345.3 1.27181e+07 13762.4 3.7884e+07 35181.9 1.35046e+07 4992.1 1.03588e+07 6242.47 9.08371e+07 3374.53 1.79442e+08 22289.8 1.71746e+07 29408.7 1.7961e+07 3569.43 1.35046e+07 2226.85 1.19317e+07 1927.99 1.08831e+07 1586.46 8.52384e+06 1.33516e+06 3.61837e+07
25757 20210520.1 28750.5 1.29803e+07 13368.8 3.81461e+07 33061.1 1.32424e+07 4808.4 1.03588e+07 6693.25 9.24099e+07 3384.47 1.63975e+08 22848.2 1.7961e+07 26661.1 1.74367e+07 3864.04 1.45532e+07 2345.45 9.31027e+06 1953.19 1.19317e+07 1609.89 8.2617e+06 1.358e+06 3.62478e+07
25784 20210520.10 28813.3 1.32424e+07 13858.9 3.76218e+07 31661.1 1.32424e+07 4505.42 1.00967e+07 6619.14 9.31964e+07 3217.56 2.03559e+08 22848.3 1.76989e+07 27695.9 1.76989e+07 3958.57 1.40289e+07 2235.92 9.04813e+06 1972.57 8.2617e+06 1587.36 7.99955e+06 1.33602e+06 3.59292e+07
25791 20210520.12 28314.1 1.29803e+07 14255.9 3.84083e+07 33569.7 1.35046e+07 4877.4 1.00967e+07 6454.49 9.08371e+07 3436.58 1.68694e+08 21932.9 1.76989e+07 25918 1.74367e+07 3950.58 1.4291e+07 2343.57 9.04813e+06 1890.49 8.2617e+06 1576.39 8.2617e+06 1.3369e+06 3.59298e+07
25798 20210521.1 29784.3 1.29803e+07 14285.5 3.81461e+07 35436.7 1.35046e+07 5025.47 1.03588e+07 6709.96 9.31964e+07 3445.71 1.54276e+08 22262.4 1.76989e+07 26966.9 1.74367e+07 4331.43 1.45532e+07 2300.82 1.16696e+07 1972.55 8.2617e+06 1574.05 8.2617e+06 1.32986e+06 3.61837e+07
25828 20210524.1 24615.8 1.29803e+07 14315.5 3.86704e+07 28979.7 1.35046e+07 5367.96 1.0621e+07 6603.82 9.24099e+07 3045.14 2.10113e+08 22441.6 1.76989e+07 28015.6 1.76989e+07 3985.86 1.45532e+07 2332.66 9.04813e+06 1978.53 1.08831e+07 1593.94 8.2617e+06 1.33341e+06 3.62478e+07
25892 20210524.17 28415.8 1.29803e+07 13706.4 3.76218e+07 32362.8 1.32424e+07 4932.24 1.03588e+07 6601.99 9.21478e+07 3454.54 1.61354e+08 23448.1 1.71746e+07 29118.9 1.7961e+07 3981.17 1.4291e+07 2309.92 9.31027e+06 1904.21 8.52384e+06 1601.67 8.2617e+06 1.33107e+06 3.61831e+07
25922 20210524.24 26621.4 1.32424e+07 14801 3.94568e+07 32767.8 1.29803e+07 4507.09 1.00967e+07 6500 9.24099e+07 3410.01 1.68956e+08 21317.2 1.76989e+07 26239 1.90096e+07 3974.77 1.50774e+07 2274.32 9.04813e+06 1868.57 7.99955e+06 1543.28 7.99955e+06 1.33263e+06 3.59298e+07
25924 20210525.1 24585 1.32424e+07 14673.4 4.07676e+07 34452.2 1.35046e+07 4677.54 1.0621e+07 6458.61 9.08371e+07 3401.42 1.55849e+08 22346.6 1.76989e+07 27407.5 1.76989e+07 4192.27 1.48153e+07 2290.95 9.04813e+06 1890.92 7.99955e+06 1521.2 8.2617e+06 1.33169e+06 3.62478e+07
25950 20210525.8 29701 1.29803e+07 14828.8 3.7884e+07 28178.7 1.32424e+07 4848.74 1.00967e+07 6754.79 9.34585e+07 3399.8 1.51917e+08 21381 1.71746e+07 28220 1.76989e+07 4354.69 1.48153e+07 2369.51 9.31027e+06 1904.91 1.14074e+07 1576.43 8.2617e+06 1.34014e+06 3.62478e+07
25972 20210525.14 26102.8 1.29803e+07 14213.3 3.84083e+07 33799.1 1.35046e+07 4855.83 1.00967e+07 6533.3 9.34585e+07 3367.65 1.59519e+08 21829.8 1.74367e+07 26614.7 1.76989e+07 4055.18 1.40289e+07 2284.09 9.04813e+06 1961.1 1.08831e+07 1514.05 8.2617e+06 1.33742e+06 3.62478e+07
26015 20210526.1 28345.4 1.32424e+07 13734 3.76218e+07 32508.2 1.32424e+07 4973.26 1.03588e+07 6303.33 9.03128e+07 3375.28 1.62927e+08 22150.4 1.76989e+07 29280.7 1.7961e+07 4373.44 1.53396e+07 2235.5 9.04813e+06 1971.38 8.2617e+06 1522.55 7.99955e+06 1.33506e+06 3.60563e+07
26041 20210526.8 27187.2 1.29803e+07 14503.2 3.89325e+07 30410.8 1.29803e+07 4776.61 1.00967e+07 6340.49 9.29342e+07 3115.24 2.28725e+08 21214.3 1.74367e+07 27321.7 1.82232e+07 3997.13 1.45532e+07 2242.79 9.04813e+06 1963.34 8.2617e+06 1558.47 8.2617e+06 1.34347e+06 3.5993e+07
26078 20210527.1 26100.9 1.27181e+07 14172.7 3.99811e+07 30328 1.32424e+07 4657.02 1.03588e+07 6805.87 9.21478e+07 3368.66 1.68432e+08 23124.7 1.76989e+07 29315.2 1.76989e+07 4183.67 1.50774e+07 2371.3 9.04813e+06 1902.28 1.08831e+07 1607.54 7.73741e+06 1.35998e+06 3.5993e+07
26094 20210527.6 26953.6 1.32424e+07 14258.4 3.89325e+07 34916.1 1.35046e+07 4618.86 1.00967e+07 6753.28 9.21478e+07 3303.02 1.67383e+08 22963.6 1.76989e+07 27526.9 1.76989e+07 4103.29 1.53396e+07 2392.6 9.31027e+06 1889.39 8.52384e+06 1549.12 8.52384e+06 1.34215e+06 3.61837e+07
26129 20210527.16 25978.6 1.29803e+07 14453.1 4.02433e+07 32342.4 1.32424e+07 5380.65 1.0621e+07 6232.94 9.00506e+07 3403.07 1.59781e+08 23205.4 1.74367e+07 27713.7 1.7961e+07 3972.58 1.48153e+07 2195.9 1.19317e+07 1947.44 8.2617e+06 1567.36 8.52384e+06 1.35118e+06 3.59292e+07
26149 20210528.1 27586.1 1.29803e+07 14627 3.91947e+07 34068.6 1.32424e+07 4942.58 1.03588e+07 6661.95 9.31964e+07 3421.13 1.68956e+08 21786 1.71746e+07 28018.7 1.76989e+07 4195.31 1.48153e+07 2269.33 9.31027e+06 1878.19 8.2617e+06 1553.12 8.2617e+06 1.35944e+06 3.53713e+07
26160 20210528.4 26202.4 1.27181e+07 14699.7 3.89325e+07 33541.9 1.35046e+07 5371.54 1.03588e+07 6486.65 9.24099e+07 3411.07 1.79966e+08 24478.4 1.76989e+07 26688 1.74367e+07 4239.8 1.48153e+07 2366.35 9.04813e+06 1909.24 8.2617e+06 1562.45 8.2617e+06 1.32342e+06 3.56788e+07
26178 20210528.9 25372.6 1.29803e+07 14847.2 3.81461e+07 33965.5 1.40289e+07 4860.2 1.16696e+07 6704.92 9.29342e+07 3146.53 1.99627e+08 21755.8 1.71746e+07 27319.4 1.82232e+07 4099.01 1.45532e+07 2331.02 9.04813e+06 1925.04 1.14074e+07 1511.59 8.2617e+06 1.34532e+06 3.62471e+07
26210 20210528.19 27076.8 1.32424e+07 14147 3.89325e+07 33103.1 1.35046e+07 5340.75 1.08831e+07 6559.34 9.21478e+07 3425.24 1.70791e+08 22975 1.74367e+07 27203.1 1.76989e+07 3997.61 1.48153e+07 2374.39 9.04813e+06 1957.65 1.08831e+07 1517.77 7.99955e+06 1.31593e+06 3.61199e+07

images

@achamayou
Copy link
Member

This looks good to me, as discussed it would be easier and preferable to use libuv to manage spawning and reaping processes.

The execution is best triggered in the transaction itself rather than hooks, so long as it is idempotent there will be no functional ill-effects.

@achamayou achamayou marked this pull request as ready for review May 28, 2021 08:25
@achamayou achamayou requested a review from a team as a code owner May 28, 2021 08:25
@achamayou achamayou merged commit 8741072 into microsoft:main May 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants