ci: exclude more files from artifacts, better names for downloaded files #6487
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
UdjinM6
changed the title
*.a and *.o files from artifacts
kwvg
approved these changes
Dec 14, 2024
5 tasks
5 tasks
PastaPastaPasta
added a commit
that referenced
this pull request
Dec 15, 2024
, bitcoin#24391, bitcoin#25338, bitcoin#25422, bitcoin#25493, bitcoin#26826, bitcoin#26814, bitcoin#26832, bitcoin#27418, partial bitcoin#27254 (compat backports) 2ab561d partial bitcoin#27254: Extract util/fs from util/system (Kittywhiskers Van Gogh) 280988b merge bitcoin#27418: Remove windows workaround in authproxy (Kittywhiskers Van Gogh) a497df0 merge bitcoin#26832: move (win) S_* defines into bdb (Kittywhiskers Van Gogh) db676a7 merge bitcoin#26814: remove windows-only compat.h usage in random (Kittywhiskers Van Gogh) 94e6637 merge bitcoin#26826: remove windows-only compat.h usage in randomenv (Kittywhiskers Van Gogh) bbb0cce merge bitcoin#25493: document code in compat.h (Kittywhiskers Van Gogh) 3f14309 merge bitcoin#25422: globally define NOMINMAX when building with mingw-w64 (Kittywhiskers Van Gogh) 000495d merge bitcoin#25338: Avoid incompatibility with CMake AUTOUIC feature (Kittywhiskers Van Gogh) 714ea55 build: migrate stacktrace-related flags to `DEBUG_`{`C`,`CXX`}`FLAGS` (Kittywhiskers Van Gogh) 11323c3 merge bitcoin#24391: stop overriding user autoconf flags (Kittywhiskers Van Gogh) 548121d merge bitcoin#24704: remove strnlen back-compat code (Kittywhiskers Van Gogh) b65038e merge bitcoin#23005: Delay wallet client construction (Kittywhiskers Van Gogh) f3b065c merge bitcoin#23092: Remove Windows workaround in authproxy (WinError 10053) (Kittywhiskers Van Gogh) Pull request description: ## Additional Information * This PR originally included [bitcoin#27378](bitcoin#27378) but including it caused multiple tests to fail ([build](https://gitlab.com/dashpay/dash/-/jobs/8577987174#L2224)), even after adjusting `timeout`. It has been omitted instead. <details> <summary>Test failures:</summary> ``` feature_abortnode.py | Failed | 201 s feature_dip3_v19.py | Failed | 237 s feature_llmq_rotation.py | Failed | 54 s p2p_invalid_messages.py | Failed | 5 s rpc_bind.py --nonloopback | Failed | 63 s wallet_multiwallet.py --descriptors | Failed | 79 s wallet_multiwallet.py --legacy-wallet | Failed | 89 s ALL | Failed | 8540 s (accumulated) ``` </details> <details> <summary>feature_abortnode.py failure:</summary> ``` dash@107cfae2b2b5:/src/dash$ ./test/functional/feature_abortnode.py 2024-12-09T18:31:01.776000Z TestFramework (INFO): PRNG seed is: 6843309100291447752 2024-12-09T18:31:01.776000Z TestFramework (INFO): Initializing test directory /tmp/dash_func_test_fcijb99n 2024-12-09T18:31:02.802000Z TestFramework (INFO): Waiting for crash 2024-12-09T18:34:23.026000Z TestFramework.utils (ERROR): wait_until() failed. Predicate: '''' def is_node_stopped(self): """Checks whether the node has stopped. Returns True if the node has stopped. False otherwise. This method is responsible for freeing resources (self.process).""" if not self.running: return True return_code = self.process.poll() if return_code is None: return False # process has stopped. Assert that it didn't return an error code. assert return_code == 0, self._node_msg( "Node returned non-zero exit code (%d) when stopping" % return_code) self.running = False self.process = None self.rpc_connected = False self.rpc = None self.log.debug("Node stopped") return True ''' 2024-12-09T18:34:23.027000Z TestFramework (ERROR): Assertion failed Traceback (most recent call last): File "/src/dash/test/functional/test_framework/test_framework.py", line 161, in main self.run_test() File "/src/dash/./test/functional/feature_abortnode.py", line 44, in run_test self.nodes[0].wait_until_stopped(timeout=200) File "/src/dash/test/functional/test_framework/test_node.py", line 414, in wait_until_stopped wait_until_helper(self.is_node_stopped, timeout=timeout, timeout_factor=self.timeout_factor) File "/src/dash/test/functional/test_framework/util.py", line 287, in wait_until_helper raise AssertionError("Predicate {} not true after {} seconds".format(predicate_source, timeout)) AssertionError: Predicate '''' def is_node_stopped(self): """Checks whether the node has stopped. Returns True if the node has stopped. False otherwise. This method is responsible for freeing resources (self.process).""" if not self.running: return True return_code = self.process.poll() if return_code is None: return False # process has stopped. Assert that it didn't return an error code. assert return_code == 0, self._node_msg( "Node returned non-zero exit code (%d) when stopping" % return_code) self.running = False self.process = None self.rpc_connected = False self.rpc = None self.log.debug("Node stopped") return True ''' not true after 200.0 seconds 2024-12-09T18:34:23.530000Z TestFramework (INFO): Stopping nodes 2024-12-09T18:34:23.530000Z TestFramework (ERROR): Unexpected exception caught during shutdown Traceback (most recent call last): File "/src/dash/test/functional/test_framework/test_framework.py", line 341, in shutdown self.stop_nodes() File "/src/dash/test/functional/test_framework/test_framework.py", line 673, in stop_nodes node.stop_node(expected_stderr=expected_stderr, wait=wait, wait_until_stopped=False) File "/src/dash/test/functional/test_framework/test_node.py", line 368, in stop_node self.stop(wait=wait) File "/src/dash/test/functional/test_framework/coverage.py", line 49, in __call__ return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs) File "/src/dash/test/functional/test_framework/authproxy.py", line 124, in __call__ response, status = self._request('POST', self.__url.path, postdata.encode('utf-8')) File "/src/dash/test/functional/test_framework/authproxy.py", line 103, in _request return self._get_response() File "/src/dash/test/functional/test_framework/authproxy.py", line 162, in _get_response raise JSONRPCException( test_framework.authproxy.JSONRPCException: non-JSON HTTP response with '503 Service Unavailable' from server (-342) 2024-12-09T18:34:23.531000Z TestFramework (WARNING): Not cleaning up dir /tmp/dash_func_test_fcijb99n 2024-12-09T18:34:23.531000Z TestFramework (ERROR): Test failed. Test logging available at /tmp/dash_func_test_fcijb99n/test_framework.log 2024-12-09T18:34:23.531000Z TestFramework (ERROR): 2024-12-09T18:34:23.531000Z TestFramework (ERROR): Hint: Call /src/dash/test/functional/combine_logs.py '/tmp/dash_func_test_fcijb99n' to consolidate all logs 2024-12-09T18:34:23.531000Z TestFramework (ERROR): 2024-12-09T18:34:23.531000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log. 2024-12-09T18:34:23.531000Z TestFramework (ERROR): https://github.com/dashpay/dash/issues 2024-12-09T18:34:23.531000Z TestFramework (ERROR): [node 1] Cleaning up leftover process [node 0] Cleaning up leftover process ``` </details> * [bitcoin#27254](bitcoin#27254) has been partially backported to include a missing header to allow it to build. <details> <summary>Compile error:</summary> ``` In file included from ./util/system.h:19, from util/system.cpp:7: ./compat/assumptions.h:49:22: error: 'size_t' was not declared in this scope; did you mean 'std::size_t'? 49 | static_assert(sizeof(size_t) == 4 || sizeof(size_t) == 8, "size_t assumed to be 32-bit or 64-bit"); | ^~~~~~ | std::size_t In file included from /usr/include/c++/11/limits:42, from ./compat/assumptions.h:11, from ./util/system.h:19, from util/system.cpp:7: /usr/include/x86_64-linux-gnu/c++/11/bits/c++config.h:280:33: note: 'std::size_t' declared here 280 | typedef __SIZE_TYPE__ size_t; | ^~~~~~ ``` </details> * ~~GitLab job failures as of 2339ae6, for `linux64-build` ([source](https://gitlab.com/dashpay/dash/-/jobs/8590291130#L4621)), `linux64_cxx20-build` ([source](https://gitlab.com/dashpay/dash/-/jobs/8590291134#L3497)) and `linux64_sqlite-build` ([source](https://gitlab.com/dashpay/dash/-/jobs/8590291137#L3489)) are not related to failures in compilation or unit tests but are to do with reaching the artifact limit.~~ Resolved with [dash#6487](#6487), thanks Udjin! ## Breaking Changes None expected. ## Checklist - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)** - [x] I have added or updated relevant unit/integration/functional/e2e tests - [x] I have made corresponding changes to the documentation **(note: N/A)** - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_ ACKs for top commit: UdjinM6: utACK 2ab561d PastaPastaPasta: utACK 2ab561d Tree-SHA512: 6d05305f17fd607ed93b4a2e00de195a96bde1a34bedf04f6dd84892dc71028ae71590fa5063026ec7ff728d6e411af410c363b4a88947ce6e5999d4431bb29b
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue being fixed or feature implemented
We include too many files in artifacts on
buildci step, some of which (*.aand*.o) can be pretty heavy. This was ok-ish for some time but artifacts size is getting closer to the limit and even starts to cause issues, see #6462.What was done?
Exclude
*.aand*.ofiles from artifacts. Also, change artifacts name to make it easier to distinguish them when you get a few of them from the same pipeline -build-arm-linux.zip,build-linux64.zipetc. instead ofbinaries.zip,binaries (1).zipetc., same fordepends.A simpler alternative to #6483
As a result the size of tsan artifacts for example is down from 508MB in 6462 to 154MB in this PR.
How Has This Been Tested?
Breaking Changes
Checklist: