-
Notifications
You must be signed in to change notification settings - Fork 269
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Disallow invalid pointers in arrays and tuples #226
Conversation
401a381
to
9f99b5d
Compare
915b739
to
24afe35
Compare
90ee2b7
to
f7fcbd8
Compare
2eeaa82
to
bf5cf4e
Compare
03f069c
to
d285844
Compare
a0ad898
to
04939a6
Compare
descriptive comments
04939a6
to
1681383
Compare
@@ -131,6 +132,13 @@ def __call__(self, stream: ContextFramesBytesIO) -> Any: | |||
|
|||
|
|||
class HeadTailDecoder(BaseDecoder): | |||
""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💯
@to_tuple # type: ignore[misc] # untyped decorator | ||
def decode(self, stream: ContextFramesBytesIO) -> Generator[Any, None, None]: | ||
self.validate_pointers(stream) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I could use more context here. Could this be called in the loop below and maybe allow removal of the inner decoder loops inside validate_pointers
? I'm also curious if the validation is necessary before decoding? Could validation just be part of the decode
in HeadTailDecoder
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is no way to know how long the head section of a dynamic tuple will be until you have stepped through each decoder - if the decoder is for a dynamic type, it will be 32 bytes every time (because it's a pointer), but if it's for a non-dynamic array, there will be a single decoder for multiple chunks of 32 bytes.
I think it would be possible to take the logic from validate_pointers
and put it in decode
to eliminate the second loop through the decoders (where it actually checks the pointer values against the end_of_offsets
). I like the current clarity and separation of concerns, but I can try if you like.
The validation needs to be in the tuple and array decoders, because only they have the context for how long they are. A HeadTailDecoder
only has the info for a single dynamic value.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see now what the difference means, assuming there may never be more than a few decoders at a time I don't have any concerns.
@to_tuple # type: ignore[misc] # untyped decorator | ||
def decode(self, stream: ContextFramesBytesIO) -> Generator[Any, None, None]: | ||
self.validate_pointers(stream) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see now what the difference means, assuming there may never be more than a few decoders at a time I don't have any concerns.
end_of_offsets = current_location + 32 * len_of_head | ||
total_stream_length = len(stream.getbuffer()) | ||
for decoder in self.decoders: | ||
if isinstance(decoder, HeadTailDecoder): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: It would be nice to share this logic across decoders, maybe this could become a utility function that could take the stream
and an array_size
, which could be called from here using array_size=1
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit heard and politely declined. There is enough required difference in how tuples and arrays are checked that any logic extraction have a lot of if tuple/elseif array
. And I don't foresee any future datastructures being created that would make use of such shared base methods, thus accept code that is ~repeated twice.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me! Nice work tracking it down! 🐞 I like the comments you made in the decoder too. Very helpful.
Bumps the pip group with 3 updates in the / directory: [eth-abi](https://github.com/ethereum/eth-abi), [requests](https://github.com/psf/requests) and [setuptools](https://github.com/pypa/setuptools). Updates `eth-abi` from 2.2.0 to 5.0.1 <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/ethereum/eth-abi/blob/main/docs/release_notes.rst">eth-abi's changelog</a>.</em></p> <blockquote> <h2>eth-abi v5.0.1 (2024-03-04)</h2> <p>Bugfixes</p> <pre><code> - During decoding, verify all pointers in arrays and tuples point to a valid location in the payload (`[#226](ethereum/eth-abi#226) <https://github.com/ethereum/eth-abi/issues/226>`__) - Fix memory leak warning in NodeVisitor and ABIRegistry (`[#230](ethereum/eth-abi#230) <https://github.com/ethereum/eth-abi/issues/230>`__) <p>Internal Changes - for eth-abi Contributors </code></pre></p> <ul> <li>Clear mypy <code>misc</code>-type errors and add top-level <code>py.typed</code> file back (<code>[#221](ethereum/eth-abi#221) <https://github.com/ethereum/eth-abi/issues/221></code>__)</li> </ul> <h2>eth-abi v5.0.0 (2024-01-09)</h2> <p>Breaking Changes</p> <pre><code> - Drop python 3.7 support (`[#217](ethereum/eth-abi#217) <https://github.com/ethereum/eth-abi/issues/217>`__) <p>Internal Changes - for eth-abi Contributors </code></pre></p> <ul> <li>Merge updates from the project template, notably, use <code>pre-commit</code> for linting and change the name of the <code>master</code> branch to <code>main</code> (<code>[#217](ethereum/eth-abi#217) <https://github.com/ethereum/eth-abi/issues/217></code>__)</li> <li>Corrected format of booleans in <code>pyproject.toml</code> and added a test for the presence of the <code>eth_abi.__version__</code> attribute (<code>[#219](ethereum/eth-abi#219) <https://github.com/ethereum/eth-abi/issues/219></code>__)</li> </ul> <h2>eth-abi v4.2.1 (2023-09-13)</h2> <p>Internal Changes - for eth-abi contributors</p> <pre><code> - Add ``build.os`` section to readthedocs build settings (`[#213](ethereum/eth-abi#213) <https://github.com/ethereum/eth-abi/issues/213>`__) <p>Miscellaneous changes</p> <pre><code> - `[#214](ethereum/eth-abi#214) &lt;https://github.com/ethereum/eth-abi/issues/214&gt;`__ eth-abi v4.2.0 (2023-08-28) --------------------------- &lt;/tr&gt;&lt;/table&gt; </code></pre> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="ethereum/eth-abi@be95178fda29e50c2378163cd2239abe842d8bbd"><code>be95178</code></a> Bump version: 5.0.0 → 5.0.1</li> <li><a href="ethereum/eth-abi@00f990ed135cf203f1407bf05184cfc9f336ba3f"><code>00f990e</code></a> Compile release notes for v5.0.1</li> <li><a href="ethereum/eth-abi@82c1ad37a866472562d81fedaef0f4fed0a08269"><code>82c1ad3</code></a> add validate_pointers for tuples and arrays, add tests and more</li> <li><a href="ethereum/eth-abi@75aab7cdcc2a1200b0cebaaee84d0d816f9147be"><code>75aab7c</code></a> remove warning of ABIv2 being unstable</li> <li><a href="ethereum/eth-abi@6063bf1f3f1f6fc0e0e74e4d6e7f86e62fc850b2"><code>6063bf1</code></a> add newsfragments file</li> <li><a href="ethereum/eth-abi@2367eef95b5b28945bf912ae56640682e67ff1da"><code>2367eef</code></a> 🚨 fix linting warning (rule B019)</li> <li><a href="ethereum/eth-abi@abbe2100a5e7d8d0913e83632fffd77ea2ab75a1"><code>abbe210</code></a> type codec.py and exceptions, change mypy ignore from files to error types, a...</li> <li><a href="ethereum/eth-abi@d063fc6e9195f20d9c42a88dde3f6e4b3319f067"><code>d063fc6</code></a> Bump version: 4.2.1 → 5.0.0</li> <li><a href="ethereum/eth-abi@7641e321c89b0ebba355830819036bacbb5072db"><code>7641e32</code></a> Compile release notes for v5.0.0</li> <li><a href="ethereum/eth-abi@31866a392ed3ac376ea00d8ab57e4a5f4818ceee"><code>31866a3</code></a> fix toml bools and add version test</li> <li>Additional commits viewable in <a href="ethereum/eth-abi@v2.2.0...v5.0.1">compare view</a></li> </ul> </details> <br /> </code></pre> Updates `requests` from 2.28.2 to 2.32.2 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/psf/requests/releases">requests's releases</a>.</em></p> <blockquote> <h2>v2.32.2</h2> <h2>2.32.2 (2024-05-21)</h2> <p><strong>Deprecations</strong></p> <ul> <li> <p>To provide a more stable migration for custom HTTPAdapters impacted by the CVE changes in 2.32.0, we've renamed <code>_get_connection</code> to a new public API, <code>get_connection_with_tls_context</code>. Existing custom HTTPAdapters will need to migrate their code to use this new API. <code>get_connection</code> is considered deprecated in all versions of Requests>=2.32.0.</p> <p>A minimal (2-line) example has been provided in the linked PR to ease migration, but we strongly urge users to evaluate if their custom adapter is subject to the same issue described in CVE-2024-35195. (<a href="https://redirect.github.com/psf/requests/issues/6710">#6710</a>)</p> </li> </ul> <h2>v2.32.1</h2> <h2>2.32.1 (2024-05-20)</h2> <p><strong>Bugfixes</strong></p> <ul> <li>Add missing test certs to the sdist distributed on PyPI.</li> </ul> <h2>v2.32.0</h2> <h2>2.32.0 (2024-05-20)</h2> <h2>🐍 PYCON US 2024 EDITION 🐍</h2> <p><strong>Security</strong></p> <ul> <li>Fixed an issue where setting <code>verify=False</code> on the first request from a Session will cause subsequent requests to the <em>same origin</em> to also ignore cert verification, regardless of the value of <code>verify</code>. (<a href="https://github.com/psf/requests/security/advisories/GHSA-9wx4-h78v-vm56">https://github.com/psf/requests/security/advisories/GHSA-9wx4-h78v-vm56</a>)</li> </ul> <p><strong>Improvements</strong></p> <ul> <li><code>verify=True</code> now reuses a global SSLContext which should improve request time variance between first and subsequent requests. It should also minimize certificate load time on Windows systems when using a Python version built with OpenSSL 3.x. (<a href="https://redirect.github.com/psf/requests/issues/6667">#6667</a>)</li> <li>Requests now supports optional use of character detection (<code>chardet</code> or <code>charset_normalizer</code>) when repackaged or vendored. This enables <code>pip</code> and other projects to minimize their vendoring surface area. The <code>Response.text()</code> and <code>apparent_encoding</code> APIs will default to <code>utf-8</code> if neither library is present. (<a href="https://redirect.github.com/psf/requests/issues/6702">#6702</a>)</li> </ul> <p><strong>Bugfixes</strong></p> <ul> <li>Fixed bug in length detection where emoji length was incorrectly calculated in the request content-length. (<a href="https://redirect.github.com/psf/requests/issues/6589">#6589</a>)</li> <li>Fixed deserialization bug in JSONDecodeError. (<a href="https://redirect.github.com/psf/requests/issues/6629">#6629</a>)</li> <li>Fixed bug where an extra leading <code>/</code> (path separator) could lead urllib3 to unnecessarily reparse the request URI. (<a href="https://redirect.github.com/psf/requests/issues/6644">#6644</a>)</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/psf/requests/blob/main/HISTORY.md">requests's changelog</a>.</em></p> <blockquote> <h2>2.32.2 (2024-05-21)</h2> <p><strong>Deprecations</strong></p> <ul> <li> <p>To provide a more stable migration for custom HTTPAdapters impacted by the CVE changes in 2.32.0, we've renamed <code>_get_connection</code> to a new public API, <code>get_connection_with_tls_context</code>. Existing custom HTTPAdapters will need to migrate their code to use this new API. <code>get_connection</code> is considered deprecated in all versions of Requests>=2.32.0.</p> <p>A minimal (2-line) example has been provided in the linked PR to ease migration, but we strongly urge users to evaluate if their custom adapter is subject to the same issue described in CVE-2024-35195. (<a href="https://redirect.github.com/psf/requests/issues/6710">#6710</a>)</p> </li> </ul> <h2>2.32.1 (2024-05-20)</h2> <p><strong>Bugfixes</strong></p> <ul> <li>Add missing test certs to the sdist distributed on PyPI.</li> </ul> <h2>2.32.0 (2024-05-20)</h2> <p><strong>Security</strong></p> <ul> <li>Fixed an issue where setting <code>verify=False</code> on the first request from a Session will cause subsequent requests to the <em>same origin</em> to also ignore cert verification, regardless of the value of <code>verify</code>. (<a href="https://github.com/psf/requests/security/advisories/GHSA-9wx4-h78v-vm56">https://github.com/psf/requests/security/advisories/GHSA-9wx4-h78v-vm56</a>)</li> </ul> <p><strong>Improvements</strong></p> <ul> <li><code>verify=True</code> now reuses a global SSLContext which should improve request time variance between first and subsequent requests. It should also minimize certificate load time on Windows systems when using a Python version built with OpenSSL 3.x. (<a href="https://redirect.github.com/psf/requests/issues/6667">#6667</a>)</li> <li>Requests now supports optional use of character detection (<code>chardet</code> or <code>charset_normalizer</code>) when repackaged or vendored. This enables <code>pip</code> and other projects to minimize their vendoring surface area. The <code>Response.text()</code> and <code>apparent_encoding</code> APIs will default to <code>utf-8</code> if neither library is present. (<a href="https://redirect.github.com/psf/requests/issues/6702">#6702</a>)</li> </ul> <p><strong>Bugfixes</strong></p> <ul> <li>Fixed bug in length detection where emoji length was incorrectly calculated in the request content-length. (<a href="https://redirect.github.com/psf/requests/issues/6589">#6589</a>)</li> <li>Fixed deserialization bug in JSONDecodeError. (<a href="https://redirect.github.com/psf/requests/issues/6629">#6629</a>)</li> <li>Fixed bug where an extra leading <code>/</code> (path separator) could lead urllib3 to unnecessarily reparse the request URI. (<a href="https://redirect.github.com/psf/requests/issues/6644">#6644</a>)</li> </ul> <p><strong>Deprecations</strong></p> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/psf/requests/commit/88dce9d854797c05d0ff296b70e0430535ef8aaf"><code>88dce9d</code></a> v2.32.2</li> <li><a href="https://github.com/psf/requests/commit/c98e4d133ef29c46a9b68cd783087218a8075e05"><code>c98e4d1</code></a> Merge pull request <a href="https://redirect.github.com/psf/requests/issues/6710">#6710</a> from nateprewitt/api_rename</li> <li><a href="https://github.com/psf/requests/commit/92075b330a30b9883f466a43d3f7566ab849f91b"><code>92075b3</code></a> Add deprecation warning</li> <li><a href="https://github.com/psf/requests/commit/aa1461b68aa73e2f6ec0e78c8853b635c76fd099"><code>aa1461b</code></a> Move _get_connection to get_connection_with_tls_context</li> <li><a href="https://github.com/psf/requests/commit/970e8cec988421bd43da57350723b05c8ce8dc7e"><code>970e8ce</code></a> v2.32.1</li> <li><a href="https://github.com/psf/requests/commit/d6ebc4a2f1f68b7e355fb7e4dd5ffc0845547f9f"><code>d6ebc4a</code></a> v2.32.0</li> <li><a href="https://github.com/psf/requests/commit/9a40d1277807f0a4f26c9a37eea8ec90faa8aadc"><code>9a40d12</code></a> Avoid reloading root certificates to improve concurrent performance (<a href="https://redirect.github.com/psf/requests/issues/6667">#6667</a>)</li> <li><a href="https://github.com/psf/requests/commit/0c030f78d24f29a459dbf39b28b4cc765e2153d7"><code>0c030f7</code></a> Merge pull request <a href="https://redirect.github.com/psf/requests/issues/6702">#6702</a> from nateprewitt/no_char_detection</li> <li><a href="https://github.com/psf/requests/commit/555b870eb19d497ddb67042645420083ec8efb02"><code>555b870</code></a> Allow character detection dependencies to be optional in post-packaging steps</li> <li><a href="https://github.com/psf/requests/commit/d6dded3f00afcf56a7e866cb0732799045301eb0"><code>d6dded3</code></a> Merge pull request <a href="https://redirect.github.com/psf/requests/issues/6700">#6700</a> from franekmagiera/update-redirect-to-invalid-uri-test</li> <li>Additional commits viewable in <a href="https://github.com/psf/requests/compare/v2.28.2...v2.32.2">compare view</a></li> </ul> </details> <br /> Updates `setuptools` from 67.2.0 to 70.0.0 <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pypa/setuptools/blob/main/NEWS.rst">setuptools's changelog</a>.</em></p> <blockquote> <h1>v70.0.0</h1> <h2>Features</h2> <ul> <li>Emit a warning when <code>[tools.setuptools]</code> is present in <code>pyproject.toml</code> and will be ignored. -- by :user:<code>SnoopJ</code> (<a href="https://redirect.github.com/pypa/setuptools/issues/4150">#4150</a>)</li> <li>Improved <code>AttributeError</code> error message if <code>pkg_resources.EntryPoint.require</code> is called without extras or distribution Gracefully "do nothing" when trying to activate a <code>pkg_resources.Distribution</code> with a <code>None</code> location, rather than raising a <code>TypeError</code> -- by :user:<code>Avasam</code> (<a href="https://redirect.github.com/pypa/setuptools/issues/4262">#4262</a>)</li> <li>Typed the dynamically defined variables from <code>pkg_resources</code> -- by :user:<code>Avasam</code> (<a href="https://redirect.github.com/pypa/setuptools/issues/4267">#4267</a>)</li> <li>Modernized and refactored VCS handling in package_index. (<a href="https://redirect.github.com/pypa/setuptools/issues/4332">#4332</a>)</li> </ul> <h2>Bugfixes</h2> <ul> <li>In install command, use super to call the superclass methods. Avoids race conditions when monkeypatching from _distutils_system_mod occurs late. (<a href="https://redirect.github.com/pypa/setuptools/issues/4136">#4136</a>)</li> <li>Fix finder template for lenient editable installs of implicit nested namespaces constructed by using <code>package_dir</code> to reorganise directory structure. (<a href="https://redirect.github.com/pypa/setuptools/issues/4278">#4278</a>)</li> <li>Fix an error with <code>UnicodeDecodeError</code> handling in <code>pkg_resources</code> when trying to read files in UTF-8 with a fallback -- by :user:<code>Avasam</code> (<a href="https://redirect.github.com/pypa/setuptools/issues/4348">#4348</a>)</li> </ul> <h2>Improved Documentation</h2> <ul> <li>Uses RST substitution to put badges in 1 line. (<a href="https://redirect.github.com/pypa/setuptools/issues/4312">#4312</a>)</li> </ul> <h2>Deprecations and Removals</h2> <ul> <li> <p>Further adoption of UTF-8 in <code>setuptools</code>. This change regards mostly files produced and consumed during the build process (e.g. metadata files, script wrappers, automatically updated config files, etc..) Although precautions were taken to minimize disruptions, some edge cases might be subject to backwards incompatibility.</p> <p>Support for <code>"locale"</code> encoding is now <strong>deprecated</strong>. (<a href="https://redirect.github.com/pypa/setuptools/issues/4309">#4309</a>)</p> </li> <li> <p>Remove <code>setuptools.convert_path</code> after long deprecation period. This function was never defined by <code>setuptools</code> itself, but rather a side-effect of an import for internal usage. (<a href="https://redirect.github.com/pypa/setuptools/issues/4322">#4322</a>)</p> </li> <li> <p>Remove fallback for customisations of <code>distutils</code>' <code>build.sub_command</code> after long deprecated period. Users are advised to import <code>build</code> directly from <code>setuptools.command.build</code>. (<a href="https://redirect.github.com/pypa/setuptools/issues/4322">#4322</a>)</p> </li> <li> <p>Removed <code>typing_extensions</code> from vendored dependencies -- by :user:<code>Avasam</code> (<a href="https://redirect.github.com/pypa/setuptools/issues/4324">#4324</a>)</p> </li> <li> <p>Remove deprecated <code>setuptools.dep_util</code>. The provided alternative is <code>setuptools.modified</code>. (<a href="https://redirect.github.com/pypa/setuptools/issues/4360">#4360</a>)</p> </li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pypa/setuptools/commit/5cbf12a9b63fd37985a4525617b46576b8ac3a7b"><code>5cbf12a</code></a> Workaround for release error in v70</li> <li><a href="https://github.com/pypa/setuptools/commit/9c1bcc3417bd12668123f7e731e241d9e57bfc57"><code>9c1bcc3</code></a> Bump version: 69.5.1 → 70.0.0</li> <li><a href="https://github.com/pypa/setuptools/commit/4dc0c31644b458ac43ce6148f6a9dc729a7e78b5"><code>4dc0c31</code></a> Remove deprecated <code>setuptools.dep_util</code> (<a href="https://redirect.github.com/pypa/setuptools/issues/4360">#4360</a>)</li> <li><a href="https://github.com/pypa/setuptools/commit/6c1ef5748dbd70c8c5423e12680345766ee101d9"><code>6c1ef57</code></a> Remove xfail now that test passes. Ref <a href="https://redirect.github.com/pypa/setuptools/issues/4371">#4371</a>.</li> <li><a href="https://github.com/pypa/setuptools/commit/d14fa0162c95450898c11534caf26a0f03553176"><code>d14fa01</code></a> Add all site-packages dirs when creating simulated environment for test_edita...</li> <li><a href="https://github.com/pypa/setuptools/commit/6b7f7a18afc90007544092c446dc0cd856d86b17"><code>6b7f7a1</code></a> Prevent <code>bin</code> folders to be taken as extern packages when vendoring (<a href="https://redirect.github.com/pypa/setuptools/issues/4370">#4370</a>)</li> <li><a href="https://github.com/pypa/setuptools/commit/69141f69f8bf38da34cbea552d6fdaa9c8619c53"><code>69141f6</code></a> Add doctest for vendorised bin folder</li> <li><a href="https://github.com/pypa/setuptools/commit/2a53cc1200ec4b14e08e84be3c042f8983dfb7d7"><code>2a53cc1</code></a> Prevent 'bin' folders to be taken as extern packages</li> <li><a href="https://github.com/pypa/setuptools/commit/720862807dea012f3a0e7061880691025f736f11"><code>7208628</code></a> Replace call to deprecated <code>validate_pyproject</code> command (<a href="https://redirect.github.com/pypa/setuptools/issues/4363">#4363</a>)</li> <li><a href="https://github.com/pypa/setuptools/commit/96d681aa405460f724c62c00ca125ae722ad810a"><code>96d681a</code></a> Remove call to deprecated validate_pyproject command</li> <li>Additional commits viewable in <a href="https://github.com/pypa/setuptools/compare/v67.2.0...v70.0.0">compare view</a></li> </ul> </details> <br /> Updates `urllib3` from 1.26.14 to 2.2.3 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/urllib3/urllib3/releases">urllib3's releases</a>.</em></p> <blockquote> <h2>2.2.3</h2> <h2>🚀 urllib3 is fundraising for HTTP/2 support</h2> <p><a href="https://sethmlarson.dev/urllib3-is-fundraising-for-http2-support">urllib3 is raising ~$40,000 USD</a> to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support for 2023. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects <a href="https://opencollective.com/urllib3">please consider contributing financially</a> to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.</p> <p>Thank you for your support.</p> <h2>Features</h2> <ul> <li>Added support for Python 3.13. (<a href="https://redirect.github.com/urllib3/urllib3/issues/3473">#3473</a>)</li> </ul> <h2>Bugfixes</h2> <ul> <li>Fixed the default encoding of chunked request bodies to be UTF-8 instead of ISO-8859-1. All other methods of supplying a request body already use UTF-8 starting in urllib3 v2.0. (<a href="https://redirect.github.com/urllib3/urllib3/issues/3053">#3053</a>)</li> <li>Fixed ResourceWarning on CONNECT with Python < 3.11.4 by backporting <a href="https://redirect.github.com/python/cpython/issues/103472">python/cpython#103472</a>. (`<a href="https://redirect.github.com/urllib3/urllib3/issues/3252">#3252</a>)</li> <li>Adjust tolerance for floating-point comparison on Windows to avoid flakiness in CI (<a href="https://redirect.github.com/urllib3/urllib3/issues/3413">#3413</a>)</li> <li>Fixed a crash where certain standard library hash functions were absent in restricted environments. (<a href="https://redirect.github.com/urllib3/urllib3/issues/3432">#3432</a>)</li> <li>Fixed mypy error when adding to <code>HTTPConnection.default_socket_options</code>. (<a href="https://redirect.github.com/urllib3/urllib3/issues/3448">#3448</a>)</li> </ul> <h2>HTTP/2 (experimental)</h2> <p>HTTP/2 support is still in early development.</p> <ul> <li>Excluded Transfer-Encoding: chunked from HTTP/2 request body (<a href="https://redirect.github.com/urllib3/urllib3/issues/3425">#3425</a>)</li> <li>Added version checking for <code>h2</code> (<a href="https://pypi.org/project/h2/">https://pypi.org/project/h2/</a>) usage. Now only accepting supported h2 major version 4.x.x. (<a href="https://redirect.github.com/urllib3/urllib3/issues/3290">#3290</a>)</li> <li>Added a probing mechanism for determining whether a given target origin supports HTTP/2 via ALPN. (<a href="https://redirect.github.com/urllib3/urllib3/issues/3301">#3301</a>)</li> <li>Add support for sending a request body with HTTP/2 (<a href="https://redirect.github.com/urllib3/urllib3/issues/3302">#3302</a>)</li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/urllib3/urllib3/compare/2.2.2...2.2.3">https://github.com/urllib3/urllib3/compare/2.2.2...2.2.3</a></p> <h2>2.2.2</h2> <h2>🚀 urllib3 is fundraising for HTTP/2 support</h2> <p><a href="https://sethmlarson.dev/urllib3-is-fundraising-for-http2-support">urllib3 is raising ~$40,000 USD</a> to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support for 2023. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects <a href="https://opencollective.com/urllib3">please consider contributing financially</a> to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.</p> <p>Thank you for your support.</p> <h2>Changes</h2> <ul> <li>Added the <code>Proxy-Authorization</code> header to the list of headers to strip from requests when redirecting to a different host. As before, different headers can be set via <code>Retry.remove_headers_on_redirect</code>.</li> <li>Allowed passing negative integers as <code>amt</code> to read methods of <code>http.client.HTTPResponse</code> as an alternative to <code>None</code>. (<a href="https://redirect.github.com/urllib3/urllib3/issues/3122">#3122</a>)</li> <li>Fixed return types representing copying actions to use <code>typing.Self</code>. (<a href="https://redirect.github.com/urllib3/urllib3/issues/3363">#3363</a>)</li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/urllib3/urllib3/compare/2.2.1...2.2.2">https://github.com/urllib3/urllib3/compare/2.2.1...2.2.2</a></p> <h2>2.2.1</h2> <h2>🚀 urllib3 is fundraising for HTTP/2 support</h2> <p><a href="https://sethmlarson.dev/urllib3-is-fundraising-for-http2-support">urllib3 is raising ~$40,000 USD</a> to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support for 2023. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects <a href="https://opencollective.com/urllib3">please consider contributing financially</a> to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.</p> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/urllib3/urllib3/blob/main/CHANGES.rst">urllib3's changelog</a>.</em></p> <blockquote> <h1>2.2.3 (2024-09-12)</h1> <h2>Features</h2> <ul> <li>Added support for Python 3.13. (<code>[#3473](urllib3/urllib3#3473) <https://github.com/urllib3/urllib3/issues/3473></code>__)</li> </ul> <h2>Bugfixes</h2> <ul> <li>Fixed the default encoding of chunked request bodies to be UTF-8 instead of ISO-8859-1. All other methods of supplying a request body already use UTF-8 starting in urllib3 v2.0. (<code>[#3053](urllib3/urllib3#3053) <https://github.com/urllib3/urllib3/issues/3053></code>__)</li> <li>Fixed ResourceWarning on CONNECT with Python <!-- raw HTML omitted -->`__)</li> <li>Adjust tolerance for floating-point comparison on Windows to avoid flakiness in CI (<code>[#3413](urllib3/urllib3#3413) <https://github.com/urllib3/urllib3/issues/3413></code>__)</li> <li>Fixed a crash where certain standard library hash functions were absent in restricted environments. (<code>[#3432](urllib3/urllib3#3432) <https://github.com/urllib3/urllib3/issues/3432></code>__)</li> <li>Fixed mypy error when adding to <code>HTTPConnection.default_socket_options</code>. (<code>[#3448](urllib3/urllib3#3448) <https://github.com/urllib3/urllib3/issues/3448></code>__)</li> </ul> <h2>HTTP/2 (experimental)</h2> <p>HTTP/2 support is still in early development.</p> <ul> <li> <p>Excluded Transfer-Encoding: chunked from HTTP/2 request body (<code>[#3425](urllib3/urllib3#3425) <https://github.com/urllib3/urllib3/issues/3425></code>__)</p> </li> <li> <p>Added version checking for <code>h2</code> (<a href="https://pypi.org/project/h2/">https://pypi.org/project/h2/</a>) usage.</p> <p>Now only accepting supported h2 major version 4.x.x. (<code>[#3290](urllib3/urllib3#3290) <https://github.com/urllib3/urllib3/issues/3290></code>__)</p> </li> <li> <p>Added a probing mechanism for determining whether a given target origin supports HTTP/2 via ALPN. (<code>[#3301](urllib3/urllib3#3301) <https://github.com/urllib3/urllib3/issues/3301></code>__)</p> </li> <li> <p>Add support for sending a request body with HTTP/2 (<code>[#3302](urllib3/urllib3#3302) <https://github.com/urllib3/urllib3/issues/3302></code>__)</p> </li> </ul> <h2>Deprecations and Removals</h2> <ul> <li>Note for downstream distributors: the <code>_version.py</code> file has been removed and is now created at build time by hatch-vcs. (<code>[#3412](urllib3/urllib3#3412) <https://github.com/urllib3/urllib3/issues/3412></code>__)</li> <li>Drop support for end-of-life PyPy3.8 and PyPy3.9. (<code>[#3475](urllib3/urllib3#3475) <https://github.com/urllib3/urllib3/issues/3475></code>__)</li> </ul> <h1>2.2.2 (2024-06-17)</h1> <ul> <li>Added the <code>Proxy-Authorization</code> header to the list of headers to strip from requests when redirecting to a different host. As before, different headers can be set via <code>Retry.remove_headers_on_redirect</code>.</li> <li>Allowed passing negative integers as <code>amt</code> to read methods of <code>http.client.HTTPResponse</code> as an alternative to <code>None</code>. (<code>[#3122](urllib3/urllib3#3122) <https://github.com/urllib3/urllib3/issues/3122></code>__)</li> <li>Fixed return types representing copying actions to use <code>typing.Self</code>. (<code>[#3363](urllib3/urllib3#3363) <https://github.com/urllib3/urllib3/issues/3363></code>__)</li> </ul> <h1>2.2.1 (2024-02-16)</h1> <ul> <li>Fixed issue where <code>InsecureRequestWarning</code> was emitted for HTTPS connections when using Emscripten. (<code>[#3331](urllib3/urllib3#3331) <https://github.com/urllib3/urllib3/issues/3331></code>__)</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/urllib3/urllib3/commit/2458bfcd3dacdf6c196e98d077fc6bb02a5fc1df"><code>2458bfc</code></a> Release 2.2.3</li> <li><a href="https://github.com/urllib3/urllib3/commit/9b25db6d00e43858d49303ae55c43bc4a9832668"><code>9b25db6</code></a> Only attempt to publish for upstream</li> <li><a href="https://github.com/urllib3/urllib3/commit/b9adeef8501180cd7d04cc3fb90bed4bbc34b1bb"><code>b9adeef</code></a> Drop support for EOL PyPy3.8 and PyPy3.9</li> <li><a href="https://github.com/urllib3/urllib3/commit/b1d4649d43375f11a3072b4d9b5d33425d123bae"><code>b1d4649</code></a> Add explicit support for Python 3.13</li> <li><a href="https://github.com/urllib3/urllib3/commit/cc42860721836febf3fb6ebb485ed27d7f80122d"><code>cc42860</code></a> Bump cryptography from 42.0.4 to 43.0.1 (<a href="https://redirect.github.com/urllib3/urllib3/issues/3470">#3470</a>)</li> <li><a href="https://github.com/urllib3/urllib3/commit/3dae2e9b30d2e39bf20daea2353aa7ef055640cf"><code>3dae2e9</code></a> Bump pypa/gh-action-pypi-publish from 1.9.0 to 1.10.1 (<a href="https://redirect.github.com/urllib3/urllib3/issues/3469">#3469</a>)</li> <li><a href="https://github.com/urllib3/urllib3/commit/1e94feb2a671bf28721114dfea1105a2c1f91788"><code>1e94feb</code></a> Revert "Add TLS settings for HTTP/2 (<a href="https://redirect.github.com/urllib3/urllib3/issues/3456">#3456</a>)" (<a href="https://redirect.github.com/urllib3/urllib3/issues/3466">#3466</a>)</li> <li><a href="https://github.com/urllib3/urllib3/commit/aa73abc7b22a4a67e0ee957f5a3031109f73d3d9"><code>aa73abc</code></a> Bump actions/setup-python from 5.1.0 to 5.2.0 (<a href="https://redirect.github.com/urllib3/urllib3/issues/3468">#3468</a>)</li> <li><a href="https://github.com/urllib3/urllib3/commit/abbfbcb1dd274fc54b4f0a7785fd04d59b634195"><code>abbfbcb</code></a> Add 1.26.20 to changelog and make the publish workflow the same (<a href="https://redirect.github.com/urllib3/urllib3/issues/3464">#3464</a>)</li> <li><a href="https://github.com/urllib3/urllib3/commit/d48061505e72271116c5a33b04dbca6273f2a737"><code>d480615</code></a> Add TLS settings for HTTP/2 (<a href="https://redirect.github.com/urllib3/urllib3/issues/3456">#3456</a>)</li> <li>Additional commits viewable in <a href="https://github.com/urllib3/urllib3/compare/1.26.14...2.2.3">compare view</a></li> </ul> </details> <br /> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/spherex-xyz/spherex-contracts/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…72) Bumps the pip group with 1 update in the / directory: [eth-abi](https://github.com/ethereum/eth-abi). Updates `eth-abi` from 4.2.1 to 5.0.1 <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/ethereum/eth-abi/blob/main/docs/release_notes.rst">eth-abi's changelog</a>.</em></p> <blockquote> <h2>eth-abi v5.0.1 (2024-03-04)</h2> <p>Bugfixes</p> <pre><code> - During decoding, verify all pointers in arrays and tuples point to a valid location in the payload (`[#226](ethereum/eth-abi#226) <https://github.com/ethereum/eth-abi/issues/226>`__) - Fix memory leak warning in NodeVisitor and ABIRegistry (`[#230](ethereum/eth-abi#230) <https://github.com/ethereum/eth-abi/issues/230>`__) <p>Internal Changes - for eth-abi Contributors </code></pre></p> <ul> <li>Clear mypy <code>misc</code>-type errors and add top-level <code>py.typed</code> file back (<code>[#221](ethereum/eth-abi#221) <https://github.com/ethereum/eth-abi/issues/221></code>__)</li> </ul> <h2>eth-abi v5.0.0 (2024-01-09)</h2> <p>Breaking Changes</p> <pre><code> - Drop python 3.7 support (`[#217](ethereum/eth-abi#217) <https://github.com/ethereum/eth-abi/issues/217>`__) <p>Internal Changes - for eth-abi Contributors </code></pre></p> <ul> <li>Merge updates from the project template, notably, use <code>pre-commit</code> for linting and change the name of the <code>master</code> branch to <code>main</code> (<code>[#217](ethereum/eth-abi#217) <https://github.com/ethereum/eth-abi/issues/217></code>__)</li> <li>Corrected format of booleans in <code>pyproject.toml</code> and added a test for the presence of the <code>eth_abi.__version__</code> attribute (<code>[#219](ethereum/eth-abi#219) <https://github.com/ethereum/eth-abi/issues/219></code>__)</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/ethereum/eth-abi/commit/be95178fda29e50c2378163cd2239abe842d8bbd"><code>be95178</code></a> Bump version: 5.0.0 → 5.0.1</li> <li><a href="https://github.com/ethereum/eth-abi/commit/00f990ed135cf203f1407bf05184cfc9f336ba3f"><code>00f990e</code></a> Compile release notes for v5.0.1</li> <li><a href="https://github.com/ethereum/eth-abi/commit/82c1ad37a866472562d81fedaef0f4fed0a08269"><code>82c1ad3</code></a> add validate_pointers for tuples and arrays, add tests and more</li> <li><a href="https://github.com/ethereum/eth-abi/commit/75aab7cdcc2a1200b0cebaaee84d0d816f9147be"><code>75aab7c</code></a> remove warning of ABIv2 being unstable</li> <li><a href="https://github.com/ethereum/eth-abi/commit/6063bf1f3f1f6fc0e0e74e4d6e7f86e62fc850b2"><code>6063bf1</code></a> add newsfragments file</li> <li><a href="https://github.com/ethereum/eth-abi/commit/2367eef95b5b28945bf912ae56640682e67ff1da"><code>2367eef</code></a> 🚨 fix linting warning (rule B019)</li> <li><a href="https://github.com/ethereum/eth-abi/commit/abbe2100a5e7d8d0913e83632fffd77ea2ab75a1"><code>abbe210</code></a> type codec.py and exceptions, change mypy ignore from files to error types, a...</li> <li><a href="https://github.com/ethereum/eth-abi/commit/d063fc6e9195f20d9c42a88dde3f6e4b3319f067"><code>d063fc6</code></a> Bump version: 4.2.1 → 5.0.0</li> <li><a href="https://github.com/ethereum/eth-abi/commit/7641e321c89b0ebba355830819036bacbb5072db"><code>7641e32</code></a> Compile release notes for v5.0.0</li> <li><a href="https://github.com/ethereum/eth-abi/commit/31866a392ed3ac376ea00d8ab57e4a5f4818ceee"><code>31866a3</code></a> fix toml bools and add version test</li> <li>Additional commits viewable in <a href="https://github.com/ethereum/eth-abi/compare/v4.2.1...v5.0.1">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=eth-abi&package-manager=pip&previous-version=4.2.1&new-version=5.0.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/spherex-xyz/spherex-contracts/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
What was wrong?
Incorrect values for in pointers can cause problems. If a pointer value is not large enough, i.e. it points to an area in the payload that is still within the pointers section, the encoding is malformed. In certain situations, ~infinite loops can occur.
How was it fixed?
When decoding pointers, determine the location in the stream that divides pointers and values and make sure all pointers point past that location. Also check for pointers that point beyond the end of the payload.
Added some code comments to make it easier to remember how
HeadTailDecoder
works.Added
pytest-timeout
to dependencies, as if the new tests are run without the added offset checking, they'll spin for a long time before failing.Todo:
Clean up commit history
Clear any breakpoints
clean up testing
Add or update documentation related to these changes
Add entry to the release notes
Cute Animal Picture