Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🛑 DNM Deserialization: zero-copy merge subframes when possible #5112

Closed

Commits on Jul 23, 2021

  1. Configuration menu
    Copy the full SHA
    4fc3dae View commit details
    Browse the repository at this point in the history
  2. make tests pass

    This is a hack because in real use, I am pretty sure that `frames` will be either all memoryviews or all not, since they'll either be coming from a comm or from a bytestring. But in tests where we call `loads` directly, this may not be the case.
    gjoseph92 committed Jul 23, 2021
    Configuration menu
    Copy the full SHA
    1869b18 View commit details
    Browse the repository at this point in the history
  3. Test zero-copy

    gjoseph92 committed Jul 23, 2021
    Configuration menu
    Copy the full SHA
    75922a6 View commit details
    Browse the repository at this point in the history
  4. cleanup

    gjoseph92 committed Jul 23, 2021
    Configuration menu
    Copy the full SHA
    cf62d30 View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2021

  1. Typo and readability

    Co-authored-by: crusaderky <crusaderky@gmail.com>
    gjoseph92 and crusaderky authored Jul 28, 2021
    Configuration menu
    Copy the full SHA
    cc528c0 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    934526f View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2021

  1. Configuration menu
    Copy the full SHA
    ee679fd View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2021

  1. rerun ci

    gjoseph92 committed Jul 30, 2021
    Configuration menu
    Copy the full SHA
    523957b View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    2885366 View commit details
    Browse the repository at this point in the history
  3. Assert instead of subframe copy

    Less useful, but also less complex. Short of writing the frames by hand, I couldn't come up with a test that could even trigger this behavior. I'd feel uncomfortable having that code be untested and only run on what's already an edge case. So if this assert is ever triggered, we can come back and add the zero-copy and test it.
    gjoseph92 committed Jul 30, 2021
    Configuration menu
    Copy the full SHA
    39ce2b6 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    31e0e6e View commit details
    Browse the repository at this point in the history
  5. Revert "Assert instead of subframe copy"

    This reverts commit 39ce2b6.
    gjoseph92 committed Jul 30, 2021
    Configuration menu
    Copy the full SHA
    6ace0a4 View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2021

  1. Changed mind: multiple-memoryview case with copy

    A separate `copy_frames` function makes this more readable and easier to test. Also, I came up with a test for this case that's still contrived, but not ridiculously contrived.
    
    That said, we don't want this copy to happen. And I'm pretty confident it will never happen with reall comms, because either the whole message is one buffer (TCP), or memoryviews aren't used at all. This mix-and-match only even happens in tests; see 1869b18. So maybe we should stick with the assert as a warning to future developers, so nobody messes this up and it keeps working with a silent performance regression?
    gjoseph92 committed Jul 31, 2021
    Configuration menu
    Copy the full SHA
    ab6119a View commit details
    Browse the repository at this point in the history
  2. Changed mind again: back to AssertionError

    As noted in ab6119a: I think this error is currently impossible to raise in real use, and we want to keep it that way. We'd like a future test to fail if something causes this case to happen, rather than a silent copy.
    gjoseph92 committed Jul 31, 2021
    Configuration menu
    Copy the full SHA
    e2da610 View commit details
    Browse the repository at this point in the history