Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert floats representing integers to int in sample set serialization #1354

Conversation

arcondello
Copy link
Member

@arcondello arcondello commented Aug 11, 2023

This does come with a performance hit. However it is relatively minor. Testing two edge cases on my system

import time

import dimod
import numpy as np

arr = np.full((100, 500000), 1, dtype=float)  # can be converted

t = time.perf_counter()
dimod.serialization.utils.serialize_ndarray(arr)
print(time.perf_counter() - t)

takes ~5.5 seconds on this branch and ~1 second on dimod 0.12.10.

import time

import dimod
import numpy as np

arr = np.full((100, 500000), 1.5, dtype=float)  # cannot be converted

t = time.perf_counter()
dimod.serialization.utils.serialize_ndarray(arr)
print(time.perf_counter() - t)

takes ~3.5 seconds on this branch and ~1 second on dimod 0.12.10.

For context, actually Json-serializing the resulting object at this scale takes ~27.8 seconds to dump to a file and ~6.7 seconds to dump to a string.

@arcondello arcondello force-pushed the feature/sampleset-integer-serialization branch 2 times, most recently from 55fd909 to ea17840 Compare August 11, 2023 16:39
@arcondello arcondello force-pushed the feature/sampleset-integer-serialization branch from ea17840 to 7b75d7f Compare August 11, 2023 16:48
@arcondello arcondello marked this pull request as ready for review August 11, 2023 16:48
@codecov
Copy link

codecov bot commented Aug 11, 2023

Codecov Report

Merging #1354 (3055f08) into main (e5a8e3b) will decrease coverage by 0.01%.
The diff coverage is 92.30%.

@@            Coverage Diff             @@
##             main    #1354      +/-   ##
==========================================
- Coverage   95.08%   95.07%   -0.01%     
==========================================
  Files          95       95              
  Lines        9902     9914      +12     
==========================================
+ Hits         9415     9426      +11     
- Misses        487      488       +1     
Files Changed Coverage Δ
dimod/serialization/utils.py 88.70% <91.66%> (+0.70%) ⬆️
dimod/sampleset.py 91.43% <100.00%> (ø)

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Member

@randomir randomir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

``int``.

This function assumes some uniformity of the list structure. For instance giving it
a list like ``[0.0, 0]`` or ``[0.0, [0.0]`` will cause it to fail.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: missing ]


else:
raise ValueError("expected a (possibly nested) list of floats, "
f"recieved a (possible nested) list of {type(arr[0])}")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typos: received, possibly

@arcondello arcondello merged commit 2d35745 into dwavesystems:main Aug 17, 2023
@arcondello arcondello deleted the feature/sampleset-integer-serialization branch August 17, 2023 17:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants