Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for UFloat in PintArray (#139) #140

Open
wants to merge 58 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
b5954fd
Add support for UFloat in PintArray (#139)
MichaelTiemannOSC Oct 15, 2022
0ad1cf9
Fix failures and errors found by test_pandas_extensions test suite.
MichaelTiemannOSC Oct 19, 2022
52ab185
Preserve incoming np.array when promoting float to ufloat in PintArray
MichaelTiemannOSC Oct 21, 2022
f3cdcad
Fix logic to detect heterogeneous arrays of Ufloats and floats.
MichaelTiemannOSC Oct 25, 2022
3ffb617
Add support for UFloat in PintArray (#139)
MichaelTiemannOSC Oct 15, 2022
2f89897
Fix failures and errors found by test_pandas_extensions test suite.
MichaelTiemannOSC Oct 19, 2022
dce2668
Preserve incoming np.array when promoting float to ufloat in PintArray
MichaelTiemannOSC Oct 21, 2022
d4ca9f0
Fix logic to detect heterogeneous arrays of Ufloats and floats.
MichaelTiemannOSC Oct 25, 2022
c375aeb
Merge branch 'ducks-unlimited' of https://github.com/MichaelTiemannOS…
MichaelTiemannOSC Nov 3, 2022
9fffcc5
Update pint_array.py
MichaelTiemannOSC Jan 2, 2023
8b06708
Update pint_array.py
MichaelTiemannOSC Jan 3, 2023
c5b7926
Update pint_array.py
MichaelTiemannOSC Jan 3, 2023
232857c
Merge branch 'master' into ducks-unlimited
MichaelTiemannOSC Jun 26, 2023
0b0e4d4
Fix and blacken merge
MichaelTiemannOSC Jun 26, 2023
959570f
Fix ruff complaints in testsuite
MichaelTiemannOSC Jun 26, 2023
5270a46
Fix numerous regressions in test_pandas_extensiontests
MichaelTiemannOSC Jun 28, 2023
6ddf204
Update pint_array.py
MichaelTiemannOSC Jun 28, 2023
e1d367c
Update to us pd.NA instead of np.nan / _ufloat_nan
MichaelTiemannOSC Jul 2, 2023
dbf5ad1
Update pint_array.py
MichaelTiemannOSC Jul 2, 2023
3c6eff4
Progress: 2608 pass, 97 skip, 84 xfail, 6 xpass
MichaelTiemannOSC Jul 5, 2023
a0625f8
Make ruff and black happy
MichaelTiemannOSC Jul 5, 2023
94d3524
Make ruff happy (na_frame fixture import vs F811)
MichaelTiemannOSC Jul 5, 2023
a6c4040
Make black happy
MichaelTiemannOSC Jul 5, 2023
1506df2
Make black happy
MichaelTiemannOSC Jul 5, 2023
772636b
Fix DataFrame reduction for upcoming Pandas
MichaelTiemannOSC Jul 23, 2023
b759adb
Make black happy...
MichaelTiemannOSC Jul 23, 2023
bfb4a99
Update pint_array.py
MichaelTiemannOSC Jul 24, 2023
9d169f1
Switch to np.nan as NaN value
MichaelTiemannOSC Jul 28, 2023
289c604
Updated to Pandas 2.1.0.dev0+1401.gb0bfd0effd
MichaelTiemannOSC Aug 6, 2023
602a804
Keep up with pandas21_compat changes
MichaelTiemannOSC Aug 12, 2023
866bf7a
Merge branch 'master' into ducks-unlimited
MichaelTiemannOSC Aug 12, 2023
9f723f9
Merge remote-tracking branch 'upstream/master' into ducks-unlimited
MichaelTiemannOSC Aug 14, 2023
f0c7e64
Cleanups after merge
MichaelTiemannOSC Aug 15, 2023
5b39a3e
Update test_issues.py
MichaelTiemannOSC Aug 15, 2023
8ed4c1d
Add uncertainties to CI/CD
MichaelTiemannOSC Aug 15, 2023
2cb50f4
Update CI/CD to anticipate, not install or test uncertainties
MichaelTiemannOSC Aug 15, 2023
8c4bf7d
Update CHANGES
MichaelTiemannOSC Aug 15, 2023
37c6f6d
Merge branch 'master' into ducks-unlimited
MichaelTiemannOSC Sep 8, 2023
fc2814a
Merge branch 'master' into ducks-unlimited
MichaelTiemannOSC Sep 15, 2023
e365cbc
Test with Pint-0.23rc0 and uncertainties in ci/cd
MichaelTiemannOSC Sep 18, 2023
108cb71
2nd attempt integrating uncertainties and CI/CD
MichaelTiemannOSC Sep 18, 2023
a5758e7
Use `include` to handle uncertainties testing
MichaelTiemannOSC Sep 19, 2023
82442ab
Only test `uncertainties` in ci.yml
MichaelTiemannOSC Sep 19, 2023
5d351fc
Update ci.yml
MichaelTiemannOSC Sep 19, 2023
c4e3a06
Update ci.yml
MichaelTiemannOSC Sep 19, 2023
8d5feb9
Update ci.yml
MichaelTiemannOSC Sep 19, 2023
bfe9a77
Update ci.yml
MichaelTiemannOSC Sep 19, 2023
25adf35
Update ci.yml
MichaelTiemannOSC Sep 19, 2023
afc3eb4
Update ci.yml
MichaelTiemannOSC Sep 19, 2023
e281dfc
Update ci.yml
MichaelTiemannOSC Sep 19, 2023
a208163
Update ci.yml
MichaelTiemannOSC Sep 19, 2023
296bbdc
Update ci.yml
MichaelTiemannOSC Sep 19, 2023
585f38d
Update ci.yml
MichaelTiemannOSC Sep 19, 2023
75d4c56
Update ci.yml
MichaelTiemannOSC Sep 19, 2023
6988212
Update ci.yml
MichaelTiemannOSC Sep 19, 2023
764b609
Update ci.yml
MichaelTiemannOSC Sep 19, 2023
9ed23c1
Update ci-*.yml files
MichaelTiemannOSC Sep 19, 2023
b609001
Merge branch 'master' into ducks-unlimited
MichaelTiemannOSC Nov 14, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGES
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ pint-pandas Changelog
- Fixed bug preventing adding to empty slices of PintArrays #69
- Notebook updated to show plotting #116 #9 #43
- Tests reorganised #131
- Allow UFloat as type of magnitude supported in PintArray

0.2 (2021-03-23)
----------------
Expand Down
75 changes: 61 additions & 14 deletions pint_pandas/pint_array.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,14 @@
from pandas.core.arrays.base import ExtensionOpsMixin
from pandas.core.indexers import check_array_indexer
from pint import compat, errors
from pint.quantity import _Quantity
from pint.unit import _Unit
from pint.compat import HAS_UNCERTAINTIES
from pint.facets.plain.quantity import PlainQuantity as _Quantity
from pint.facets.plain.unit import PlainUnit as _Unit

if HAS_UNCERTAINTIES:
from uncertainties import UFloat, ufloat
from uncertainties import unumpy as unp
_ufloat_nan = ufloat(np.nan, 0)


class PintType(ExtensionDtype):
Expand Down Expand Up @@ -131,7 +137,10 @@ def name(self):

@property
def na_value(self):
return self.ureg.Quantity(np.nan, self.units)
if HAS_UNCERTAINTIES:
return self.ureg.Quantity(_ufloat_nan, self.units)
else:
return self.ureg.Quantity(np.nan, self.units)

def __hash__(self):
# make myself hashable
Expand Down Expand Up @@ -216,13 +225,31 @@ def __init__(self, values, dtype=None, copy=False):
if not isinstance(values, np.ndarray):
values = np.array(values, copy=copy)
copy = False
if not np.issubdtype(values.dtype, np.floating):
warnings.warn(
f"pint-pandas does not support magnitudes of {values.dtype}. Converting magnitudes to float.",
category=RuntimeWarning,
)
values = values.astype(float)
copy = False
if HAS_UNCERTAINTIES:
if np.issubdtype(values.dtype, np.floating):
pass
elif not all([isinstance(v, UFloat) for v in values]):
warnings.warn(
f"pint-pandas does not support certain magnitudes of {values.dtype}. Converting magnitudes to ufloat.",
category=RuntimeWarning,
)
for i in range(len(values)):
# List comprehensions are great, but they are not np.arrays!
if not isinstance(values[i], UFloat):
if np.isnan(values[i]):
values[i] = _ufloat_nan
else:
values[i] = ufloat(values[i], 0)
elif unp.isnan(values[i]):
values[i] = _ufloat_nan
copy = False
elif not np.issubdtype(values.dtype, np.floating):
warnings.warn(
f"pint-pandas does not support magnitudes of {values.dtype}. Converting magnitudes to float.",
category=RuntimeWarning,
)
values = values.astype(float)
copy = False
if copy:
values = values.copy()
self._data = values
Expand Down Expand Up @@ -269,7 +296,7 @@ def __getitem__(self, item):
item : scalar or PintArray
"""
if is_integer(item):
return self._data[item] * self.units
return self._Q(self._data[item], self.units)

item = check_array_indexer(self, item)

Expand Down Expand Up @@ -336,6 +363,12 @@ def isna(self):
-------
missing : np.array
"""
if HAS_UNCERTAINTIES:
# GH https://github.com/lebigot/uncertainties/issues/164
if isinstance(self._data, np.ndarray) and len(self._data)==0:
# True or False doesn't matter--we just need the value for the type
return np.full((0), True)
return unp.isnan(self._data)
return np.isnan(self._data)

def astype(self, dtype, copy=True):
Expand Down Expand Up @@ -483,8 +516,17 @@ def _from_sequence(cls, scalars, dtype=None, copy=False):
dtype = PintType(master_scalar.units)

def quantify_nan(item):
if type(item) is float:
return item * dtype.units
if HAS_UNCERTAINTIES:
if type(item) is UFloat:
return item * dtype.units
if type(item) is float:
if np.isnan(item):
return _ufloat_nan * dtype.units
else:
return UFloat(item, 0) * dtype.units
else:
if type(item) is float:
return item * dtype.units
return item

if isinstance(master_scalar, _Quantity):
Expand All @@ -504,6 +546,8 @@ def _from_factorized(cls, values, original):

def _values_for_factorize(self):
arr = self._data
if HAS_UNCERTAINTIES:
return arr, _ufloat_nan
return arr, np.NaN

def value_counts(self, dropna=True):
Expand Down Expand Up @@ -531,7 +575,10 @@ def value_counts(self, dropna=True):
# compute counts on the data with no nans
data = self._data
if dropna:
data = data[~np.isnan(data)]
if HAS_UNCERTAINTIES:
data = data[~unp.isnan(data)]
else:
data = data[~np.isnan(data)]

data_list = data.tolist()
index = list(set(data))
Expand Down
37 changes: 34 additions & 3 deletions pint_pandas/testsuite/test_issues.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,15 @@
from pandas.tests.extension.base.base import BaseExtensionTests
from pint.testsuite import helpers

try:
import uncertainties.unumpy as unp
from uncertainties import ufloat, UFloat
HAS_UNCERTAINTIES = True
except ImportError:
unp = np
ufloat = Ufloat = None
HAS_UNCERTAINTIES = False

from pint_pandas import PintArray, PintType

ureg = PintType.ureg
Expand All @@ -15,9 +24,9 @@
class TestIssue21(BaseExtensionTests):
@pytest.mark.filterwarnings("ignore::RuntimeWarning")
def test_offset_concat(self):
q_a = ureg.Quantity(np.arange(5), ureg.Unit("degC"))
q_b = ureg.Quantity(np.arange(6), ureg.Unit("degC"))
q_a_ = np.append(q_a, np.nan)
q_a = ureg.Quantity(np.arange(5)+ufloat(0,0), ureg.Unit("degC"))
q_b = ureg.Quantity(np.arange(6)+ufloat(0,0), ureg.Unit("degC"))
q_a_ = np.append(q_a, ufloat(np.nan, 0))

a = pd.Series(PintArray(q_a))
b = pd.Series(PintArray(q_b))
Expand Down Expand Up @@ -107,3 +116,25 @@ def test_issue_88():
q_mm = ureg.Quantity([1000, 2000], "mm")
b = PintArray(q_mm, "m")
helpers.assert_quantity_almost_equal(q_m, b.quantity)

def test_issue_139():
from pint.compat import HAS_UNCERTAINTIES
assert(HAS_UNCERTAINTIES)
from uncertainties import ufloat
from uncertainties import unumpy as unp

q1 = 1.234
q2 = 5.678
q_nan = np.nan

u1 = ufloat(1, 0)
u2 = ufloat(3, 0)
u_nan = ufloat(np.nan, 0.0)
u_plus_or_minus_nan = ufloat(0.0, np.nan)
u_nan_plus_or_minus_nan = ufloat(np.nan, np.nan)

a_m = PintArray([q1, u1, q2, u2, q_nan, u_nan, u_plus_or_minus_nan, u_nan_plus_or_minus_nan], ureg.m)
a_cm = a_m.astype('pint[cm]')
assert np.all(a_m[0:4] == a_cm[0:4])
for x, y in zip(a_m[4:], a_cm[4:]):
assert unp.isnan(x) == unp.isnan(y)
Loading