Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fixes for nightly: Statistics stdlib #583

Merged
merged 4 commits into from
Jul 16, 2018
Merged

fixes for nightly: Statistics stdlib #583

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b924155
Statistics stdlib
stevengj Jun 29, 2018
f7760df
fixes for 0.6
stevengj Jun 29, 2018
5ff9b18
compatibility with old Compat versions on 0.6 for dims keyword
stevengj Jul 5, 2018
939482e
missing varm
stevengj Jul 5, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Project.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ Serialization = "9e88b42a-f829-5b0c-bbe9-9e923198166b"
SharedArrays = "1a1011a3-84de-559e-8e89-a11a2f7dc383"
Sockets = "6462fe0b-24de-5631-8697-dd941f90decc"
SparseArrays = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"
Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
UUIDs = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"
Unicode = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"
12 changes: 6 additions & 6 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,9 @@ Currently, the `@compat` macro supports the following syntaxes:
* `using Compat.Sockets` is provided on versions older than 0.7, where this library is
not yet part of the standard library ([#25935])

* `using Compat.Statistics` is provided on versions older than 0.7, where this library is
not yet part of the standard library ([#27834]).

## New functions, macros, and methods

* `@views` takes an expression and converts all slices to views ([#20164]), while
Expand Down Expand Up @@ -217,10 +220,7 @@ Currently, the `@compat` macro supports the following syntaxes:
* `Base.rtoldefault` how takes a third parameter `atol`.
The two argument form is deprecated in favor of the three arguments form with `atol=0`.

* The `corrected` optional argument of `cov` becomes a keyword argument.
Due to conflict with 0.5 deprecation,
`cov(::AbstractVector; corrected=)` and `cov(::AbstractVector, ::AbstractVector; corrected=)`
are only available on 0.6. ([#21709])
* The `corrected` optional argument of `cov` becomes a keyword argument of `Compat.Statistics.cov` ([#21709]).

* `isequal`, `==` and `in` have one argument "curried" forms. For example `isequal(x)`
returns a function that compares its arguments to `x` using `isequal` ([#26436]).
Expand Down Expand Up @@ -282,8 +282,8 @@ Currently, the `@compat` macro supports the following syntaxes:

* `Compat.accumulate`, `Compat.accumulate!`, `Compat.all`, `Compat.any`,
`Compat.cumprod`, `Compat.cumprod!`, `Compat.cumsum`, `Compat.cumsum!`,
`Compat.findmax`, `Compat.findmin`, `Compat.mapreduce`, `Compat.maximum`, `Compat.mean`,
`Compat.median`, `Compat.minimum`, `Compat.prod`, `Compat.reduce`, `Compat.sort`,
`Compat.findmax`, `Compat.findmin`, `Compat.mapreduce`, `Compat.maximum`, `Compat.Statistics.mean`,
`Compat.Statistics.median`, `Compat.minimum`, `Compat.prod`, `Compat.reduce`, `Compat.sort`,
and `Compat.sum` with `dims` keyword argument ([#25989],[#26369]).

* `Compat.mapreduce` and `Compat.reduce` with `init` keyword argument ([#27711]).
Expand Down
61 changes: 37 additions & 24 deletions src/Compat.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -857,6 +857,37 @@ else
import Serialization
end

if VERSION < v"0.7.0-beta.85"
@eval module Statistics
if VERSION < v"0.7.0-DEV.4064"
varm(A::AbstractArray, m; dims=nothing, kwargs...) =
dims===nothing ? Base.varm(A, m; kwargs...) : Base.varm(A, m, dims; kwargs...)
if VERSION < v"0.7.0-DEV.755"
cov(a::AbstractMatrix; dims=1, corrected=true) = Base.cov(a, dims, corrected)
cov(a::AbstractVecOrMat, b::AbstractVecOrMat; dims=1, corrected=true) =
Base.cov(a, b, dims, corrected)
else
cov(a::AbstractMatrix; dims=nothing, kwargs...) =
dims===nothing ? Base.cov(a; kwargs...) : Base.cov(a, dims; kwargs...)
cov(a::AbstractVecOrMat, b::AbstractVecOrMat; dims=nothing, kwargs...) =
dims===nothing ? Base.cov(a, b; kwargs...) : Base.cov(a, b, dims; kwargs...)
end
cor(a::AbstractMatrix; dims=nothing) = dims===nothing ? Base.cor(a) : Base.cor(a, dims)
cor(a::AbstractVecOrMat, b::AbstractVecOrMat; dims=nothing) =
dims===nothing ? Base.cor(a, b) : Base.cor(a, b, dims)
mean(a::AbstractArray; dims=nothing) = dims===nothing ? Base.mean(a) : Base.mean(a, dims)
median(a::AbstractArray; dims=nothing) = dims===nothing ? Base.median(a) : Base.median(a, dims)
var(a::AbstractArray; dims=nothing, kwargs...) =
dims===nothing ? Base.var(a; kwargs...) : Base.var(a, dims; kwargs...)
std(a::AbstractArray; dims=nothing, kwargs...) =
dims===nothing ? Base.std(a; kwargs...) : Base.std(a, dims; kwargs...)
end
export cor, cov, std, stdm, var, varm, mean!, mean, median!, median, middle, quantile!, quantile
end
else
import Statistics
end

@static if VERSION < v"0.7.0-DEV.4592"
struct Fix2{F,T} <: Function
f::F
Expand Down Expand Up @@ -1728,7 +1759,10 @@ if VERSION < v"0.7.0-DEV.4585"
end

if VERSION < v"0.7.0-DEV.4064"
for f in (:mean, :cumsum, :cumprod, :sum, :prod, :maximum, :minimum, :all, :any, :median)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we remove these we'll break any packages which depend on this, right? I'm afraid we'll have to keep them, at least on Julia 0.6. That's what we have done for several other features which have changed over the 0.7 development period.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I guess we'll need to replace this loop with a loop that imports them from Compat.Statistics.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While keeping them to avoid breaking packages that have already adapted to the keyword, we should erase them from the README though, to encourage use of Compat.Statistics instead.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll just change the mentions in the README to Compat.Statistics.mean etcetera.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be fixed now

for f in (:mean, :median, :var, :varm, :std, :cov, :cor)
@eval import .Statistics: $f # compatibility with old Compat versions
end
for f in (:cumsum, :cumprod, :sum, :prod, :maximum, :minimum, :all, :any)
@eval begin
$f(a::AbstractArray; dims=nothing) =
dims===nothing ? Base.$f(a) : Base.$f(a, dims)
Expand All @@ -1751,35 +1785,14 @@ if VERSION < v"0.7.0-DEV.4064"
end
end
end
# TODO add deprecation warning to switch to StatsBase to var and std
for f in (:var, :std, :sort)
@eval begin
$f(a::AbstractArray; dims=nothing, kwargs...) =
dims===nothing ? Base.$f(a; kwargs...) : Base.$f(a, dims; kwargs...)
end
end
@eval sort(a::AbstractArray; dims=nothing, kwargs...) =
dims===nothing ? Base.sort(a; kwargs...) : Base.sort(a, dims; kwargs...)
for f in (:cumsum!, :cumprod!)
@eval $f(out, a; dims=nothing) =
dims===nothing ? Base.$f(out, a) : Base.$f(out, a, dims)
end
end
if VERSION < v"0.7.0-DEV.4064"
# TODO add deprecation warning to switch to StatsBase to varm, cov, and cor
varm(A::AbstractArray, m; dims=nothing, kwargs...) =
dims===nothing ? Base.varm(A, m; kwargs...) : Base.varm(A, m, dims; kwargs...)
if VERSION < v"0.7.0-DEV.755"
cov(a::AbstractMatrix; dims=1, corrected=true) = Base.cov(a, dims, corrected)
cov(a::AbstractVecOrMat, b::AbstractVecOrMat; dims=1, corrected=true) =
Base.cov(a, b, dims, corrected)
else
cov(a::AbstractMatrix; dims=nothing, kwargs...) =
dims===nothing ? Base.cov(a; kwargs...) : Base.cov(a, dims; kwargs...)
cov(a::AbstractVecOrMat, b::AbstractVecOrMat; dims=nothing, kwargs...) =
dims===nothing ? Base.cov(a, b; kwargs...) : Base.cov(a, b, dims; kwargs...)
end
cor(a::AbstractMatrix; dims=nothing) = dims===nothing ? Base.cor(a) : Base.cor(a, dims)
cor(a::AbstractVecOrMat, b::AbstractVecOrMat; dims=nothing) =
dims===nothing ? Base.cor(a, b) : Base.cor(a, b, dims)
mapreduce(f, op, a::AbstractArray; dims=nothing, init=nothing) =
init === nothing ? (dims===nothing ? Base.mapreduce(f, op, a) : Base.mapreducedim(f, op, a, dims)) :
(dims===nothing ? Base.mapreduce(f, op, init, a) : Base.mapreducedim(f, op, a, dims, init))
Expand Down
74 changes: 37 additions & 37 deletions test/runtests.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1502,9 +1502,9 @@ end
# the wrong result between 0.7.0-DEV.3262 and 0.7.0-DEV.4646
# see https://github.com/JuliaLang/julia/issues/26488
Issue26488 = VERSION < v"0.7.0-DEV.3262" || VERSION >= v"0.7.0-DEV.4646"
@test Compat.mean([1 2; 3 4]) == 2.5
@test Compat.mean([1 2; 3 4], dims=1) == [2 3]
@test Compat.mean([1 2; 3 4], dims=2) == hcat([1.5; 3.5])
@test Compat.Statistics.mean([1 2; 3 4]) == 2.5
@test Compat.Statistics.mean([1 2; 3 4], dims=1) == [2 3]
@test Compat.Statistics.mean([1 2; 3 4], dims=2) == hcat([1.5; 3.5])
@test Compat.cumsum([1 2; 3 4], dims=1) == [1 2; 4 6]
@test Compat.cumsum([1 2; 3 4], dims=2) == [1 3; 3 7]
@test Compat.cumprod([1 2; 3 4], dims=1) == [1 2; 3 8]
Expand Down Expand Up @@ -1559,40 +1559,40 @@ Issue26488 && @test Compat.prod(x -> x+1, [1 2; 3 4], dims=2) == hcat([6; 20])
if VERSION < v"0.7.0-DEV.5238"
# Test these functions if their counterparts are defined in Base. In the future, this
# will be dealt with in StatsBase
@test Compat.varm([1 2; 3 4], -1) == 18
@test Compat.varm([1 2; 3 4], [-1 -2], dims=1) == [20 52]
@test Compat.varm([1 2; 3 4], [-1, -2], dims=2) == hcat([13, 61])
@test Compat.var([1 2; 3 4]) == 5/3
@test Compat.var([1 2; 3 4], dims=1) == [2 2]
@test Compat.var([1 2; 3 4], dims=2) == hcat([0.5, 0.5])
@test Compat.var([1 2; 3 4], corrected=false) == 1.25
@test Compat.var([1 2; 3 4], corrected=false, dims=1) == [1 1]
@test Compat.var([1 2; 3 4], corrected=false, dims=2) == hcat([0.25, 0.25])
@test Compat.std([1 2; 3 4]) == sqrt(5/3)
@test Compat.std([1 2; 3 4], dims=1) == [sqrt(2) sqrt(2)]
@test Compat.std([1 2; 3 4], dims=2) == hcat([sqrt(0.5), sqrt(0.5)])
@test Compat.std([1 2; 3 4], corrected=false) == sqrt(1.25)
@test Compat.std([1 2; 3 4], corrected=false, dims=1) == [sqrt(1) sqrt(1)]
@test Compat.std([1 2; 3 4], corrected=false, dims=2) == hcat([sqrt(0.25), sqrt(0.25)])
@test Compat.cov([1 2; 3 4]) == [2 2; 2 2]
@test Compat.cov([1 2; 3 4], dims=1) == [2 2; 2 2]
@test Compat.cov([1 2; 3 4], dims=2) == [0.5 0.5; 0.5 0.5]
@test Compat.cov([1 2; 3 4], [4; 5]) == hcat([1; 1])
@test Compat.cov([1 2; 3 4], [4; 5], dims=1) == hcat([1; 1])
@test Compat.cov([1 2; 3 4], [4; 5], dims=2) == hcat([0.5; 0.5])
@test Compat.cov([1 2; 3 4], [4; 5], corrected=false) == hcat([0.5; 0.5])
@test Compat.cov([1 2; 3 4], [4; 5], corrected=false, dims=1) == hcat([0.5; 0.5])
@test Compat.cov([1 2; 3 4], [4; 5], corrected=false, dims=2) == hcat([0.25; 0.25])
@test Compat.cor([1 2; 3 4]) ≈ [1 1; 1 1]
@test Compat.cor([1 2; 3 4], dims=1) ≈ [1 1; 1 1]
@test Compat.cor([1 2; 3 4], dims=2) ≈ [1 1; 1 1]
@test Compat.cor([1 2; 3 4], [4; 5]) ≈ [1; 1]
@test Compat.cor([1 2; 3 4], [4; 5], dims=1) ≈ [1; 1]
@test Compat.cor([1 2; 3 4], [4; 5], dims=2) ≈ [1; 1]
end
@test Compat.median([1 2; 3 4]) == 2.5
@test Compat.median([1 2; 3 4], dims=1) == [2 3]
@test Compat.median([1 2; 3 4], dims=2) == hcat([1.5; 3.5])
@test Compat.Statistics.varm([1 2; 3 4], -1) == 18
@test Compat.Statistics.varm([1 2; 3 4], [-1 -2], dims=1) == [20 52]
@test Compat.Statistics.varm([1 2; 3 4], [-1, -2], dims=2) == hcat([13, 61])
@test Compat.Statistics.var([1 2; 3 4]) == 5/3
@test Compat.Statistics.var([1 2; 3 4], dims=1) == [2 2]
@test Compat.Statistics.var([1 2; 3 4], dims=2) == hcat([0.5, 0.5])
@test Compat.Statistics.var([1 2; 3 4], corrected=false) == 1.25
@test Compat.Statistics.var([1 2; 3 4], corrected=false, dims=1) == [1 1]
@test Compat.Statistics.var([1 2; 3 4], corrected=false, dims=2) == hcat([0.25, 0.25])
@test Compat.Statistics.std([1 2; 3 4]) == sqrt(5/3)
@test Compat.Statistics.std([1 2; 3 4], dims=1) == [sqrt(2) sqrt(2)]
@test Compat.Statistics.std([1 2; 3 4], dims=2) == hcat([sqrt(0.5), sqrt(0.5)])
@test Compat.Statistics.std([1 2; 3 4], corrected=false) == sqrt(1.25)
@test Compat.Statistics.std([1 2; 3 4], corrected=false, dims=1) == [sqrt(1) sqrt(1)]
@test Compat.Statistics.std([1 2; 3 4], corrected=false, dims=2) == hcat([sqrt(0.25), sqrt(0.25)])
@test Compat.Statistics.cov([1 2; 3 4]) == [2 2; 2 2]
@test Compat.Statistics.cov([1 2; 3 4], dims=1) == [2 2; 2 2]
@test Compat.Statistics.cov([1 2; 3 4], dims=2) == [0.5 0.5; 0.5 0.5]
@test Compat.Statistics.cov([1 2; 3 4], [4; 5]) == hcat([1; 1])
@test Compat.Statistics.cov([1 2; 3 4], [4; 5], dims=1) == hcat([1; 1])
@test Compat.Statistics.cov([1 2; 3 4], [4; 5], dims=2) == hcat([0.5; 0.5])
@test Compat.Statistics.cov([1 2; 3 4], [4; 5], corrected=false) == hcat([0.5; 0.5])
@test Compat.Statistics.cov([1 2; 3 4], [4; 5], corrected=false, dims=1) == hcat([0.5; 0.5])
@test Compat.Statistics.cov([1 2; 3 4], [4; 5], corrected=false, dims=2) == hcat([0.25; 0.25])
@test Compat.Statistics.cor([1 2; 3 4]) ≈ [1 1; 1 1]
@test Compat.Statistics.cor([1 2; 3 4], dims=1) ≈ [1 1; 1 1]
@test Compat.Statistics.cor([1 2; 3 4], dims=2) ≈ [1 1; 1 1]
@test Compat.Statistics.cor([1 2; 3 4], [4; 5]) ≈ [1; 1]
@test Compat.Statistics.cor([1 2; 3 4], [4; 5], dims=1) ≈ [1; 1]
@test Compat.Statistics.cor([1 2; 3 4], [4; 5], dims=2) ≈ [1; 1]
end
@test Compat.Statistics.median([1 2; 3 4]) == 2.5
@test Compat.Statistics.median([1 2; 3 4], dims=1) == [2 3]
@test Compat.Statistics.median([1 2; 3 4], dims=2) == hcat([1.5; 3.5])
@test Compat.mapreduce(string, *, [1 2; 3 4]) == "1324"
Issue26488 && @test Compat.mapreduce(string, *, [1 2; 3 4], dims=1) == ["13" "24"]
Issue26488 && @test Compat.mapreduce(string, *, [1 2; 3 4], dims=2) == hcat(["12", "34"])
Expand Down