Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update codebase to 1.0 and 1.x (continue #27) #31

Merged
merged 36 commits into from
Jan 29, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
86fbdb5
add Project.toml
johnnychen94 Dec 15, 2019
42aa2f0
update test for cache and minor fix
johnnychen94 Dec 15, 2019
061ddc9
update test for rotation and minor fix
johnnychen94 Dec 15, 2019
e3b61db
update flipdm to reverse
johnnychen94 Dec 15, 2019
766744f
update test for crop and minor fix
johnnychen94 Dec 15, 2019
c471003
update test for resize
johnnychen94 Dec 15, 2019
1cbe9e4
update test for scale and minor fix
johnnychen94 Dec 15, 2019
75b433a
update test for zoom and minor fix
johnnychen94 Dec 15, 2019
3612f17
WIP: update test for distortion and minor fix
johnnychen94 Dec 15, 2019
970a691
update test for either and minor fix
johnnychen94 Dec 15, 2019
c666766
fix pipeline test
johnnychen94 Dec 16, 2019
e33c6ae
update summary reference for distortedview
johnnychen94 Dec 16, 2019
f096c05
WIP: fix tests for augment and operations
johnnychen94 Dec 16, 2019
8099666
drop julia 0.7 and update CI
johnnychen94 Dec 16, 2019
9d6a9b3
use FileIO v1.1.0
johnnychen94 Dec 16, 2019
d3e60f2
add 1.x tests to CI
johnnychen94 Dec 16, 2019
2e97f13
revert back unnecessary changes
johnnychen94 Dec 16, 2019
6c95157
test windows in travis
johnnychen94 Dec 16, 2019
4891a58
fix randomly failed test cases for inacuraccy reasons
johnnychen94 Dec 16, 2019
23ddbd1
unify how types are displayed
johnnychen94 Dec 16, 2019
dc3682e
update to julia 1.1
johnnychen94 Dec 16, 2019
4758e77
update to julia 1.2 -- part I
johnnychen94 Dec 16, 2019
fdbb15c
WIP: update to julia 1.2 -- part II
johnnychen94 Dec 16, 2019
2530508
add more test versions
johnnychen94 Dec 16, 2019
1587955
move safe_rand to compat.jl
johnnychen94 Dec 17, 2019
19aeb9e
don't allow failures for julia 1.3
johnnychen94 Dec 17, 2019
733e32b
fix commit "WIP: update 1.0"
johnnychen94 Dec 17, 2019
67e9784
use explicit and intuitive CartesianIndex(1, 1)
johnnychen94 Dec 17, 2019
a698382
add method specialization for tweight
johnnychen94 Jan 27, 2020
5f96ab5
allow FileIO 1.2
johnnychen94 Jan 27, 2020
61d8cee
fix type instability for Either
johnnychen94 Jan 28, 2020
2bf0975
update reference for FixedPointNumbers v0.7
johnnychen94 Jan 28, 2020
f483d2c
try: relax equality check for scale
johnnychen94 Jan 28, 2020
2a0c7c9
restore test cases in tst_scale.jl
johnnychen94 Jan 29, 2020
8443246
Revert "add method specialization for tweight"
johnnychen94 Jan 29, 2020
e915c94
Augmentor v0.6.0-pre
johnnychen94 Jan 29, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion src/Augmentor.jl
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ export

testpattern

include("compat.jl")
include("utils.jl")
include("types.jl")
include("operation.jl")
Expand Down Expand Up @@ -103,7 +104,10 @@ include("augment.jl")
include("augmentbatch.jl")

function __init__()
rand_mutex[] = Threads.Mutex()
if VERSION < v"1.3"
# see compat.jl
rand_mutex[] = Threads.Mutex()
end
end

end # module
22 changes: 22 additions & 0 deletions src/compat.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
if VERSION >= v"1.3"
const safe_rand = rand
else
# rand is thread safe after julia 1.3
# TODO: delete this when we decide to drop 1.0 compatibility

# --------------------------------------------------------------------
# rand() is not threadsafe (https://discourse.julialang.org/t/4683)

# Because we only require random numbers to sample parameters
# and not the actual expensive computation, this seems like a better
# approach than using separate RNG per thread.
const rand_mutex = Ref{Threads.Mutex}()

# constant overhead of about 80 ns compared to unsafe rand
function safe_rand(args...)
lock(rand_mutex[])
result = rand(args...)
unlock(rand_mutex[])
result
end
end
16 changes: 0 additions & 16 deletions src/utils.jl
Original file line number Diff line number Diff line change
Expand Up @@ -15,22 +15,6 @@ function use_testpattern()
testpattern()
end

# --------------------------------------------------------------------
# rand() is not threadsafe (https://discourse.julialang.org/t/4683)

# Because we only require random numbers to sample parameters
# and not the actual expensive computation, this seems like a better
# approach than using separate RNG per thread.
const rand_mutex = Ref{Threads.Mutex}()

# constant overhead of about 80 ns compared to unsafe rand
function safe_rand(args...)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little worried here that I might misunderstand what "threadsafe" means here. Is that mean the following two scripts should output the same results?

Random.seed!(0)
Threads.@threads for i in 1:5
    println(rand())
end
Random.seed!(0)
for i in 1:5
    println(rand())
end

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no. This has no influence on the ordering that threads invoke save_rand. It really just is about the function being threadsafe so that consecutive return values (regardless of calling thread) can be considered random

lock(rand_mutex[])
result = rand(args...)
unlock(rand_mutex[])
result
end

# --------------------------------------------------------------------
"""
contiguous(A::AbstractArray)
Expand Down
1 change: 1 addition & 0 deletions test/runtests.jl
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ checkers = Gray{N0f8}[1 0 1 0 1; 0 1 0 1 0; 1 0 1 0 1]
rgb_rect = rand(RGB{N0f8}, 2, 3)

tests = [
"tst_compat.jl",
"tst_utils.jl",
"operations/tst_channels.jl",
"operations/tst_dims.jl",
Expand Down
16 changes: 16 additions & 0 deletions test/tst_compat.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# TODO: remove this testset when we dicide to drop 1.0 compatibility
if VERSION < v"1.3"
@testset "safe_rand" begin
mutex = Augmentor.rand_mutex[]
typeof(mutex) <: Threads.Mutex
# check that its not a null pointer
@test reinterpret(Int, mutex.handle) > 0

num = @inferred Augmentor.safe_rand()
@test 0 <= num <= 1
@test typeof(num) <: Float64
num = @inferred Augmentor.safe_rand(2)
@test all(0 .<= num .<= 1)
@test typeof(num) <: Vector{Float64}
end
end
16 changes: 0 additions & 16 deletions test/tst_utils.jl
Original file line number Diff line number Diff line change
Expand Up @@ -7,22 +7,6 @@
@test tp == tp2
end

@testset "rand_mutex" begin
mutex = Augmentor.rand_mutex[]
typeof(mutex) <: Threads.Mutex
# check that its not a null pointer
@test reinterpret(Int, mutex.handle) > 0
end

@testset "safe_rand" begin
num = @inferred Augmentor.safe_rand()
@test 0 <= num <= 1
@test typeof(num) <: Float64
num = @inferred Augmentor.safe_rand(2)
@test all(0 .<= num .<= 1)
@test typeof(num) <: Vector{Float64}
end

@testset "contiguous" begin
A = [1 2 3; 4 5 6; 7 8 9]
Ao = OffsetArray(A, (-2,-1))
Expand Down