Skip to content

Commit

Permalink
AVRO-1514: Clean up Perl dependencies (#2962)
Browse files Browse the repository at this point in the history 
AVRO-1514: 

* Sort Perl dependencies in Github action

This minimises the chance of duplicates sneaking by.

* Drop system dependencies in Perl Github action

* Manually set Perl repository metadata

* Drop dependency on IO::String

* Add missing dependency on Test::Pod

* Mark Test::More as a test dependency

* Add Module::Install as a configure requires

* Be explicit about JSON::MaybeXS dependency

JSON::MaybeXS comes installed in the Perl container we get from
the perl setup Github action, but it's probably a good idea to
be explicit about it.
  • Loading branch information
@jjatria
jjatria authored Jun 24, 2024
1 parent 7bb9827 commit 6f7be10
Show file tree
Hide file tree
Showing 8 changed files with 82 additions and 93 deletions.
57 changes: 22 additions & 35 deletions .github/workflows/test-lang-perl-ARM.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -49,29 +49,19 @@ jobs:

- name: Install Dependencies
run: |
sudo apt-get -qqy install --no-install-recommends libjansson-dev \
libcompress-raw-zlib-perl \
libcpan-uploader-perl \
libencode-perl \
libio-string-perl \
libjson-xs-perl \
libmodule-install-perl \
libmodule-install-readmefrompod-perl \
libobject-tiny-perl \
libperl-critic-perl \
libsnappy-dev \
libtest-exception-perl \
libtest-pod-perl
cpanm --mirror https://www.cpan.org/ install Compress::Zstd \
Encode \
Error::Simple \
Module::Install::Repository \
Regexp::Common \
Try::Tiny \
inc::Module::Install \
JSON::MaybeXS \
Module::Install \
Module::Install::ReadmeFromPod \
Object::Tiny \
Perl::Critic \
Regexp::Common \
Test::Exception \
IO::String \
Object::Tiny
Test::More \
Test::Pod \
Try::Tiny
- name: Lint
run: ./build.sh lint
Expand All @@ -95,25 +85,22 @@ jobs:

- name: Install Dependencies
run: |
sudo apt-get -qqy install --no-install-recommends libcompress-raw-zlib-perl \
libcpan-uploader-perl \
libencode-perl \
libio-string-perl \
libjansson-dev \
libjson-xs-perl \
libmodule-install-perl \
libmodule-install-readmefrompod-perl \
libobject-tiny-perl \
libsnappy-dev \
libtest-exception-perl \
libtest-pod-perl
cpanm --mirror https://www.cpan.org/ install Compress::Zstd \
sudo apt-get -qqy install --no-install-recommends libjansson-dev \
libsnappy-dev
cpanm --mirror https://www.cpan.org/ install CPAN::Uploader \
Compress::Zstd \
Encode \
Error::Simple \
Module::Install::Repository \
JSON::MaybeXS \
Module::Install \
Module::Install::ReadmeFromPod \
Object::Tiny \
Perl::Critic \
Regexp::Common \
Try::Tiny \
inc::Module::Install
Test::Exception \
Test::More \
Test::Pod \
Try::Tiny
- name: Cache Local Maven Repository
uses: actions/cache@v4
Expand Down
56 changes: 21 additions & 35 deletions .github/workflows/test-lang-perl.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -49,29 +49,19 @@ jobs:

- name: Install Dependencies
run: |
sudo apt-get -qqy install --no-install-recommends libjansson-dev \
libcompress-raw-zlib-perl \
libcpan-uploader-perl \
libencode-perl \
libio-string-perl \
libjson-xs-perl \
libmodule-install-perl \
libmodule-install-readmefrompod-perl \
libobject-tiny-perl \
libperl-critic-perl \
libsnappy-dev \
libtest-exception-perl \
libtest-pod-perl
cpanm --mirror https://www.cpan.org/ install Compress::Zstd \
Encode \
Error::Simple \
Module::Install::Repository \
Regexp::Common \
Try::Tiny \
inc::Module::Install \
JSON::MaybeXS \
Module::Install \
Module::Install::ReadmeFromPod \
Object::Tiny \
Perl::Critic \
Regexp::Common \
Test::Exception \
IO::String \
Object::Tiny
Test::More \
Test::Pod \
Try::Tiny
- name: Lint
run: ./build.sh lint
Expand All @@ -95,25 +85,21 @@ jobs:

- name: Install Dependencies
run: |
sudo apt-get -qqy install --no-install-recommends libcompress-raw-zlib-perl \
libcpan-uploader-perl \
libencode-perl \
libio-string-perl \
libjansson-dev \
libjson-xs-perl \
libmodule-install-perl \
libmodule-install-readmefrompod-perl \
libobject-tiny-perl \
libsnappy-dev \
libtest-exception-perl \
libtest-pod-perl
cpanm --mirror https://www.cpan.org/ install Compress::Zstd \
sudo apt-get -qqy install --no-install-recommends libjansson-dev \
libsnappy-dev
cpanm --mirror https://www.cpan.org/ install CPAN::Uploader \
Compress::Zstd \
Encode \
Error::Simple \
Module::Install::Repository \
JSON::MaybeXS \
Module::Install \
Module::Install::ReadmeFromPod \
Object::Tiny \
Regexp::Common \
Try::Tiny \
inc::Module::Install
Test::Exception \
Test::More \
Test::Pod \
Try::Tiny
- name: Cache Local Maven Repository
uses: actions/cache@v4
Expand Down
4 changes: 2 additions & 2 deletions BUILD.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ The following packages must be installed before Avro can be built:
- Perl: Perl 5.24.1 or greater, gmake, Module::Install,
Module::Install::ReadmeFromPod, Module::Install::Repository,
Math::BigInt, JSON::MaybeXS, Try::Tiny, Regexp::Common, Encode,
IO::String, Object::Tiny, Compress::ZLib, Error::Simple,
Test::More, Test::Exception, Test::Pod
Object::Tiny, Compress::ZLib, Error::Simple, Test::More,
Test::Exception, Test::Pod
- Rust: rustc and Cargo 1.65.0 or greater
- Apache Ant 1.7
- md5sum, sha1sum, used by top-level dist target
Expand Down
2 changes: 2 additions & 0 deletions lang/perl/Changes
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ Revision history for Perl extension Avro

- Switch from JSON::XS to JSON::MaybeXS to support
multiple JSON backends
- Drop dependency on IO::String, since we don't need
it now we depend on Perl 5.10.1 or greater

1.00 Fri Jan 17 15:00:00 2014
- Relicense under apache license 2.0
Expand Down
1 change: 0 additions & 1 deletion lang/perl/MANIFEST
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ inc/Module/Install/Makefile.pm
inc/Module/Install/MakeMaker.pm
inc/Module/Install/Metadata.pm
inc/Module/Install/ReadmeFromPod.pm
inc/Module/Install/Repository.pm
lib/Avro.pm
lib/Avro/BinaryDecoder.pm
lib/Avro/BinaryEncoder.pm
Expand Down
14 changes: 11 additions & 3 deletions lang/perl/Makefile.PL
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,17 +24,26 @@ chomp $version;

license 'apache';
version $version;
homepage 'http://avro.apache.org/';
repository {
type => 'git',
url => 'git://git.apache.org/avro.git',
web => 'http://github.com/apache/avro',
};
bugtracker { web => 'http://issues.apache.org/jira/browse/AVRO/' };
readme_from 'lib/Avro.pm';
all_from 'lib/Avro.pm';
build_requires 'Test::More', 0.88;
configure_requires 'Module::Install';
configure_requires 'Module::Install::ReadmeFromPod';
test_requires 'Math::BigInt';
test_requires 'Perl::Critic';
test_requires 'Test::Exception';
test_requires 'Test::More', 0.88;
test_requires 'Test::Pod';
requires 'Compress::Zlib';
requires 'Compress::Zstd';
requires 'Encode';
requires 'Error::Simple';
requires 'IO::String';
requires 'JSON::MaybeXS';
requires 'Object::Tiny';
requires 'Regexp::Common';
Expand All @@ -43,7 +52,6 @@ requires 'parent';
unless ($Config{use64bitint}) {
requires 'Math::BigInt';
}
auto_set_repository();

my %packages = (
'Avro' => 'lib/Avro.pm',
Expand Down
6 changes: 3 additions & 3 deletions lang/perl/lib/Avro/BinaryDecoder.pm
View file
Original file line number Diff line number Diff line change
Expand Up @@ -49,9 +49,9 @@ The schema we want to use to decode the data.
=item * reader
An object implementing a straightforward interface. C<read($buf, $nbytes)> and
C<seek($nbytes, $whence)> are expected. Typically a IO::String object or a
IO::File object. It is expected that this calls will block the decoder, if not
A file handle, or an object implementing a similar interface, like L<IO::File>.
Specifically, it must support C<read($buf, $nbytes)> and
C<seek($nbytes, $whence)>. These calls will block the decoder if not
enough data is available for read.
=back
Expand Down
35 changes: 21 additions & 14 deletions lang/perl/t/03_bin_decode.t
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,15 +23,14 @@ use Avro::Schema;
use Avro::BinaryEncoder;
use Test::More;
use Test::Exception;
use IO::String;

use_ok 'Avro::BinaryDecoder';

## spec examples
{
my $enc = "\x06\x66\x6f\x6f";
my $schema = Avro::Schema->parse(q({ "type": "string" }));
my $reader = IO::String->new($enc);
open my $reader, '<', \$enc or die "Can't open memory file: $!";
my $dec = Avro::BinaryDecoder->decode(
writer_schema => $schema,
reader_schema => $schema,
Expand All @@ -49,7 +48,7 @@ use_ok 'Avro::BinaryDecoder';
]
}
EOJ
$reader = IO::String->new("\x36\x06\x66\x6f\x6f");
open $reader, '<', \"\x36\x06\x66\x6f\x6f" or die "Can't open memory file: $!";
$dec = Avro::BinaryDecoder->decode(
writer_schema => $schema,
reader_schema => $schema,
Expand All @@ -58,7 +57,7 @@ EOJ
is_deeply $dec, { a => 27, b => 'foo' },
"Binary_Encodings.Complex_Types.Records";

$reader = IO::String->new("\x04\x06\x36\x00");
open $reader, '<', \"\x04\x06\x36\x00" or die "Can't open memory file: $!";
$schema = Avro::Schema->parse(q({"type": "array", "items": "long"}));
$dec = Avro::BinaryDecoder->decode(
writer_schema => $schema,
Expand All @@ -67,7 +66,7 @@ EOJ
);
is_deeply $dec, [3, 27], "Binary_Encodings.Complex_Types.Arrays";

$reader = IO::String->new("\x02");
open $reader, '<', \"\x02" or die "Can't open memory file: $!";
$schema = Avro::Schema->parse(q(["string","null"]));
$dec = Avro::BinaryDecoder->decode(
writer_schema => $schema,
Expand All @@ -76,7 +75,7 @@ EOJ
);
is $dec, undef, "Binary_Encodings.Complex_Types.Unions-null";

$reader = IO::String->new("\x00\x02\x61");
open $reader, '<', \"\x00\x02\x61" or die "Can't open memory file: $!";
$dec = Avro::BinaryDecoder->decode(
writer_schema => $schema,
reader_schema => $schema,
Expand All @@ -102,10 +101,11 @@ EOP
data => $data,
emit_cb => sub { $enc = ${ $_[0] } },
);
open my $reader, '<', \$enc or die "Cannot open memory file: $!";
my $dec = Avro::BinaryDecoder->decode(
writer_schema => $w_enum,
reader_schema => $r_enum,
reader => IO::String->new($enc),
reader => $reader,
);
is $dec, $data, "decoded!";
}
Expand All @@ -116,10 +116,11 @@ EOP
data => $data,
emit_cb => sub { $enc = ${ $_[0] } },
);
open my $reader, '<', \$enc or die "Cannot open memory file: $!";
throws_ok { Avro::BinaryDecoder->decode(
writer_schema => $w_enum,
reader_schema => $r_enum,
reader => IO::String->new($enc),
reader => $reader,
)} "Avro::Schema::Error::Mismatch", "schema problem";
}
}
Expand Down Expand Up @@ -147,22 +148,24 @@ EOJ
data => $data,
emit_cb => sub { $enc .= ${ $_[0] } },
);
open my $reader, '<', \$enc or die "Cannot open memory file: $!";
my $dec = Avro::BinaryDecoder->decode(
writer_schema => $w_schema,
reader_schema => $r_schema,
reader => IO::String->new($enc),
reader => $reader,
);
is $dec->{a}, 1, "easy";
ok ! exists $dec->{bonus}, "bonus extra field ignored";
is $dec->{t}, 37.5, "default t from reader used";

## delete the default for t
delete $r_schema->fields->[0]{default};
open $reader, '<', \$enc or die "Cannot open memory file: $!";
throws_ok {
Avro::BinaryDecoder->decode(
writer_schema => $w_schema,
reader_schema => $r_schema,
reader => IO::String->new($enc),
reader => $reader,
);
} "Avro::Schema::Error::Mismatch", "no default value!";
}
Expand All @@ -182,10 +185,11 @@ EOP
data => $data,
emit_cb => sub { $enc .= ${ $_[0] } },
);
open my $reader, '<', \$enc or die "Cannot open memory file: $!";
my $dec = Avro::BinaryDecoder->decode(
writer_schema => $w_schema,
reader_schema => $r_schema,
reader => IO::String->new($enc),
reader => $reader,
);

is_deeply $dec, $data, "decoded!";
Expand All @@ -207,18 +211,20 @@ EOP
data => $data,
emit_cb => sub { $enc .= ${ $_[0] } },
);
open my $reader, '<', \$enc or die "Cannot open memory file: $!";
throws_ok {
Avro::BinaryDecoder->decode(
writer_schema => $w_schema,
reader_schema => $r_schema,
reader => IO::String->new($enc),
reader => $reader,
)
} "Avro::Schema::Error::Mismatch", "recursively... fails";

open $reader, '<', \$enc or die "Cannot open memory file: $!";
my $dec = Avro::BinaryDecoder->decode(
writer_schema => $w_schema,
reader_schema => $w_schema,
reader => IO::String->new($enc),
reader => $reader,
);
is_deeply $dec, $data, "decoded succeeded!";
}
Expand All @@ -239,10 +245,11 @@ EOP
data => $data,
emit_cb => sub { $enc .= ${ $_[0] } },
);
open my $reader, '<', \$enc or die "Cannot open memory file: $!";
my $dec = Avro::BinaryDecoder->decode(
writer_schema => $w_schema,
reader_schema => $w_schema,
reader => IO::String->new($enc),
reader => $reader,
);
is_deeply $dec, $data, "decoded succeeded! +upgrade";
is $dec->{one}[0], 1.0, "kind of dumb test";
Expand Down

0 comments on commit 6f7be10

Please sign in to comment.