add missing Pandoc dependency for DROP 1.0.3 #12801

smoors · 2021-05-03T14:51:48Z

(created using eb --new-pr)

Micket

Is this truly a runtime dep? I would have thought that it maybe was a builddep for documentation or some such?

smoors · 2021-05-04T09:02:23Z

yes it is, for creating reports, dependency graphs and such.

not everyone will need it, but it's just one extra dep so not a big deal, or do you prefer to comment it out?

Micket · 2021-05-04T12:11:31Z

Well, it does mean that we end up with amd64 support only, since Pandoc is a binary package. Perhaps best to keep it commented out for this reason?

boegel · 2021-05-04T12:19:11Z

easybuild/easyconfigs/p/Pandoc/Pandoc-2.13.eb

+toolchain = SYSTEM
+
+source_urls = ['https://github.com/jgm/pandoc/releases/download/%(version)s/']
+sources = ['%(namelower)s-%(version)s-linux-amd64.tar.gz']


Hmm, we should probably use a versionsuffix here, since they (now?) provide arm64 builds too.

If so, wouldn't it be better to provide multiple sources here instead?

@smoors See smoors#5

There's a better option, we can avoid the versionsuffix for Pandoc, and stick to a single easyconfig file that supports both x86_64 and aarch64, but we need easybuilders/easybuild-framework#3670 for that...

Huh, I thought we could do that already? Is it (currently) just implemented as a template variable for a few selected easyblocks (like Java) before?

I'm also a bit confused.
a lot of easyconfigs already use %(arch)s in their sources string, why can't we use that here too?

is it because we cannot translate %(arch)s into amd64 or arm64 in the easyconfig?

Hm, I thought we even supported using a dictionary in sources, but maybe that's just dependencies (like Java) or checksums (like CUDA) does?

@Micket changed to use the ARCH constant, now that easybuilders/easybuild-framework#3670 is merged.
Java uses 'arch=xxx': 'yyy' syntax, but that's indeed only supported for dependencies
other easyconfigs use the %(arch)s template for sources, but that's only possible if the sources have the exact %(arch)s string in their name, since the template is only resolved after parsing the easyconfig.

…e %(arch)s Pandoc dependency for DROP

add versionsuffix to Pandoc 2.13 to indicate target architecture + use %(arch)s Pandoc dependency for DROP

… architectures

changes done

Micket · 2021-05-07T16:31:23Z

easybuild/easyconfigs/p/Pandoc/Pandoc-2.13.eb

+source_urls = ['https://github.com/jgm/pandoc/releases/download/%(version)s/']
+_archs = {'x86_64': 'amd64', 'aarch64': 'arm64'}
+sources = ['%%(namelower)s-%%(version)s-linux-%s.tar.gz' % _archs[ARCH]]
+checksums = ['7404aa88a6eb9fbb99d9803b80170a3a546f51959230cc529c66a2ce6b950d4c']


We need multiple checksums for x86_64+aarch64

Micket · 2021-05-07T16:32:50Z

easybuild/easyconfigs/d/DROP/DROP-1.0.3-foss-2020b-R-4.0.3.eb

@@ -23,6 +23,7 @@ dependencies = [
    ('HTSlib', '1.11'),  # for tabix
    ('SAMtools', '1.11'),
    ('BCFtools', '1.11'),
+    ('Pandoc', '2.13', '', True),


I feel it might still be worth leaving it commented, if it's was a niche usecase, and the fact that our PowerPC friends can't use it still. I don't know the software well enough, so, I'll leave it up to you to decide what's best.

There's still --filter-deps to easily opt out of depending on Pandoc, so I'd leave it in...

boegel · 2021-05-10T08:03:21Z

Test report by @boegel
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
node3513.doduo.os - Linux RHEL 8.2, x86_64, AMD EPYC 7552 48-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/b5f89be7a33cb3c13ed310a504f8c4e1 for a full test report.

boegel · 2021-05-10T08:11:44Z

Test report by @boegel
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
node3139.skitty.os - Linux centos linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz, Python 3.6.8
See https://gist.github.com/33a0404367d8c09d7ce14ad5f92e1247 for a full test report.

boegel · 2021-05-10T13:16:24Z

@boegelbot please test @ generoso

boegelbot · 2021-05-10T13:20:08Z

@boegel: Request for testing this PR well received on generoso

PR test command 'EB_PR=12801 EB_ARGS= /apps/slurm/default/bin/sbatch --job-name test_PR_12801 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 17072

Test results coming soon (I hope)...

- notification for comment with ID 836686884 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2021-05-10T13:20:44Z

Test report by @boegelbot
FAILED
Build succeeded for 1 out of 3 (2 easyconfigs in total)
generoso-c1-s-2 - Linux centos linux 8.2.2004, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/692442974c3dbbdf656289da674fe9e6 for a full test report.

boegel · 2021-05-10T13:26:27Z

Test report by @boegel
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
select-pika-c6gd-2xlarge-0001 - Linux centos linux 8.3.2011, AArch64, ARM UNKNOWN (graviton2), Python 3.6.8
See https://gist.github.com/5736c63d5e10881e1763c1fb256d77f3 for a full test report.

boegel · 2021-05-10T13:48:20Z

Test on generoso failed due to clash with running test for #12836, I'll re-trigger it later...

boegel · 2021-05-10T16:57:58Z

@boegelbot please test @ generoso

boegelbot · 2021-05-10T17:00:09Z

@boegel: Request for testing this PR well received on generoso

PR test command 'EB_PR=12801 EB_ARGS= /apps/slurm/default/bin/sbatch --job-name test_PR_12801 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 17074

Test results coming soon (I hope)...

- notification for comment with ID 836966357 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2021-05-10T17:02:47Z

Test report by @boegelbot
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
generoso-c1-s-1 - Linux centos linux 8.2.2004, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/b6192bef3525252722bb60d57abbb7ce for a full test report.

multiple checksums added

boegel

lgtm

boegel · 2021-05-10T17:21:26Z

Going in, thanks @smoors!

add missing Pandoc dependency for DROP 1.0.3

193a006

smoors added update bug fix labels May 3, 2021

smoors added this to the next release (4.3.5?) milestone May 3, 2021

Micket reviewed May 3, 2021

View reviewed changes

Micket mentioned this pull request May 4, 2021

{tools}[system/system] Pandoc v2.13 #12809

Closed

boegel previously requested changes May 4, 2021

View reviewed changes

add versionsuffix to Pandoc 2.13 to indicate target architecture + us…

375b9f9

…e %(arch)s Pandoc dependency for DROP

boegel mentioned this pull request May 4, 2021

add versionsuffix to Pandoc 2.13 to indicate target architecture + use %(arch)s Pandoc dependency for DROP smoors/easybuild-easyconfigs#5

Merged

smoors and others added 2 commits May 4, 2021 18:22

Merge pull request #5 from boegel/20210503165142_new_pr_DROP103

c009644

add versionsuffix to Pandoc 2.13 to indicate target architecture + use %(arch)s Pandoc dependency for DROP

use ARCH constant to avoid different Pandoc easyconfigs for different…

66d54ca

… architectures

remove arch-specific Pandoc easyconfigs

9231da3

Micket previously requested changes May 7, 2021

View reviewed changes

specify both aarch64 and x86_64 checksum in Pandoc 2.13 easyconfig

23e83a3

boegel approved these changes May 10, 2021

View reviewed changes

boegel merged commit d965b86 into easybuilders:develop May 10, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add missing Pandoc dependency for DROP 1.0.3 #12801

add missing Pandoc dependency for DROP 1.0.3 #12801

smoors commented May 3, 2021

Micket left a comment

smoors commented May 4, 2021

Micket commented May 4, 2021

boegel May 4, 2021

Micket May 4, 2021

boegel May 4, 2021

boegel May 4, 2021

Micket May 4, 2021

smoors May 4, 2021

smoors May 4, 2021

Micket May 4, 2021

smoors May 7, 2021

Micket May 7, 2021

Micket May 7, 2021

boegel May 10, 2021

boegel commented May 10, 2021

boegel commented May 10, 2021

boegel commented May 10, 2021

boegelbot commented May 10, 2021

boegelbot commented May 10, 2021

boegel commented May 10, 2021

boegel commented May 10, 2021

boegel commented May 10, 2021

boegelbot commented May 10, 2021

boegelbot commented May 10, 2021

boegel left a comment

boegel commented May 10, 2021

add missing Pandoc dependency for DROP 1.0.3 #12801

add missing Pandoc dependency for DROP 1.0.3 #12801

Conversation

smoors commented May 3, 2021

Micket left a comment

Choose a reason for hiding this comment

smoors commented May 4, 2021

Micket commented May 4, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

boegel commented May 10, 2021

boegel commented May 10, 2021

boegel commented May 10, 2021

boegelbot commented May 10, 2021

boegelbot commented May 10, 2021

boegel commented May 10, 2021

boegel commented May 10, 2021

boegel commented May 10, 2021

boegelbot commented May 10, 2021

boegelbot commented May 10, 2021

boegel left a comment

Choose a reason for hiding this comment

boegel commented May 10, 2021