Use curl instead of wget for Spark and Julia downloads

[mathbunnyru]

Describe your changes

Issue ticket if applicable

Checklist (especially for first-time contributors)

• I have performed a self-review of my code
• If it is a core feature, I have added thorough tests
• I will try not to use force-push to make the review process easier for reviewers
• I have updated the documentation for significant changes

[mathbunnyru]

I used nmap --script ssl-enum-ciphers -p 443 URL to check both spark and julia download websites protocols.
https://stackoverflow.com/a/55764641/4881441

And then forced wget to use exactly such protocol.
https://stackoverflow.com/a/55764641/4881441

[mathbunnyru]

dlcdn.apache.org supports both TLSv1.2 and TLSv1.3. I will use v3.

[bjornjorgensen]

but its it secure when we use https instead of http

[mathbunnyru]

but its it secure when we use https instead of http

Sorry, I don't understand what you're saying.

[bjornjorgensen]

what does "Force tls version on some wget downloads" have to do with run-one-until-success

[mathbunnyru]

what does "Force tls version on some wget downloads" have to do with run-one-until-success

Thanks, I changed the PR name to something more appropriate.
I tried a few solutions here, but they don't seem to work fine.

[bjornjorgensen]

is run-one-until-success only for ubuntu and debian?
Are there any reasons why we need it?

[mathbunnyru]

is run-one-until-success only for ubuntu and debian?

run-one-until-success is a part of run-one package.
I don't know which OSs provide such a package.

Are there any reasons why we need it?

We need it because spark and julia downloads fail on aarch64 machines.
I tried to understand why it happens and tried several solutions.
I don't know the exact reason and if someone has a better solution, I would really appreciate a proper fix.
My attempts haven't fixed the problem.
So, run-one-until-success mitigates it in a simple manner.

We already install run-one package to support RESTARTABLE option, so the image size is not affected.

The only downside I can see is that when spark or julia downloads actually fail, we will get timeout errors, instead of download errors. Unfortunately, there is no attempts parameter to pass to run-one-until-success.

[mathbunnyru]

Retries work well: https://github.com/jupyter/docker-stacks/actions/runs/5694780636/job/15436934951?pr=1950#step:7:40

[bjornjorgensen]

so this is a bug for wget on aarch64?

can we try replace wget --progress=dot:giga "https://julialang-s3.julialang.org/bin/linux/${JULIA_SHORT_ARCH}/${JULIA_MAJOR_MINOR}/${JULIA_INSTALLER}"

curl -o ${JULIA_INSTALLER} -L "https://julialang-s3.julialang.org/bin/linux/${JULIA_SHORT_ARCH}/${JULIA_MAJOR_MINOR}/${JULIA_INSTALLER}" --progress-bar

[mathbunnyru]

@bjornjorgensen I've tried to run it manually:

runner-user@jupyter-runner-1:~$ curl --output julia.tgz https://julialang-s3.julialang.org/bin/linux/x64/1.9/julia-1.9.2-linux-x86_64.tar.gz --progress-bar
##O=- #     #

I ran it with verbose as well:
logs.txt

[bjornjorgensen]

you must add -L

curl -L -o ${JULIA_INSTALLER} "https://julialang-s3.julialang.org/bin/linux/${JULIA_SHORT_ARCH}/${JULIA_MAJOR_MINOR}/${JULIA_INSTALLER}" --progress-bar

or

curl -o ${JULIA_INSTALLER} -L "https://julialang-s3.julialang.org/bin/linux/${JULIA_SHORT_ARCH}/${JULIA_MAJOR_MINOR}/${JULIA_INSTALLER}" --progress-bar

[mathbunnyru]

Thank you so much @bjornjorgensen!
Could you please take a look here?
Will curl with -L fix spark download as well?
https://github.com/jupyter/docker-stacks/actions/runs/5674349686/job/15378214762#step:7:289

[bjornjorgensen]

"The -L flag tells curl to follow any redirects, so it will continue to the new location provided by the server. This is an important flag to include when downloading files, because the URL you are trying to download from might be a redirect to the actual file's URL."

I hope so.. :)

[bjornjorgensen]

it works.. if you update the title for this PR I will approve it 👍 and Thank you :)

[mathbunnyru]

Thank you @bjornjorgensen.
I added you as a co-author for the squashed commit.