[v24.1.x] cleanup tiered storage temporary cache file if exceptions are thrown during download

[vbotbuildovich]

Backport #24000 to branch v24.1.x. This issue was created as a placeholder because the original PR's commit(s) could not be automatically cherry-picked.
Here are the commands to execute:

git checkout v24.1.x
git checkout -b manual-backport-24000-v24.1.x-504
git cherry-pick -x 6e1e998021 be77f709bc 5a93114f8a 50fef29c79 b34aab6da1 e90eafbbfb

git push origin manual-backport-24000-v24.1.x-504
gh pr create \
  --title "[v24.1.x] cleanup tiered storage temporary cache file if exceptions are thrown during download" \
  --base "v24.1.x" \
  --label "kind/backport" \
  --head "manual-backport-24000-v24.1.x-504" \
  --draft \
  --repo "redpanda-data/redpanda" \
  --milestone "v24.1.x-next" \
  --body "Backport of PR https://github.com/redpanda-data/redpanda/pull/24000 "