Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restoring a backup with no pods is PartiallyFailed with error: fail to patch dynamic PV, err: context deadline exceeded #7866

Closed
shubham-pampattiwar opened this issue Jun 7, 2024 · 3 comments · Fixed by #7953
Assignees

Comments

@shubham-pampattiwar
Copy link
Collaborator

shubham-pampattiwar commented Jun 7, 2024

What steps did you take and what happened:

  • Install Velero 1.14 (rc in this case)
  • Create a backup with no pods(scaled down the replicas to 0 for deployments involved), but include the volumes (via csi snapshot method, non-datamover)
  • Delete application ns
  • Restore from the backup
  • Restore is partially failed with fail to patch dynamic PV, err: context deadline exceeded
  • Note that the StorageClass used in this scenario has VolumeBindingMode as WaitForFirstConsumer

What did you expect to happen:

  • In earlier versions of velero, ie. before velero 1.14, the restore described above was not marked PartiallyFailed because there was no Finalizing Phase for Restore Operation that tried to patch the PV
  • In our scenario as the StorageClass has VolumeBindingMode as WaitForFirstConsumer, the PV will be in pending state as there are no pods in the backup and this is turn will fail the PV patch part in restore operation after the timeout, thus marking the Restore partially failed.

The following information will help us better understand what's going on:

If you are using velero v1.7.0+:
Please use velero debug --backup <backupname> --restore <restorename> to generate the support bundle, and attach to this issue, more options please refer to velero debug --help

If you are using earlier versions:
Please provide the output of the following commands (Pasting long output into a GitHub gist or other pastebin is fine.)

  • kubectl logs deployment/velero -n velero
  • velero backup describe <backupname> or kubectl get backup/<backupname> -n velero -o yaml
  • velero backup logs <backupname>
  • velero restore describe <restorename> or kubectl get restore/<restorename> -n velero -o yaml
  • velero restore logs <restorename>

Anything else you would like to add:

Environment:

  • Velero version (use velero version):
  • Velero features (use velero client config get features):
  • Kubernetes version (use kubectl version):
  • Kubernetes installer & version:
  • Cloud provider or hardware configuration:
  • OS (e.g. from /etc/os-release):

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

  • 👍 for "I would like to see this bug fixed as soon as possible"
  • 👎 for "There are more important bugs to focus on right now"
@shubham-pampattiwar
Copy link
Collaborator Author

/cc: @sseago @weshayutin

@shubham-pampattiwar
Copy link
Collaborator Author

Re-opening for 1.14 cherry-pick PR #8006

@Lyndon-Li
Copy link
Contributor

Close as completed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment