mtime check causing mm-source-mtime error during mirroring while files are identical

[deplab]

Expected behavior

Hello,

Hope you can help.

I suspect that #3121 change caused the following issue:

When mirroring files from minio A to minio B the following error appears:

mc: <ERROR> Failed to perform mirroring, with error condition (mm-source-mtime) Overwrite not allowed for `http://example-url:9000/bucket-name/path/filename.pdf`. Use `--overwrite` to override this behavior.

May I ask you:

1. Could we ignore modified time when comparing the files because modified time in source minio A is different from destination minio B, because mtime in minio A is when the file was modified on minio A while mtime on minio B is when a file was copied to minio B?
2. The checksums for the same files in minio A are identical with minio B, could we compare checksums instead as valid source for file comparison, rather than mtime which may differ for the same file when a file gets copied?

Actual behavior

When mirroring files from minio A to minio B the following error appears:

mc: <ERROR> Failed to perform mirroring, with error condition (mm-source-mtime) Overwrite not allowed for `http://example-url:9000/bucket-name/path/filename.pdf`. Use `--overwrite` to override this behavior.

Steps to reproduce the behavior

Both minio A and minio B are running as containers in a pod in Kubernetes cluster. There is valid network policy between minio A and minio B

kubectl --namespace=namespace-name exec -i "${minio-A}" sh <<EOS
     mc config host add --quiet "${name}" "http://${minio-B-endpoint}" ${minio_B_access_key} ${minio_B_secret_key}
     mc mb -p "${name}/bucket-name"
     mc mirror /storage/bucket-name "${name}/bucket-name"
EOS

mc --version

mc version RELEASE.2020-11-25T23-04-07Z

System information

• Kubernetes
• Container is run with dockerhub image minio:RELEASE.2020-11-25T22-36-25Z

  NAME="Alpine Linux"
  ID=alpine
  VERSION_ID=3.12.0
  PRETTY_NAME="Alpine Linux v3.12"
  HOME_URL="https://alpinelinux.org/"
  BUG_REPORT_URL="https://bugs.alpinelinux.org/"
  

[ebozduman]

@deplab ,

Did you try the recommended --overwrite flag?
When you use the flag, do you see the behavior you expect?
If the answer is a "No", then this is a potential bug, if not, then this is how it works.

First of all, we don't use checksum, since it is quite expensive, especially when there are thousands of objects in your MinIO.

So, we have to rely on the object name, size, and modification time.

We support the scenario where a user wants to keep only the latest version of an object.
That is; we choose to keep the latest version of the object, whichever has the later modification time and this is the default behavior.
In your scenario, since you don't care about it, you want to override the default behavior. So, --overwrite flag is for you.

Please let us know if you disagree and feel free to reopen the issue.