Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refs platform/#2978: implement init filesystem and seed archives feature #14

Merged
merged 1 commit into from
Aug 7, 2024

Conversation

Monska85
Copy link
Contributor

@Monska85 Monska85 commented Aug 7, 2024

No description provided.

@Monska85 Monska85 force-pushed the feat/2978_initfs_and_archives_features branch 3 times, most recently from 224d5fe to 39fd87e Compare August 7, 2024 11:30
README.md Outdated

If the `INITFILESYSTEM_FOLDER` path is not empty, the files in the folder are copied to the `BUCKET_ROOT` destination folder as a startup filesystem for the MinIO server. **The folder must contain a valid MinIO filesystem structure.** **ATTENTION**: when the `INITFILESYSTEM_FOLDER` is used, the startup process takes care of checking that the `BUCKET_NAME` is configured accordingly with the final MinIO server configuration, performing the filesystem check (the `BUCKET_NAME` folder is present in the `BUCKET_ROOT` folder) and the MinIO client check (the `mc ls ${MC_ALIAS}/${BUCKET_NAME}` command does not return an error).

If this initialization method is used, **no other initialization actions are performed**. The final bucket content is only the data present in the `INITFILESYSTEM_FOLDER` folder.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am wondering if we should support both methods, seed files + bucket files, they shouldn't collide and can be processed in a chain.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes we can. They can collide in the end, and the seed files (archives or files) will overwrite any existing objects. If the versioning is enabled, the new file in the chain will superseed the previous one.

I will implement this logic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implemented. This is the new seed logic.

If the bucket exists at the startup, no seed process will be executed. In other words, if the /data folder is a valid MinIO filesystem with an already created bucket, nothing happens. To test this scenario, you can mount a local data folder in /data container mount point and exec make start twice. At the first run the container will seed the mounted /data folder, at the second run the container will skip the initialization process.

README.md Outdated

### Process the seed files

If the `INITFILES_FOLDER` path is not empty, the files in the folder are uploaded in the bucket defined by the `BUCKET_NAME` environment variable using the MinIO client (`mc cp --recursive`). As for the archives, if the `INITFILES_FOLDER` contains files with the same name of one or more files already present in the bucket (processed by the `INITARCHIVES_FOLDER`), the files in the `INITFILES_FOLDER` will overwrite the files in the bucket. If the bucket has a versioning enabled, the previous versions of the files are preserved.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would chance FOLDER with DIR

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@Monska85 Monska85 force-pushed the feat/2978_initfs_and_archives_features branch 4 times, most recently from 6fcfce5 to 11cdf51 Compare August 7, 2024 14:48
@Monska85 Monska85 force-pushed the feat/2978_initfs_and_archives_features branch from 11cdf51 to 341b903 Compare August 7, 2024 14:58
@paolomainardi paolomainardi merged commit 060fd02 into main Aug 7, 2024
2 checks passed
@paolomainardi paolomainardi deleted the feat/2978_initfs_and_archives_features branch August 7, 2024 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants