mc tag command #3117

BigUstad · 2020-03-16T02:22:16Z

No description provided.

cmd/tag-remove.go

kannappanr · 2020-03-16T18:45:17Z

cmd/tag-set.go

+		key := splitStr(tag, ":", 2)[0]
+		val := splitStr(tag, ":", 2)[1]
+		if key != "" && val != "" || key != "" && val == "" {
+			ilmTagKVMap[key] = val


Why is this variable called ilm.....

kannappanr · 2020-03-16T18:46:00Z

cmd/tag-set.go

+		}
+		key := splitStr(tag, ":", 2)[0]
+		val := splitStr(tag, ":", 2)[1]
+		if key != "" && val != "" || key != "" && val == "" {


Please change this logic. Not sure what you are trying to do here

kannappanr · 2020-03-16T18:49:01Z

cmd/tag-set.go

+			return nil, errors.New("Tag argument(#" + strconv.Itoa(tagIdx+1) + ") `" + tag + "` not in `key:value` format")
+		}
+		key := splitStr(tag, ":", 2)[0]
+		val := splitStr(tag, ":", 2)[1]


I also think we use a different format for --attr. If ; is a valid character, should we look at & as the field separator?

There is no need for a field separator. Arguments will be entered like mc tag set --tags "key1:value1" --tags "key2:value2" --tags "key3:value3"

kannappanr · 2020-03-16T18:49:13Z

cmd/tag-set.go

+	ilmTagKVMap := make(map[string]string)
+	tagValues := ctx.StringSlice(strings.ToLower("tags"))
+	for tagIdx, tag := range tagValues {
+		if !strings.Contains(tag, ":") {


I think we can remove this check

kannappanr · 2020-03-16T19:06:16Z

cmd/tag-show.go

+}
+
+func getTagSpacePad(padLen int) int {
+	if padLen%4 == 0 {


Can this be done with a switch command? The values are when modulo division is done, it is either 0,1,2or3 Using switch is better to read.

kannappanr · 2020-03-17T03:09:19Z

cmd/client-s3.go

@@ -42,6 +43,7 @@ import (
 	"github.com/minio/minio-go/v6/pkg/encrypt"
 	"github.com/minio/minio-go/v6/pkg/policy"
 	"github.com/minio/minio-go/v6/pkg/s3utils"
+	tagging "github.com/minio/minio/pkg/bucket/object/tagging"


This will probably be tagging by default

kannappanr · 2020-03-17T03:10:01Z

cmd/client-s3.go

+func (c *s3Client) GetObjectTag() (tagging.Tagging, *probe.Error) {
+	var err error
+	bucketName, objectName := c.url2BucketAndObject()
+	if bucketName == "" && objectName == "" {


it has to be ||

kannappanr · 2020-03-17T04:41:07Z

cmd/tag-main.go

+
+var tagCmd = cli.Command{
+	Name:   "tag",
+	Usage:  "configure tags, show tags for objects",


Suggested change

Usage: "configure tags, show tags for objects",

Usage: "set/get/delete tags for objects",

kannappanr · 2020-03-17T04:44:34Z

cmd/client-s3.go

+func (c *s3Client) GetObjectTag() (tagging.Tagging, *probe.Error) {
+	var err error
+	bucketName, objectName := c.url2BucketAndObject()
+	if bucketName == "" && objectName == "" {


Suggested change

if bucketName == "" && objectName == "" {

if bucketName == "" || objectName == "" {

kannappanr · 2020-03-17T04:45:03Z

cmd/client-s3.go

+	var err error
+	bucketName, objectName := c.url2BucketAndObject()
+	if bucketName == "" && objectName == "" {
+		err = errors.New("Bucket name, Object name cannot be empty")


Suggested change

err = errors.New("Bucket name, Object name cannot be empty")

err = errors.New("Bucket name or object name cannot be empty")

kannappanr · 2020-03-17T05:06:30Z

cmd/tag-remove.go

+  {{range .VisibleFlags}}{{.}}
+  {{end}}
+DESCRIPTION:
+   Delete object tags previously assigned (if any) to target.


Suggested change

Delete object tags previously assigned (if any) to target.

Delete object tags assigned (if any) to an object.

kannappanr · 2020-03-17T05:07:32Z

cmd/tag-remove.go

+
+var tagRemoveCmd = cli.Command{
+	Name:   "remove",
+	Usage:  "remove tags assigned to the object",


Suggested change

Usage: "remove tags assigned to the object",

Usage: "remove tags assigned to an object",

kannappanr · 2020-03-17T05:13:21Z

cmd/tag-set.go

+var tagSetFlags = []cli.Flag{
+	cli.StringFlag{
+		Name:  "tags",
+		Usage: "format '<key1>=<value1>&<key2>=<value2>'; key1=value1 is a key value pair, different key value pairs separated by '&'",


Suggested change

Usage: "format '<key1>=<value1>&<key2>=<value2>'; key1=value1 is a key value pair, different key value pairs separated by '&'",

Usage: "format '<key1>=<value1>&<key2>=<value2>'; where <key1>=<value1> is a key value pair, different key value pairs separated by '&'",

kannappanr · 2020-03-17T05:14:19Z

cmd/tag-show.go

+
+var tagShowCmd = cli.Command{
+	Name:   "show",
+	Usage:  "show tags for objects",


Suggested change

Usage: "show tags for objects",

Usage: "Display tags for an object",

kannappanr · 2020-03-17T05:17:05Z

cmd/tag-show.go

+func getTagObj(urlStr string) (tagging.Tagging, error) {
+	clnt, pErr := newClient(urlStr)
+	if pErr != nil {
+		console.Errorln(pErr.ToGoError().Error)


Why is this error required? If you already have FatalIf in the next line

kannappanr · 2020-03-17T05:17:19Z

cmd/tag-show.go

+func getTagObj(urlStr string) (tagging.Tagging, error) {
+	clnt, pErr := newClient(urlStr)
+	if pErr != nil {
+		console.Errorln(pErr.ToGoError().Error)


Suggested change

console.Errorln(pErr.ToGoError().Error)

kannappanr · 2020-03-17T05:18:23Z

cmd/tag-show.go

+	}
+}
+
+func getTagSpacePad(padLen int) int {


Suggested change

func getTagSpacePad(padLen int) int {

func showTagPaddingSpace(padLen int) int {

kannappanr · 2020-03-17T05:18:53Z

cmd/tag-show.go

+	console.Println(console.Colorize(tagMainHeader, objectName))
+}
+
+func getTagObj(urlStr string) (tagging.Tagging, error) {


Suggested change

func getTagObj(urlStr string) (tagging.Tagging, error) {

func getObjTagging(urlStr string) (tagging.Tagging, error) {

kannappanr · 2020-03-17T05:19:51Z

cmd/tag-show.go

+		console.Errorln(pErr.ToGoError().Error)
+		fatalIf(probe.NewError(errors.New("Unable to show tags")), "Unable to obtain client from provided url "+urlStr)
+	}
+	s3c, ok := clnt.(*s3Client)


you shouldn't do this.

is this called only in the case of object storage

kannappanr · 2020-03-17T06:00:39Z

cmd/tag-show.go

+	"github.com/minio/cli"
+	json "github.com/minio/mc/pkg/colorjson"
+	"github.com/minio/mc/pkg/probe"
+	tagging "github.com/minio/minio/pkg/bucket/object/tagging"


Suggested change

tagging "github.com/minio/minio/pkg/bucket/object/tagging"

"github.com/minio/minio/pkg/bucket/object/tagging"

kannappanr · 2020-03-19T12:24:47Z

add command will give a different meaning here, are we going to read existing tags and then append, and if there is already an existing tag with the same key, do we replace it? I think set makes sense here if we do not have that functionality yet.

kannappanr · 2020-03-19T12:30:40Z

$ mc tag list s3/bkt/bkt 
Tags not added or not available.

This should probably be something like Tags not set for ....

kannappanr · 2020-03-19T12:32:34Z

$ mc tag remove s3/bkt/obj 
mc: <ERROR> Failed to remove tags: 204 No Content.

Http Status should not be displayed here. In this case the object did not have tags set

kannappanr · 2020-03-19T12:34:51Z

Even if tags are set

$ mc tag remove s3/kannappan300/13
mc: <ERROR> Failed to remove tags: 204 No Content.

kannappanr · 2020-03-19T12:44:06Z

@BigUstad You need to use the latest minio-go master in go.mod

kannappanr · 2020-03-19T12:47:36Z

$ mc tag add 
Name:
  mc tag add - add tags for an object

USAGE:
  mc tag add [COMMAND FLAGS] TARGET

FLAGS:
  --tags value                  format '<key1>=<value1>&<key2>=<value2>';  <key1>=<value1> is a key value pair, different key value pairs are separated by '&'
  --config-dir value, -C value  path to configuration folder (default: "/home/kannappan/.mc")
  --quiet, -q                   disable progress bar display
  --no-color                    disable color theme
  --json                        enable JSON formatted output
  --debug                       enable debug output
  --insecure                    disable SSL certificate verification
  --help, -h                    show help
  
DESCRIPTION:
   Assign object tags (key,value) to target.

Description can be something similar to Assign tags to an object

  --tags value                  format '<key1>=<value1>&<key2>=<value2>';  where <key1>=<value1> is a key value pair, different key value pairs are separated by '&'

kannappanr · 2020-03-19T12:49:12Z

Adding a tags flag where the command is pu-object-tagging seems redundant. You should remove the tags flag

kannappanr · 2020-03-19T21:26:38Z

$ mc tag list s3/bkt/bkt 
Tags not added or not available.

This should probably be something like Tags not set for ....

BigUstad · 2020-03-19T21:34:28Z

mc tag list s3/bkt/bkt
Yes that is the change I made as per your previous comment.

mc/cmd/tag-list.go

Line 103 in 27a39c6

t.Status = "Tags not set or not available."

cmd/tag-set.go

kannappanr · 2020-03-20T05:28:47Z

cmd/tag-remove.go

+	pErr = clnt.DeleteObjectTagging()
+	fatalIf(pErr.Trace(objectURL), "Failed to remove tags")
+	tagObj, err := getObjTagging(objectURL)
+	var tMsg tagRemoveMessage


Suggested change

var tMsg tagRemoveMessage

var msg tagRemoveMessage

cmd/tag-remove.go

kannappanr · 2020-03-20T07:54:47Z

cmd/tag-remove.go

+type tagRemoveMessage struct {
+	Status string `json:"status"`
+	Name   string `json:"name"`
+	Err    error  `json:"err,omitempty"`


Suggested change

Err error `json:"err,omitempty"`

Error error `json:"error,omitempty"`

kannappanr · 2020-03-20T08:47:41Z

cmd/tag-set.go

+	return string(msgBytes)
+}
+
+func parseTagSetMessage(tags string, urlStr string, err error) tagSetMessage {


Suggested change

func parseTagSetMessage(tags string, urlStr string, err error) tagSetMessage {

func parseTagSetMessage(urlStr string, err error) tagSetMessage {

The name should not be parse... you are not parsing anything, you are setting up values in a structure

kannappanr · 2020-03-20T16:00:57Z

Please resolve the go.mod and go.sum conflicts

* Fix zh_CN doc about policy errors (minio#2927) * Fix Windows config path (minio#2898) Remove '.exe' (case insensitive) from executable name for consistent config folder. Potentially breaking: If people (for whatever reason) used 'mc.exe', this will potentially break their setup. * Fix removing empty prefixes with `mc rm ` command (minio#2912) Fixes: 2909 * Add --type flag to `mc admin console` (minio#2914) This allows filtering error logs by type * Update mc to use secure-io package (minio#2931) * Refactor mc cp and make session optional. (minio#2910) Now `mc cp --recursive --continue` creates copy session on failure or resumes if previous failed session is found. Fixes minio#2893 * continue flag: Fix inconsistent punctuation in usage text (minio#2933) * fix mc cp termination on signals (minio#2934) * Adapt new changes in StorageInfo (minio#2889) * Fix arrow signs to use more common U+2191 & U+2193 (minio#2942) * Avoid index out of range errors (minio#2943) Fixes minio#2941 * Fix typo in error message (minio#2945) Change the error message from `Cannot remove new user` to `Cannot remove <User>`, when a non-existent user is removed. * Add options to preserve filesystem attributes (minio#2938) * mc admin config set/get/del/history/help (minio#2835) Related to minio/minio#8392 * print disk usage as a plain integer when formatting as json (minio#2946) * Fix the storage prints for xl/fs disks in `admin info server` (minio#2947) * Fix config history to list in git style format (minio#2951) ``` ~ mc admin config history list -n 2 myminio RestoreId: df0ebb1e-69b0-4043-b9dd-ab54508f2897 Date: Mon, 04 Nov 2019 17:27:27 GMT region name="us-east-1" state="on" region name="us-east-1" state="on" region name="us-east-1" state="on" region name="us-east-1" state="on" RestoreId: ecc6873a-0ed3-41f9-b03e-a2a1bab48b5f Date: Mon, 04 Nov 2019 17:28:23 GMT region name=us-east-1 state=off ``` * add lock command (minio#2880) `lock` command enables to set, get and clear object lock configuration of given bucket. * update to latest minio (minio#2956) * Fix encrypt-key flag parsing (minio#2955) * update to latest minio (minio#2957) * Rectify usage section of policy help (minio#2959) * help command is now part of `mc admin config get/set` (minio#2961) * mirror object lock configuration to dest bucket (minio#2964) if bucket does not exist on the target side. * Add default values in 'get' (minio#2962) * Add mirror command to compare metadata and copy (minio#2965) mirror now uses the newly introduced ListObjectsV2WithMetadata API in minio-go * Handle PutObjectRetention notification events correctly (minio#2966) * Retention command to apply retention to objects with a Prefix (minio#2963) * Fix totalSize and totalCounts calculation (minio#2969) * Add import/export command (minio#2974) * Add support for auto-complete in 'mc admin config' commands (minio#2975) * Remove duplicated API method(s) for object retention (minio#2973) * Fix progress bar update regression (minio#2976) Regression introduced in cf6defe * Update export command to print disabled sub-sys as comments (minio#2978) * Add finalized command line options (minio#2979) * Avoid re-parsing config history elements (minio#2981) * change target to name in config help (minio#2983) * fix: Source content can be nil when updating progress bar (minio#2984) * Continue mirroring inspite of error events (minio#2986) * watch: Add site wide level support (minio#2987) * Continue listing upon error during diff and mirror (minio#2988) * Support for multi master sites (minio#2991) * Retry mirror forever on watch (minio#2992) * Fix multi-master setup help text (minio#2993) * Do full scan diff+mirror in multi-site mirror mode (minio#2994) * update minio-go to master (minio#2995) * Avoid mirror fatalIf in multi-master (minio#2998) * Mirror with fallback list in multimaster mode, copy missing files (minio#2999) * Fix mirroring of PutObjectRetention event (minio#3004) * Multi-master only run single startMirror at a time (minio#3003) Also randomize the sleep appropriately based on Unix() time source. * mc sql: Use compression parameter (minio#2996) The compression parameter is not actually used, but always guessed from the file extension. Only guess the compression type if it is not explicitly set. Uploading a file called `out.csv.geezipd`: Before: ``` λ mc sql -e="SELECT * from s3object LIMIT 1" --compression=GZIP --csv-input "rd=\n,fh=USE,fd=;" local/test/out.csv.geezipd �""��k↔��Xs��W.v� ��)��{2#��,r��:�|D▲�.��f♫��→�i�p�2l�♫��ʭ��Q��v▲�5ر�c ``` (content actually not ungzipped) After: ``` λ mc sql -e="SELECT * from s3object LIMIT 1" --compression=GZIP --csv-input "rd=\n,fh=USE,fd=;" local/test/out.csv.geezipd "281,1285,159625,159627,2637,20000827.0117590018,1239029,1663,-6.7535419,1533,0.53597438,8" ``` We leave validation of the compression value to the server. * Replace Timestamp separator (minio#3002) Replace time stamp separator to use `-` * New admin info (minio#2982) * Stops showing usage info if Bucket#=0 (minio#3007) * Extraneous line in 'mc admin info' output (minio#3010) * cp: Only use session when -c is passed (minio#2989) * Upgrade to new Listen bucket API (minio#3008) * fix: ignore APINotImplemented errors from filesystem (minio#3018) * Update to latest minio/minio (minio#3020) * Mirror: Generate STag and ETag on destination only in Multimaster mode (minio#3021) * Update mc dependencies to recent dep change on MinIO (minio#3026) * Use colorjson to indent json properly in cp (minio#3030) Additionally also do not initialize progress bar in json or quiet mode in mirror. Fixes minio#3029 Fixes minio#3017 * fix default flag behaviour in du command (minio#3031) - `mc du` without args should print help like other `mc` commands - `mc du` should behave like `du -h --max-depth=1` as default - optional `--recursive` if they want capacity for each folder. fixes minio#2871 * Add lock and retention docs (minio#3034) * Only rename config directories (minio#3032) If the user placed an `mc.exe` in the user directory and wasn't used it, it would be renamed. * fix: crash in cp when cpURLs return error (minio#3035) * fix: du to ignore too many recursive symlinks (minio#3033) * fix: update docs for --attr command (minio#3040) Also canonicalize metadata headers automatically before, minio-go consumes it. * retention: Error out early when url argument does not point to S3 server (minio#3041) * fix: update minio-go to support snowball data-transfer (minio#3042) * cp: add missing } in help template (minio#3047) * Update pkg/madmin to support multiple profilers (minio#3048) * admin/console: Fix a crash in console string generation (minio#3049) * fix: best effort save permissions always (minio#3050) * upgrade to latest server to remove replace tags (minio#3051) * fix: update admin docs for latest changes (minio#3054) * fix: handle symlinks by reading/following as needed (minio#3053) Avoid using filepath.EvalSymlinks and let the caller fail appropriately for symlinks. Fixes minio#3011 * check for bucket exists in 'admin heal' (minio#3059) * Ensure 64bit alignment on 32bit builds for atomics (minio#3065) Keep elements that are used with atomic.* functions as first element of struct because it guarantees 64bit alignment on 32 bit machines. atomic.* functions crash if operand is not aligned at 64bit. See golang/go#599 * add the `mc admin kms` subcommands (minio#2882) This commit adds the first set of KMS admin commands. For new there is only the `mc admin kms key status <endpoint> [key-ID]` command that can be used to fetch status information for a KMS master key. Listing and rotation commands are planed. * Fix go.sum file missing entries (minio#3071) * add debug support for 'config host add' (minio#3076) * Fix crash in mc admin console (minio#3079) Fixes : minio#3078 * use latest minio-go version (minio#3080) * allow mirror to copy zero byte file (minio#3084) this PR also cleans up code and avoids extra calls for source target * go mod tidy (minio#3081) * honor prefix for object and directory names in 'rm -r' (minio#3012) * fix go mod tidy again * added pkg/disk/stat_freebsd.go (minio#3090) * fix handling of directories in Stat (minio#3092) Fixes minio#3089 Fixes minio#3091 * retention - cleanup unused code (minio#3094) * Use listObjectsV1 for GCS S3 implementation (minio#3101) Fixes minio#3073 * Support arm32, 386 compilation: (minio#3103) * deprecate 'mc session' documentation (minio#3104) * Add support of threads & goroutines profiling type (minio#3108) * fix go mod tidy * Update minio-go to bring some fixes (minio#3106) * resolve build issues on illumos (minio#3098) * Update docs for mc admin update (minio#3110) * fix: Make sure Ctrl-C cancels `mc cp` right away (minio#3113) * mc cp -continue: Fix escape/colorization of JSON (minio#3116) * info: Fix showing offline nodes in the pretty output (minio#3112) Also do some simplification in the code. * Secret key prompt (minio#3083) * heal: Show scheduled next healing (minio#3111) mc admin heal alias command will show the next healing time. Also, show 'Never completed' when the server returns zero last healing activity * mc cp: Fix prefix handling in TypeC (minio#3115) for copy from filesystem to objectstorage. Avoid prefixing dirname to file on target side when doing a `mc cp /tmp/dir myminio/bucket --r` to make `mc mirror` and `mc cp` behavior consistent * info: Add missing new lines in the output (minio#3119) * fix for incorrect mtime handling (minio#3121) * mc retention: add --recursive and --bypass flags. (minio#3100) --bypass flag allows overriding governance if user has override permissions * cleanup legacy metadata handling (minio#3118) * Add global context and adapt context change in admin API (minio#3125) This commit adds a new global context, monitors os signals and cancels the global context in such cases to exit on going process gracefully. It will also update minio-go and adapts the code with the new context changes in the admin API. * Add tag command (minio#3117) * cleanup tag command (minio#3126) * Add legalhold command (minio#3095) * mc watch windows URL/absolute path event path (minio#3133) * Onboard Diagnostics (minio#3114) Also colorize JSON output in all admin commands * Make BuildS3Config public (minio#3132) * add build trimpaths to remove gopath (minio#3137) * add --disable-multipart flag in cp (minio#3135) * set S3Client struct as public * set s3client struct as public to be used by mcs Co-authored-by: yiranzai <wuqingdzx@gmail.com> Co-authored-by: Klaus Post <klauspost@gmail.com> Co-authored-by: poornas <poornas@users.noreply.github.com> Co-authored-by: Harshavardhana <harsha@minio.io> Co-authored-by: Bala FA <bala.gluster@gmail.com> Co-authored-by: Ashish Kumar Sinha <ashi.sinha.87@gmail.com> Co-authored-by: Praveen raj Mani <praveen@minio.io> Co-authored-by: Nitish Tiwari <nitish@minio.io> Co-authored-by: kannappanr <30541348+kannappanr@users.noreply.github.com> Co-authored-by: Wesley Collin Wright <deinlebenandern@gmail.com> Co-authored-by: Krishna Srinivas <634494+krishnasrinivas@users.noreply.github.com> Co-authored-by: Anis Elleuch <vadmeste@users.noreply.github.com> Co-authored-by: BigUstad <priyank.raj@gatech.edu> Co-authored-by: ebozduman <ersan.bozduman@gmail.com> Co-authored-by: Maël Valais <mael.valais@gmail.com> Co-authored-by: Reinhard Koehn <rkoehn@users.noreply.github.com> Co-authored-by: Andreas Auernhammer <aead@mail.de> Co-authored-by: Dmitry Wagin <dmitry.wagin@ya.ru> Co-authored-by: Kody A Kantor <kody.kantor@gmail.com> Co-authored-by: Jeffrey Clark <h0tw1r3@users.noreply.github.com> Co-authored-by: Sidhartha Mani <wlan0@users.noreply.github.com> Co-authored-by: Daniel Valdivia <hola@danielvaldivia.com>

mc tag commond

577f712

BigUstad force-pushed the mc-object-tagging branch 2 times, most recently from e9aed1c to 765d174 Compare March 16, 2020 05:54

kannappanr reviewed Mar 16, 2020

View reviewed changes

cmd/tag-remove.go Outdated Show resolved Hide resolved

kannappanr reviewed Mar 16, 2020

View reviewed changes

BigUstad force-pushed the mc-object-tagging branch 4 times, most recently from 5dc2b0a to 55e44ae Compare March 17, 2020 02:42

kannappanr reviewed Mar 17, 2020

View reviewed changes

BigUstad force-pushed the mc-object-tagging branch 2 times, most recently from 403634f to 9813e57 Compare March 17, 2020 07:19

BigUstad force-pushed the mc-object-tagging branch from 75cc2ce to c1207f7 Compare March 18, 2020 09:02

BigUstad force-pushed the mc-object-tagging branch from 27a39c6 to b1297e5 Compare March 19, 2020 23:51