Encoding to JSON produces null values

[fatih]

Hi,

Encoding any type to JSON produces nil values to create the null value in the final output. Here is an example code to reproduce it (http://play.golang.org/p/nxETBcTjcj):

package main

import (
    "encoding/json"
    "fmt"
    "log"
    "os"

    "github.com/aws/aws-sdk-go/aws"
)

type Image struct {
    Name        *string
    BlockDevice *string
    ImageId     *string `json:",omitempty"`
    ImageType   *string
}

func main() {
    image := &Image{
        Name:        aws.String("example"),
        BlockDevice: aws.String("/dev/sda1"),
    }

    out, err := json.MarshalIndent(&image, "", "  ")
    if err != nil {
        log.Println(err)
        os.Exit(1)
    }

    fmt.Println(string(out))
}

Output:

{
  "Name": "example",
  "BlockDevice": "/dev/sda1",
  "ImageType": null
}

As seen in the example, ImageID is not encoded because it has the omitempty struct tag, however ImageType is just null. Many types have over 20-30 fields and it causes it to create lots of null values.

Is it possible to attach the json:",omitempty" tag to the struct fields?

Thanks

[fatih]

Additional data:

gce: https://godoc.org/google.golang.org/api/compute/v1#ImageList
do: https://godoc.org/github.com/digitalocean/godo#Image

As seen types of other respective API's all have omitempty.

[lsegal]

Are you looking specifically to print output structures to disk using JSON?

[fatih]

Yes, I want to create JSON outputs of certain types, so I can pass them to other services who consume JSON. Another use case is displaying them (think of it as a CLI), like:

aws ec2 describe-images --owners self --output json

However the types are not JSON friendly. I've checked gce and do and they both already supports it. This is btw very common in Go community and JSON is very popular so everyone would benefit from it in the long term.

[lsegal]

@fatih I generally agree with this. We could add omitempty (and I'll keep this open to track progress), but I'm concerned that we will get similar requests like this in the future for any other supported protocol (like XML). Basically, I'm not sure this is a sustainable solution.

I really think that the encoder should ultimately be responsible for how the data is marshalled, not the struct declaration itself. For example, xml.Decoder has a few toggles to control how data is marshalled into types for the general case. I believe similar toggled could probably be added to Go's json.Encoder type (like an "OmitEmpty" flag), and I would recommend opening an issue with go to request this. This would allow the consumer to control the data formatting on their own without requiring every library to opt-in on every single data structure.

[clbanning]

http://play.golang.org/p/MODtdRa6ng

[jasdel]

@clbanning Interesting proposal, but I'd be hesitant to suggest using unsafe, because if the order of the parameters of the awsImage are changed in anyway or new parameters are added it will produced unexpected results by overlaying myImage.

http://play.golang.org/p/X-nWQhnXod // Adding an ID field produces an unexpected result.

[jasdel]

It would be a good feature for the SDK to allow (un)marshalling JSON blobs into the SDK API types. I think the SDK would need to expose a custom JSON marshaller the SDK could use to unmarshal a JSON io.Reader or []byte to a SDK shape

[github-actions]

We have noticed this issue has not recieved attention in 3 years. We will close this issue for now. If you think this is in error, please feel free to comment and reopen the issue.