datacontract export --format avro - array issue

[gpthome]

I'm trying to generate AVRO output which includes ARRAYS. I tried a couple different versions of ...

  simple_array:
    type: array
    items: {'type': 'string'}
    default: []
    required: false

Instead of items: {'type': 'string'}, I tried items: string and received "DataContractException: Run operation failed: [lint] Check that data contract YAML is valid - None - failed - data.models.catalog.fields.simple_array.items must be object - datacontract". Not sure if I'm defining it correctly.

Current output no matter the 'required' value...

    {
      "name": "simple_array",
      "type": "array"
    },

Expected output with required: true...

        {
            "name": "simple_array",
            "type": {
                "type": "array",
                "items": "string",
                "default": []
            }
        },

Expected output with required: false...

        {
            "name": "simple_array",
            "type": [
                "null",
                {
                    "type": "array",
                    "items": "string"
                }
            ],
            "default": null
        },

I believe all data types are missing the "null" union in the export when required: false,.

[jochenchrist]

Confirming, this is a bug.

[jochenchrist]

The correct way to model arrays is (as you already figured out):

      tags:
        type: array
        items:
          type: string

The expected output is:

    {
      "name": "tags",
      "type": {
        "type": "array",
        "items": "string"
      }
    },

which I fixed for next release v0.10.8.

The required: false feature for avro export is another topic that has not been addressed, yet. If you need this feature, feel free to open an issue or PR.

[gpthome]

Perfect. Yes, I will create a separate issue for the 'required' request.

Thank you!