Skip to content

Commit

Permalink
datacontract export --format avro - array
Browse files Browse the repository at this point in the history
resovles #243
  • Loading branch information
jochenchrist committed Jun 7, 2024
1 parent bc8bec3 commit 2f24263
Show file tree
Hide file tree
Showing 5 changed files with 16 additions and 3 deletions.
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Introduced optional dependencies/extras for significantly faster installation times. (#213)
- Added delta-lake as an additional optional dependency

### Fixed
- `datacontract export --format avro` fixed array structure (#243)

## [0.10.7] - 2024-05-31

### Added
Expand Down
3 changes: 1 addition & 2 deletions datacontract/export/avro_converter.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,7 @@ def to_avro_type(field: Field, field_name: str) -> str | dict:
elif field.type in ["binary"]:
return "bytes"
elif field.type in ["array"]:
# TODO support array structs
return "array"
return {"type": "array", "items": to_avro_type(field.items, field_name)}
elif field.type in ["null"]:
return "null"
else:
Expand Down
4 changes: 4 additions & 0 deletions tests/fixtures/avro/export/datacontract.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,10 @@ models:
type: string
orderunits:
type: double
tags:
type: array
items:
type: string
address:
type: object
fields:
Expand Down
7 changes: 7 additions & 0 deletions tests/fixtures/avro/export/orders_with_datefields.avsc
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,13 @@
"name": "orderunits",
"type": "double"
},
{
"name": "tags",
"type": {
"type": "array",
"items": "string"
}
},
{
"name": "address",
"type": {
Expand Down
2 changes: 1 addition & 1 deletion tests/test_export_avro.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ def test_to_avro_schema():
assert json.loads(result) == json.loads(expected_avro_schema)


def test_to_avro_schema_with_logicalTypes():
def test_to_avro_schema_with_logical_types():
data_contract = DataContractSpecification.from_file("fixtures/avro/export/datacontract_logicalType.yaml")
with open("fixtures/avro/export/datacontract_logicalType.avsc") as file:
expected_avro_schema = file.read()
Expand Down

0 comments on commit 2f24263

Please sign in to comment.