Skip to content

Commit

Permalink
Prepare for release version v0.5.0.
Browse files Browse the repository at this point in the history
  • Loading branch information
morazow committed Oct 31, 2019
1 parent c87eaac commit c490ca5
Show file tree
Hide file tree
Showing 5 changed files with 59 additions and 32 deletions.
66 changes: 47 additions & 19 deletions CHANGES.md
Original file line number Diff line number Diff line change
@@ -1,108 +1,136 @@
## v0.5.0

> 2019 OCT 31 :jack_o_lantern:
* Adds Apach Kafka consumer UDF to import Avro formatted data from Kafka
clusters. [#40](https://github.com/exasol/cloud-storage-etl-udfs/issues/40)
[#39](https://github.com/exasol/cloud-storage-etl-udfs/pull/39)
[#48](https://github.com/exasol/cloud-storage-etl-udfs/pull/48)

* Adds several new Kafka consumer settings as user provided UDF parameters.
[#41](https://github.com/exasol/cloud-storage-etl-udfs/issues/41)
[#43](https://github.com/exasol/cloud-storage-etl-udfs/pull/43)

* Refactors the UDF user properties handling in order to make more robust and
maintainable. [#46](https://github.com/exasol/cloud-storage-etl-udfs/pull/46)

* Reworks the documentation to incorporate the new Kafka import changes.
[#45](https://github.com/exasol/cloud-storage-etl-udfs/issues/45)
[#47](https://github.com/exasol/cloud-storage-etl-udfs/pull/47)

## v0.4.4

> 2019 OCT 27
* Adds Shared Access Signature (SAS) token authentication when using Azure Blob
Storage [#42](https://github.com/exasol/cloud-storage-etl-udfs/issues/42)
Storage. [#42](https://github.com/exasol/cloud-storage-etl-udfs/issues/42)
[#44](https://github.com/exasol/cloud-storage-etl-udfs/pull/44)

## v0.4.3

> 2019 JUL 01
* Deserialize Orc BYTE format as a Long value
* Deserialize Orc BYTE format as a Long value.
[#33](https://github.com/exasol/cloud-storage-etl-udfs/issues/33)
[#38](https://github.com/exasol/cloud-storage-etl-udfs/pull/38)
* Improve import process

* Improve import process.
[#34](https://github.com/exasol/cloud-storage-etl-udfs/issues/34)
[#38](https://github.com/exasol/cloud-storage-etl-udfs/pull/38)

## v0.4.2

> 2019 MAY 28
* Adds batch size parameter for export that helps exporting large tables
* Adds batch size parameter for export that helps exporting large tables.
[#28](https://github.com/exasol/cloud-storage-etl-udfs/issues/28)
[#32](https://github.com/exasol/cloud-storage-etl-udfs/pull/32)
* Applied some refactoring

* Applied some refactoring.
[#29](https://github.com/exasol/cloud-storage-etl-udfs/pull/29)

## v0.4.1

> 2019 APR 15
* Adds support for Azure Data Lake (Gen1) Storage
* Adds support for Azure Data Lake (Gen1) Storage.
[#22](https://github.com/exasol/cloud-storage-etl-udfs/issues/22)
[#25](https://github.com/exasol/cloud-storage-etl-udfs/pull/25)
* Support ORC formatted data import

* Support ORC formatted data import.
[#23](https://github.com/exasol/cloud-storage-etl-udfs/issues/23)
[#26](https://github.com/exasol/cloud-storage-etl-udfs/pull/26)

## v0.4.0

> 2019 MAR 21
* Add Avro format import support
* Add Avro format import support.
[#18](https://github.com/exasol/cloud-storage-etl-udfs/issues/18)
[#21](https://github.com/exasol/cloud-storage-etl-udfs/pull/21)

## v0.3.1

> 2019 MAR 08
* Adds Google Cloud Storage and Azure Blob Storage export as Parquet format
* Adds Google Cloud Storage and Azure Blob Storage export as Parquet format.
[#17](https://github.com/exasol/cloud-storage-etl-udfs/issues/17)
[#20](https://github.com/exasol/cloud-storage-etl-udfs/pull/20)

## v0.3.0

> 2019 FEB 12
* Adds feature to export Exasol tables into AWS S3 as Parquet format
* Adds feature to export Exasol tables into AWS S3 as Parquet format.
[#14](https://github.com/exasol/cloud-storage-etl-udfs/issues/14)
[#16](https://github.com/exasol/cloud-storage-etl-udfs/pull/16)
* Imports date and timestamp values correctly into Exasol

* Imports date and timestamp values correctly into Exasol.
[#14](https://github.com/exasol/cloud-storage-etl-udfs/issues/14)
[#16](https://github.com/exasol/cloud-storage-etl-udfs/pull/16)

## v0.2.2

> 2018 DEC 13
* Update Hadoop, Exasol Jdbc and Scala versions
* Update Hadoop, Exasol Jdbc and Scala versions.
[#13](https://github.com/exasol/cloud-storage-etl-udfs/pull/13)
* Fixes issue [#12](https://github.com/exasol/cloud-storage-etl-udfs/issues/12)

* Fixes issue.
[#12](https://github.com/exasol/cloud-storage-etl-udfs/issues/12)

## v0.2.1

> 2018 DEC 06
* Add initial Azure Blob Store import support
* Add initial Azure Blob Store import support.
[#10](https://github.com/exasol/cloud-storage-etl-udfs/pull/10)

## v0.2.0

> 2018 DEC 05
* Add initial Google Compute Storage (GCS) import functionality
* Add initial Google Compute Storage (GCS) import functionality.
[#9](https://github.com/exasol/cloud-storage-etl-udfs/pull/9)

## v0.1.1

> 2018 NOV 28
* Added AWS S3 endpoint as a parameter
* Added AWS S3 endpoint as a parameter.
[#8](https://github.com/exasol/cloud-storage-etl-udfs/pull/8)

## v0.1.0

> 2018 NOV 27
* Add Travis based automated build
* Add Travis based automated build.
[#3](https://github.com/exasol/cloud-storage-etl-udfs/pull/3)
* Renamed packages and external files to more generic cloudetl from s3etl

* Renamed packages and external files to more generic cloudetl from s3etl.
[#4](https://github.com/exasol/cloud-storage-etl-udfs/pull/4)
* Added automated Github release when a tag is pushed

* Added automated Github release when a tag is pushed.
[#5](https://github.com/exasol/cloud-storage-etl-udfs/pull/5)

## v0.0.1
Expand Down
2 changes: 1 addition & 1 deletion docs/developer_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ Follow these steps in order to create a new release:
- Push tags to remote, `git push --tags`.

Please make sure that the new version tag follows the [Semantic Versioning
2.0.0](https://semver.org/)
2.0.0](https://semver.org/).

The next Travis CI run will detect the new tag and create a new Github release
and publish the artifacts.
Expand Down
9 changes: 4 additions & 5 deletions docs/kafka/import.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@ clusters.

Before start, you should deploy the latest jar file to the Exasol BucketFS
bucket and create UDF scripts for importing Avro data from Kafka cluster.
Please follow the instructions from [deployment guide](docs/deployment_guide.md)
Please follow the instructions from [deployment guide](../deployment_guide.md)
if you have not done so.

Additionally, you can read the [user guide](docs/user_guide.md) in order to get
Additionally, you can read the [user guide](../user_guide.md) in order to get
familiar with cloud-storage-etl-udfs in general.

In this guide, we will be using `RETAIL` schema and `SALES_POSITIONS` table to
Expand Down Expand Up @@ -78,8 +78,7 @@ The **TABLE_NAME** is the Exasol table name that we are going to import Kafka
topic data.

For more information on Kafka import parameters, please refer to the [Kafka
import properties in the user
guide](docs/user_guide.md#kafka-import-properties).
import properties in the user guide](../user_guide.md#kafka-import-properties).

### Usage

Expand Down Expand Up @@ -120,7 +119,7 @@ and [Kafka client configurations][kafka-secure-clients] for more information.

Additionally, we have provide extra parameters to the UDF in order to enable
secure connection to Kafka cluster. Please check out the [Kafka import
properties in the user guide](docs/user_guide.md#kafka-import-properties) for
properties in the user guide](../user_guide.md#kafka-import-properties) for
secure property descriptions.

### Usage with SSL enabled
Expand Down
4 changes: 2 additions & 2 deletions docs/storage/cloud_storages.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@ formats.
Before start, you should deploy the latest jar file to the Exasol BucketFS
bucket and create UDF scripts for importing or exporing data from cloud storage
filesystems. Please follow the instructions from [deployment
guide](docs/deployment_guide.md) if you have not done so.
guide](../deployment_guide.md) if you have not done so.

Additionally, you can read the [user guide](docs/user_guide.md) in order to get
Additionally, you can read the [user guide](../user_guide.md) in order to get
familiar with cloud-storage-etl-udfs in general.

In this guide, we will be using `RETAIL` schema and `SALES_POSITIONS` table in
Expand Down
10 changes: 5 additions & 5 deletions docs/user_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ started with using the cloud-storage-etl-udfs.
In order to use the cloud-storage-etl-udfs, you should deploy the jar file and
create UDF scripts.

Please follow the steps from [deployment guide](docs/deployment_guide.md).
Please follow the steps from [deployment guide](./deployment_guide.md).

## Getting started

Expand Down Expand Up @@ -145,15 +145,15 @@ importing or exporting data.
* ``BUCKET_PATH`` - It specifies a path to the cloud storage filesystem.
Additionally, it should start with storage specific schema, such as `s3a`. You
can check out the currently [supported list of cloud storage
systems](docs/storage/cloud_storages.md).
systems](storage/cloud_storages.md).

* ``DATA_FORMAT`` - It defines the data file format in the user provided path.
Currently, we support importing data from **Avro**, **Orc** and **Parquet**
file formats and exporting to only **Parquet** format.

* Additional storage related properties that enable accessing storage
filesystems. Please refer to the [supported cloud storage
systems](docs/storage/cloud_storages.md) documentation for more information.
systems](storage/cloud_storages.md) documentation for more information.

#### Optional Properties

Expand All @@ -166,7 +166,7 @@ These are optional parameters that usually have default values.
For example, multiply with a number when importing `PARALLELISM = 'nproc()*4'`
or append additional numbers when exporting `PARALLELISM = 'iproc(),
floor(random()*4)'`. Please check out the [supported cloud storage
systems](docs/storage/cloud_storages.md) for more examples.
systems](storage/cloud_storages.md) for more examples.

* ``PARQUET_COMPRESSION_CODEC`` - This property is only used in export SQL
statement. It defines the compression codec to use when exporting data into
Expand Down Expand Up @@ -208,7 +208,7 @@ Kafka topic partitions. That is, when importing data from Kafka topic, we will
be importing from each topic partition in parallel. Therefore, it is important
to configure Kafka topics with several partitions.

Please check out the [Kafka import examples](docs/kafka/import.md) for more
Please check out the [Kafka import examples](kafka/import.md) for more
information.

#### Optional Properties
Expand Down

0 comments on commit c490ca5

Please sign in to comment.