Create simple imports with the Extract, Transform, Load pattern.
composer require camillebaronnet/php-etl
This example extract some Github's repositories, apply some transformations
<?php
use Camillebaronnet\ETL\Etl;
use Camillebaronnet\ETL\Extractor\Http;
use Camillebaronnet\ETL\Loader\DebugLoader;
use Camillebaronnet\ETL\Transformer\DateTime;
use Camillebaronnet\ETL\Transformer\Decode;
use Camillebaronnet\ETL\Transformer\Flatten;
use Camillebaronnet\ETL\Transformer\Map;
use Camillebaronnet\ETL\Transformer\Sleep;
$etl = (new Etl)
->extract(Http::class, ['url' => 'https://api.github.com/users/camillebaronnet/repos'])
->add(Decode::class)
->add(Sleep::class, ['seconds' => .2])
->add(Flatten::class, ['glue' => '_'])
->add(Map::class, [
'fields' => [
'id',
'name',
'full_name' => 'fullName',
'owner_login' => 'ownerLogin',
'owner_url' => 'ownerUrl',
'url',
'ssh_url' => 'sshUrl',
'created_at' => 'createdAt'
]
])
->add(DateTime::class, [
'fields' => ['createdAt'],
'from' => 'Y-m-d\TH:i:s\Z',
'to' => 'd/m/Y',
])
;
$etl->process(DebugLoader::class);
-
EXTRACT : Extract can output one or more items
-
TRANFORM : A transform step takes the result of the previous step (extractor or transformer) apply an operation and optionally split the input into several subsets of items (example with Decode).
-
LOADER : A loader can by placed at the end of the pipeline or between transformers. Several Loader can be setting up.
Name | Description |
---|---|
HTTP | Simple wrapper for the libCurl |
Name | Description |
---|---|
Decode | Decode JSON, YAML, XML, CSV and more using Symfony's DecoderInterface |
Map | Rename, keep and remove some fields |
Flatten | Flattens a multi-dimensional collection into a single dimension |
Trim | Strip whitespace from the beginning and end of a string |
Sleep | Delay execution |
DateTime | Parse/Convert dates |
Name | Description |
---|---|
Debug | Display items in output |
You can easily create your own custom Extractors, Transformers, Loader or Strategy by implementing the corresponding interface.
Submit yours. Send a pull-request