Home

Kiba is a lightweight Ruby ETL framework (Extract - Transform - Load) framework. It makes it easy to write reliable, concise, well-tested & maintainable data-processing code in Ruby.

Kiba development & maintenance is funded by LoGeek, a French data-oriented consultancy.

Read for more info:

Kiba

• Core Concepts
• How to define ETL jobs with Kiba
• How to run ETL jobs with Kiba
• Implementing ETL sources
• Implementing ETL transforms
• Implementing ETL destinations
• Implementing pre and post-processors
• Implementation Guidelines
• How to extend the Kiba DSL
• Can Kiba handle multiple sources and destinations?
• Ruby 2.7+ keyword arguments compatibility note

Kiba Common

Kiba Common is an OSS companion gem to Kiba, bringing commonly used features to Kiba:

• Enumerable Source (turn any enumerable into a stream of rows)
• Enumerable Exploder (explodes an enumerable row into a sequence of output rows)
• Source Transform Adapter (dynamical sources using input rows as instantiation parameters)
• CSV Source
• CSV Destination
• Lambda Destination
• How to add logging to Kiba?
• How to pretty-color-print rows in Kiba?

Kiba Pro

Kiba Pro adds powerful vendor-backed extensions for Kiba ETL. Purchasing a Kiba Pro subscription (via https://www.logeek.fr) also supports the continued maintenance & development of Kiba Open-Source.

Available components

• SQL Source
• SQL Upsert Destination
• SQL Bulk Insert/Upsert Destination
• File Lock (avoids concurrent ETL jobs)

Purchasing, installation, misc

• Commercial FAQ (license & pricing)
• Purchase
• Changelog
• How to install Kiba Pro
• Kiba Blueprints

Upcoming components

Roadmap (in no specific order):

• SQL Bulk (batched) Keys Lookup
• Parallelized (multithreaded) Transform
• Amazon S3 Integration (Source & Destination)
• Single-Run Locking (avoid concurrent jobs)