-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Faker source gains purchases and products #13248
Conversation
Codecov Report
@@ Coverage Diff @@
## master #13248 +/- ##
=========================================
Coverage ? 98.40%
=========================================
Files ? 2
Lines ? 125
Branches ? 0
=========================================
Hits ? 123
Misses ? 2
Partials ? 0 Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome, who needs real data anyway!? 👍
couple of nits and small suggestions so no need for a second review, approving with this one.
airbyte-integrations/connectors/source-faker/source_faker/source.py
Outdated
Show resolved
Hide resolved
f8e839f
to
77d72c0
Compare
/publish connector=connectors/source-faker
|
6776cfe
to
ebbb72f
Compare
/test connector=connectors/source-faker
Build PassedTest summary info:
|
/publish connector=connectors/source-faker
|
…ehq/airbyte into evan/faker-gets-purchases
* Faker source gains purchases and products * ranges start at 1 * fix nits * lint * user_ids start at 1 * v0.1.3 * fix IDs in test output recods * auto-bump connector version * lint Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Part of #13201
The faker command now has 3 streams: Users, Products, and Purchases. The list of Products is a static list of 100 cars. The list of purchases is generated randomly against certain percentages. Each purchase also has 3 timestamps: added_to_cart, purchased, and returned - these are always in the proper chronological order, and happen with various percentages: 70% likely to move from cart -> purchase and 30% likely to return the item if it was purchased.
This dataset was designed to allow for exploration and transformation with dbt and various reporting tools to produce data like "user lifetime value" and "avg rate of return".
There are also now additional configuration options to control the size of each read and stream-slice (page). This will prove helpful for debugging things (like checkpointing) down the road.