🎉 Source Shopify: implement `BalanceTransactions` stream #10204

harshithmullapudi · 2022-02-09T06:57:41Z

What

harshithmullapudi · 2022-02-09T06:58:20Z

/test connector=connectors/source-shopify

🕑 connectors/source-shopify https://github.com/airbytehq/airbyte/actions/runs/1816604909
❌ connectors/source-shopify https://github.com/airbytehq/airbyte/actions/runs/1816604909
🐛 https://gradle.com/s/tbpzk2db4xnai
Python short test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs0] - docker.errors.Contai...
FAILED test_full_refresh.py::TestFullRefresh::test_sequential_reads[inputs0]
FAILED test_incremental.py::TestIncremental::test_two_sequential_reads[inputs0]
FAILED test_incremental.py::TestIncremental::test_state_with_abnormally_large_values[inputs0]
================== 4 failed, 34 passed in 1570.73s (0:26:10) ===================

codecov · 2022-02-09T07:00:39Z

Codecov Report

❗ No coverage uploaded for pull request base (master@1f98286). Click here to learn what that means.
The diff coverage is n/a.

❗ Current head 3c957ac differs from pull request most recent head 8373e94. Consider uploading reports for the commit 8373e94 to get more accurate results

@@            Coverage Diff            @@
##             master   #10204   +/-   ##
=========================================
  Coverage          ?   72.04%           
=========================================
  Files             ?        5           
  Lines             ?      415           
  Branches          ?        0           
=========================================
  Hits              ?      299           
  Misses            ?      116           
  Partials          ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1f98286...8373e94. Read the comment docs.

harshithmullapudi · 2022-03-10T09:50:57Z

/test connector=connectors/source-shopify

🕑 connectors/source-shopify https://github.com/airbytehq/airbyte/actions/runs/1962357657
❌ connectors/source-shopify https://github.com/airbytehq/airbyte/actions/runs/1962357657
🐛 https://gradle.com/s/vvvnsrcsmvinm

harshithmullapudi · 2022-03-21T05:08:13Z

/test connector=connectors/source-shopify

🕑 connectors/source-shopify https://github.com/airbytehq/airbyte/actions/runs/2014261075
❌ connectors/source-shopify https://github.com/airbytehq/airbyte/actions/runs/2014261075
🐛 https://gradle.com/s/y3r67ew43egoe
Python short test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs0] - AssertionError: All ...
================== 1 failed, 37 passed in 2208.29s (0:36:48) ===================

harshithmullapudi · 2022-03-22T07:33:47Z

/test connector=connectors/source-shopify

🕑 connectors/source-shopify https://github.com/airbytehq/airbyte/actions/runs/2020905533
✅ connectors/source-shopify https://github.com/airbytehq/airbyte/actions/runs/2020905533
Python tests coverage:

Name                                                 Stmts   Miss  Cover
------------------------------------------------------------------------
source_acceptance_test/utils/__init__.py                 6      0   100%
source_acceptance_test/tests/__init__.py                 4      0   100%
source_acceptance_test/__init__.py                       2      0   100%
source_acceptance_test/tests/test_full_refresh.py       52      2    96%
source_acceptance_test/utils/asserts.py                 37      2    95%
source_acceptance_test/config.py                        74      6    92%
source_acceptance_test/utils/json_schema_helper.py     105     13    88%
source_acceptance_test/utils/common.py                  70     17    76%
source_acceptance_test/utils/compare.py                 62     23    63%
source_acceptance_test/tests/test_core.py              275    106    61%
source_acceptance_test/base.py                          10      4    60%
source_acceptance_test/utils/connector_runner.py       110     48    56%
source_acceptance_test/tests/test_incremental.py        69     38    45%
------------------------------------------------------------------------
TOTAL                                                  876    259    70%
Name                          Stmts   Miss  Cover
-------------------------------------------------
source_shopify/__init__.py        2      0   100%
source_shopify/transform.py      58      3    95%
source_shopify/utils.py          58      6    90%
source_shopify/auth.py           20      4    80%
source_shopify/source.py        292    112    62%
-------------------------------------------------
TOTAL                           430    125    71%

harshithmullapudi · 2022-03-22T08:35:12Z

@bazarnov @ron-damon I made the changes here to reduce the back and forth. Kindly look at them

ron-damon · 2022-03-22T09:28:45Z

I'd refactored the incremental logic for streams based purely on id (does not support Incremental Refresh based on datetime fields) because it's exactly the same behavior for all "purely id" streams, but it's your call! 😃
LGTM! 🚀

harshithmullapudi · 2022-03-22T10:17:57Z

@ron-damon are there more streams similar to collects and balance transactions if so I think we can go with what you have suggested. wdyt?

bazarnov · 2022-03-22T10:29:36Z

@harshithmullapudi @ron-damon
Please, checkout the latest commit, as my suggestion to cover new id-based streams and cleared a code a bit.

airbyte-integrations/connectors/source-shopify/source_shopify/source.py

bazarnov · 2022-03-22T12:44:04Z

/test connector=connectors/source-shopify

🕑 connectors/source-shopify https://github.com/airbytehq/airbyte/actions/runs/2022396569
✅ connectors/source-shopify https://github.com/airbytehq/airbyte/actions/runs/2022396569
Python tests coverage:

Name                                                 Stmts   Miss  Cover
------------------------------------------------------------------------
source_acceptance_test/utils/__init__.py                 6      0   100%
source_acceptance_test/tests/__init__.py                 4      0   100%
source_acceptance_test/__init__.py                       2      0   100%
source_acceptance_test/tests/test_full_refresh.py       52      2    96%
source_acceptance_test/utils/asserts.py                 37      2    95%
source_acceptance_test/config.py                        74      6    92%
source_acceptance_test/utils/json_schema_helper.py     105     13    88%
source_acceptance_test/utils/common.py                  70     17    76%
source_acceptance_test/utils/compare.py                 62     23    63%
source_acceptance_test/tests/test_core.py              275    106    61%
source_acceptance_test/base.py                          10      4    60%
source_acceptance_test/utils/connector_runner.py       110     48    56%
source_acceptance_test/tests/test_incremental.py        69     38    45%
------------------------------------------------------------------------
TOTAL                                                  876    259    70%
Name                          Stmts   Miss  Cover
-------------------------------------------------
source_shopify/__init__.py        2      0   100%
source_shopify/transform.py      58      3    95%
source_shopify/utils.py          58      6    90%
source_shopify/auth.py           20      4    80%
source_shopify/source.py        281    103    63%
-------------------------------------------------
TOTAL                           419    116    72%

bazarnov · 2022-03-22T13:34:04Z

@ron-damon

I think we should remove the order parameter when using since_id, because it's not allowed to use both. The output of a request with order and since_id parameters

It's should be fine, if the order_field is id, otherwise, the endpoint returns the error. The tests are passed as expected.

bazarnov

We are good to go)

airbyte-integrations/connectors/source-shopify/source_shopify/source.py

…gcloud#539

bazarnov · 2022-03-22T18:09:07Z

/publish connector=connectors/source-shopify

🕑 connectors/source-shopify https://github.com/airbytehq/airbyte/actions/runs/2024127143
✅ connectors/source-shopify https://github.com/airbytehq/airbyte/actions/runs/2024127143

ron-damon and others added 7 commits January 26, 2022 18:35

add stream's logic and schema

28150cd

bump connector's version

c465794

add changelog entry

4699ad8

add test resources

7dc2a25

add stream to SCOPES_MAPPING

3d48edc

set PR

2bb1aab

Merge branch 'master' into harshith/test-pr-9850

5b041f1

github-actions bot added area/connectors Connector related issues area/documentation Improvements or additions to documentation labels Feb 9, 2022

harshithmullapudi temporarily deployed to more-secrets February 9, 2022 06:58 Inactive

octavia-squidington-iii temporarily deployed to more-secrets February 9, 2022 07:00 Inactive

Merge branch 'master' into harshith/test-pr-9850

16a6894

fix: tests

156e54a

harshithmullapudi temporarily deployed to more-secrets March 22, 2022 07:35 Inactive

harshithmullapudi temporarily deployed to more-secrets March 22, 2022 07:36 Inactive

ron-damon mentioned this pull request Mar 22, 2022

🎉 Source Shopify: Add BalanceTransactions stream #9850

Closed

40 tasks

improved the request_params filtering logic

8ae4649

bazarnov temporarily deployed to more-secrets March 22, 2022 10:30 Inactive

airbytehq deleted a comment from harshithmullapudi Mar 22, 2022

bazarnov temporarily deployed to more-secrets March 22, 2022 10:30 Inactive

bazarnov changed the title ~~Publish source shopify: added new stream~~ 🎉 Source Shopify: implement BalanceTransactions stream Mar 22, 2022

bazarnov temporarily deployed to more-secrets March 22, 2022 10:38 Inactive

ron-damon reviewed Mar 22, 2022

View reviewed changes

airbyte-integrations/connectors/source-shopify/source_shopify/source.py Show resolved Hide resolved

bazarnov self-requested a review March 22, 2022 13:37

bazarnov approved these changes Mar 22, 2022

View reviewed changes

ron-damon approved these changes Mar 22, 2022

View reviewed changes

bazarnov added 2 commits March 22, 2022 17:05

Merge remote-tracking branch 'origin/master' into harshith/test-pr-9850

64a93da

edited publish command due to the issue: google-github-actions/setup-…

5c74cdc

…gcloud#539

bazarnov temporarily deployed to more-secrets March 22, 2022 15:10 Inactive

bazarnov temporarily deployed to more-secrets March 22, 2022 15:11 Inactive

fixed @ element

6098d83

bazarnov temporarily deployed to more-secrets March 22, 2022 15:15 Inactive

fixed other actions from using @master to use @v0

73c00e5

bazarnov temporarily deployed to more-secrets March 22, 2022 15:22 Inactive

Merge remote-tracking branch 'origin/master' into harshith/test-pr-9850

3c957ac

bazarnov temporarily deployed to more-secrets March 22, 2022 17:38 Inactive

bazarnov temporarily deployed to more-secrets March 22, 2022 17:43 Inactive

bazarnov temporarily deployed to more-secrets March 22, 2022 18:05 Inactive

new spec

8373e94

bazarnov temporarily deployed to more-secrets March 22, 2022 19:20 Inactive

bazarnov merged commit 9508f61 into master Mar 22, 2022

bazarnov deleted the harshith/test-pr-9850 branch March 22, 2022 19:24

This was referenced Mar 23, 2022

Bump Airbyte version from 0.35.59-alpha to 0.35.60-alpha #11365

Closed

Bump Airbyte version from 0.35.59-alpha to 0.35.60-alpha #11373

Closed

Bump Airbyte version from 0.35.59-alpha to 0.35.60-alpha #11402

Merged

arsenlosenko mentioned this pull request Oct 6, 2022

The requested stream BalanceTransactions was not found in the source (Shopify) #11647

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🎉 Source Shopify: implement `BalanceTransactions` stream #10204

🎉 Source Shopify: implement `BalanceTransactions` stream #10204

harshithmullapudi commented Feb 9, 2022

harshithmullapudi commented Feb 9, 2022 •

edited by github-actions bot

Loading

codecov bot commented Feb 9, 2022 •

edited

Loading

harshithmullapudi commented Mar 10, 2022 •

edited by github-actions bot

Loading

harshithmullapudi commented Mar 21, 2022 •

edited by github-actions bot

Loading

harshithmullapudi commented Mar 22, 2022 •

edited by github-actions bot

Loading

harshithmullapudi commented Mar 22, 2022

ron-damon commented Mar 22, 2022

harshithmullapudi commented Mar 22, 2022

bazarnov commented Mar 22, 2022

bazarnov commented Mar 22, 2022 •

edited by github-actions bot

Loading

bazarnov commented Mar 22, 2022

bazarnov left a comment

bazarnov commented Mar 22, 2022 •

edited by github-actions bot

Loading

🎉 Source Shopify: implement BalanceTransactions stream #10204

🎉 Source Shopify: implement BalanceTransactions stream #10204

Conversation

harshithmullapudi commented Feb 9, 2022

What

harshithmullapudi commented Feb 9, 2022 • edited by github-actions bot Loading

codecov bot commented Feb 9, 2022 • edited Loading

Codecov Report

harshithmullapudi commented Mar 10, 2022 • edited by github-actions bot Loading

harshithmullapudi commented Mar 21, 2022 • edited by github-actions bot Loading

harshithmullapudi commented Mar 22, 2022 • edited by github-actions bot Loading

harshithmullapudi commented Mar 22, 2022

ron-damon commented Mar 22, 2022

harshithmullapudi commented Mar 22, 2022

bazarnov commented Mar 22, 2022

bazarnov commented Mar 22, 2022 • edited by github-actions bot Loading

bazarnov commented Mar 22, 2022

bazarnov left a comment

Choose a reason for hiding this comment

bazarnov commented Mar 22, 2022 • edited by github-actions bot Loading

🎉 Source Shopify: implement `BalanceTransactions` stream #10204

🎉 Source Shopify: implement `BalanceTransactions` stream #10204

harshithmullapudi commented Feb 9, 2022 •

edited by github-actions bot

Loading

codecov bot commented Feb 9, 2022 •

edited

Loading

harshithmullapudi commented Mar 10, 2022 •

edited by github-actions bot

Loading

harshithmullapudi commented Mar 21, 2022 •

edited by github-actions bot

Loading

harshithmullapudi commented Mar 22, 2022 •

edited by github-actions bot

Loading

bazarnov commented Mar 22, 2022 •

edited by github-actions bot

Loading

bazarnov commented Mar 22, 2022 •

edited by github-actions bot

Loading