source freshdesk, hubspot, mixpanel, paypal-transaction, salesforce, …

…zendesk-support: adding fixtures to mock time.sleep for connectors that explicitly sleep (#12335) * adding fixtures to mock time.sleep for connectors that explicitly sleep * bump connector versions * update changelog doc for each connector * auto-bump connector version * auto-bump connector version * auto-bump connector version * auto-bump connector version * auto-bump connector version * remove version bump for freshdesk because connector tests are in a bad state Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
airbytehq · May 23, 2022 · b790519 · b790519
1 parent 0d146fb
commit b790519
Show file tree

Hide file tree

Showing 27 changed files with 121 additions and 50 deletions.
diff --git a/airbyte-config/init/src/main/resources/seed/source_definitions.yaml b/airbyte-config/init/src/main/resources/seed/source_definitions.yaml
@@ -473,7 +473,7 @@
 - name: Mixpanel
   sourceDefinitionId: 12928b32-bf0a-4f1e-964f-07e12e37153a
   dockerRepository: airbyte/source-mixpanel
-  dockerImageTag: 0.1.12
+  dockerImageTag: 0.1.13
   documentationUrl: https://docs.airbyte.io/integrations/sources/mixpanel
   icon: mixpanel.svg
   sourceType: api
@@ -563,7 +563,7 @@
 - name: Paypal Transaction
   sourceDefinitionId: d913b0f2-cc51-4e55-a44c-8ba1697b9239
   dockerRepository: airbyte/source-paypal-transaction
-  dockerImageTag: 0.1.4
+  dockerImageTag: 0.1.5
   documentationUrl: https://docs.airbyte.io/integrations/sources/paypal-transaction
   icon: paypal.svg
   sourceType: api
@@ -695,7 +695,7 @@
 - name: Salesforce
   sourceDefinitionId: b117307c-14b6-41aa-9422-947e34922962
   dockerRepository: airbyte/source-salesforce
-  dockerImageTag: 1.0.5
+  dockerImageTag: 1.0.6
   documentationUrl: https://docs.airbyte.io/integrations/sources/salesforce
   icon: salesforce.svg
   sourceType: api
@@ -862,7 +862,7 @@
 - name: Zendesk Support
   sourceDefinitionId: 79c1aa37-dae3-42ae-b333-d1c105477715
   dockerRepository: airbyte/source-zendesk-support
-  dockerImageTag: 0.2.6
+  dockerImageTag: 0.2.7
   documentationUrl: https://docs.airbyte.io/integrations/sources/zendesk-support
   icon: zendesk.svg
   sourceType: api

diff --git a/airbyte-config/init/src/main/resources/seed/source_specs.yaml b/airbyte-config/init/src/main/resources/seed/source_specs.yaml
@@ -5024,7 +5024,7 @@
               path_in_connector_config:
               - "credentials"
               - "client_secret"
-- dockerImage: "airbyte/source-mixpanel:0.1.12"
+- dockerImage: "airbyte/source-mixpanel:0.1.13"
   spec:
     documentationUrl: "https://docs.airbyte.io/integrations/sources/mixpanel"
     connectionSpecification:
@@ -6122,7 +6122,7 @@
     supportsNormalization: false
     supportsDBT: false
     supported_destination_sync_modes: []
-- dockerImage: "airbyte/source-paypal-transaction:0.1.4"
+- dockerImage: "airbyte/source-paypal-transaction:0.1.5"
   spec:
     documentationUrl: "https://docs.airbyte.io/integrations/sources/paypal-transactions"
     connectionSpecification:
@@ -7346,7 +7346,7 @@
     supportsNormalization: false
     supportsDBT: false
     supported_destination_sync_modes: []
-- dockerImage: "airbyte/source-salesforce:1.0.5"
+- dockerImage: "airbyte/source-salesforce:1.0.6"
   spec:
     documentationUrl: "https://docs.airbyte.com/integrations/sources/salesforce"
     connectionSpecification:
@@ -9328,7 +9328,7 @@
               path_in_connector_config:
               - "credentials"
               - "client_secret"
-- dockerImage: "airbyte/source-zendesk-support:0.2.6"
+- dockerImage: "airbyte/source-zendesk-support:0.2.7"
   spec:
     documentationUrl: "https://docs.airbyte.io/integrations/sources/zendesk-support"
     connectionSpecification:

diff --git a/airbyte-integrations/connectors/source-freshdesk/setup.py b/airbyte-integrations/connectors/source-freshdesk/setup.py
@@ -14,6 +14,7 @@
 
 TEST_REQUIREMENTS = [
     "pytest==6.1.2",
+    "pytest-mock~=3.6",
     "requests_mock==1.8.0",
     "source-acceptance-test",
 ]

diff --git a/airbyte-integrations/connectors/source-freshdesk/unit_tests/test_client.py b/airbyte-integrations/connectors/source-freshdesk/unit_tests/test_client.py
@@ -5,11 +5,18 @@
 
 from pathlib import Path
 
+from pytest import fixture
 from source_freshdesk.client import Client
 
 HERE = Path(__file__).parent.absolute()
 
 
+@fixture(autouse=True)
+def time_sleep_mock(mocker):
+    time_mock = mocker.patch("time.sleep", lambda x: None)
+    yield time_mock
+
+
 def test_client_backoff_on_limit_reached(requests_mock):
     """Error once, check that we retry and not fail"""
     responses = [

diff --git a/airbyte-integrations/connectors/source-hubspot/setup.py b/airbyte-integrations/connectors/source-hubspot/setup.py
@@ -14,6 +14,7 @@
 
 TEST_REQUIREMENTS = [
     "pytest==6.1.2",
+    "pytest-mock~=3.6",
     "requests_mock==1.8.0",
     "source-acceptance-test",
 ]

diff --git a/airbyte-integrations/connectors/source-hubspot/unit_tests/test_source.py b/airbyte-integrations/connectors/source-hubspot/unit_tests/test_source.py
@@ -29,6 +29,12 @@
 logger = logging.getLogger("test_client")
 
 
+@pytest.fixture(autouse=True)
+def time_sleep_mock(mocker):
+    time_mock = mocker.patch("time.sleep", lambda x: None)
+    yield time_mock
+
+
 def test_check_connection_ok(requests_mock, config):
     responses = [
         {"json": [], "status_code": 200},

diff --git a/airbyte-integrations/connectors/source-hubspot/unit_tests/test_streams.py b/airbyte-integrations/connectors/source-hubspot/unit_tests/test_streams.py
@@ -33,6 +33,12 @@
 from .utils import read_full_refresh, read_incremental
 
 
+@pytest.fixture(autouse=True)
+def time_sleep_mock(mocker):
+    time_mock = mocker.patch("time.sleep", lambda x: None)
+    yield time_mock
+
+
 def test_updated_at_field_non_exist_handler(requests_mock, common_params, fake_properties_list):
     stream = ContactLists(**common_params)
 

diff --git a/airbyte-integrations/connectors/source-mixpanel/Dockerfile b/airbyte-integrations/connectors/source-mixpanel/Dockerfile
@@ -13,5 +13,5 @@ ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
 ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]
 
 
-LABEL io.airbyte.version=0.1.12
+LABEL io.airbyte.version=0.1.13
 LABEL io.airbyte.name=airbyte/source-mixpanel
diff --git a/airbyte-integrations/connectors/source-mixpanel/unit_tests/test_streams.py b/airbyte-integrations/connectors/source-mixpanel/unit_tests/test_streams.py
@@ -41,6 +41,12 @@ def patch_incremental_base_class(mocker):
     mocker.patch.object(IncrementalMixpanelStream, "__abstractmethods__", set())
 
 
+@pytest.fixture(autouse=True)
+def time_sleep_mock(mocker):
+    time_mock = mocker.patch("time.sleep", lambda x: None)
+    yield time_mock
+
+
 def test_url_base(patch_base_class):
     stream = MixpanelStream(authenticator=MagicMock())
 

diff --git a/airbyte-integrations/connectors/source-paypal-transaction/Dockerfile b/airbyte-integrations/connectors/source-paypal-transaction/Dockerfile
@@ -12,5 +12,5 @@ RUN pip install .
 ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
 ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]
 
-LABEL io.airbyte.version=0.1.4
+LABEL io.airbyte.version=0.1.5
 LABEL io.airbyte.name=airbyte/source-paypal-transaction
diff --git a/airbyte-integrations/connectors/source-paypal-transaction/setup.py b/airbyte-integrations/connectors/source-paypal-transaction/setup.py
@@ -11,6 +11,7 @@
 
 TEST_REQUIREMENTS = [
     "pytest~=6.1",
+    "pytest-mock~=3.6",
     "source-acceptance-test",
 ]
 

diff --git a/airbyte-integrations/connectors/source-paypal-transaction/unit_tests/unit_test.py b/airbyte-integrations/connectors/source-paypal-transaction/unit_tests/unit_test.py
@@ -6,9 +6,16 @@
 
 from airbyte_cdk.sources.streams.http.auth import NoAuth
 from dateutil.parser import isoparse
+from pytest import fixture
 from source_paypal_transaction.source import Balances, PaypalTransactionStream, Transactions
 
 
+@fixture(autouse=True)
+def time_sleep_mock(mocker):
+    time_mock = mocker.patch("time.sleep", lambda x: None)
+    yield time_mock
+
+
 def test_get_field():
 
     record = {"a": {"b": {"c": "d"}}}

diff --git a/airbyte-integrations/connectors/source-salesforce/Dockerfile b/airbyte-integrations/connectors/source-salesforce/Dockerfile
@@ -13,5 +13,5 @@ RUN pip install .
 
 ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]
 
-LABEL io.airbyte.version=1.0.5
+LABEL io.airbyte.version=1.0.6
 LABEL io.airbyte.name=airbyte/source-salesforce
diff --git a/airbyte-integrations/connectors/source-salesforce/setup.py b/airbyte-integrations/connectors/source-salesforce/setup.py
@@ -7,7 +7,7 @@
 
 MAIN_REQUIREMENTS = ["airbyte-cdk", "vcrpy==4.1.1", "pandas"]
 
-TEST_REQUIREMENTS = ["pytest~=6.1", "requests_mock", "source-acceptance-test", "pytest-timeout"]
+TEST_REQUIREMENTS = ["pytest~=6.1", "pytest-mock~=3.6", "requests_mock", "source-acceptance-test", "pytest-timeout"]
 
 setup(
     name="source_salesforce",

diff --git a/airbyte-integrations/connectors/source-salesforce/source_salesforce/streams.py b/airbyte-integrations/connectors/source-salesforce/source_salesforce/streams.py
@@ -485,6 +485,7 @@ class Describe(Stream):
     Stream of sObjects' (Salesforce Objects) describe:
     https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/resources_sobject_describe.htm
     """
+
     name = "Describe"
     primary_key = "name"
 

diff --git a/airbyte-integrations/connectors/source-salesforce/unit_tests/api_test.py b/airbyte-integrations/connectors/source-salesforce/unit_tests/api_test.py
@@ -24,6 +24,12 @@
 )
 
 
+@pytest.fixture(autouse=True)
+def time_sleep_mock(mocker):
+    time_mock = mocker.patch("time.sleep", lambda x: None)
+    yield time_mock
+
+
 def test_bulk_sync_creation_failed(stream_config, stream_api):
     stream: BulkIncrementalSalesforceStream = generate_stream("Account", stream_config, stream_api)
     with requests_mock.Mocker() as m:
@@ -482,7 +488,8 @@ def test_forwarding_sobject_options(stream_config, stream_names, catalog_stream_
                         "flag1": True,
                         "queryable": True,
                     }
-                    for stream_name in stream_names if stream_name != "Describe"
+                    for stream_name in stream_names
+                    if stream_name != "Describe"
                 ],
             },
         )

diff --git a/airbyte-integrations/connectors/source-salesforce/unit_tests/discovery_test.py b/airbyte-integrations/connectors/source-salesforce/unit_tests/discovery_test.py
@@ -9,6 +9,12 @@
 from source_salesforce.exceptions import TypeSalesforceException
 
 
+@pytest.fixture(autouse=True)
+def time_sleep_mock(mocker):
+    time_mock = mocker.patch("time.sleep", lambda x: None)
+    yield time_mock
+
+
 @pytest.mark.parametrize(
     "streams_criteria,predicted_filtered_streams",
     [

diff --git a/airbyte-integrations/connectors/source-salesforce/unit_tests/test_memory.py b/airbyte-integrations/connectors/source-salesforce/unit_tests/test_memory.py
@@ -11,6 +11,12 @@
 from source_salesforce.streams import BulkIncrementalSalesforceStream
 
 
+@pytest.fixture(autouse=True)
+def time_sleep_mock(mocker):
+    time_mock = mocker.patch("time.sleep", lambda x: None)
+    yield time_mock
+
+
 @pytest.mark.parametrize(
     "n_records, first_size, first_peak",
     (

diff --git a/airbyte-integrations/connectors/source-zendesk-support/Dockerfile b/airbyte-integrations/connectors/source-zendesk-support/Dockerfile
@@ -25,5 +25,5 @@ COPY source_zendesk_support ./source_zendesk_support
 ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
 ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]
 
-LABEL io.airbyte.version=0.2.6
+LABEL io.airbyte.version=0.2.7
 LABEL io.airbyte.name=airbyte/source-zendesk-support
diff --git a/airbyte-integrations/connectors/source-zendesk-support/setup.py b/airbyte-integrations/connectors/source-zendesk-support/setup.py
@@ -7,7 +7,7 @@
 
 MAIN_REQUIREMENTS = ["airbyte-cdk~=0.1.36", "pytz", "requests-futures~=1.0.0", "pendulum~=2.1.2"]
 
-TEST_REQUIREMENTS = ["pytest~=6.1", "source-acceptance-test", "requests-mock==1.9.3"]
+TEST_REQUIREMENTS = ["pytest~=6.1", "pytest-mock~=3.6", "source-acceptance-test", "requests-mock==1.9.3"]
 
 setup(
     version="0.1.0",

diff --git a/airbyte-integrations/connectors/source-zendesk-support/unit_tests/test_futures.py b/airbyte-integrations/connectors/source-zendesk-support/unit_tests/test_futures.py
@@ -21,6 +21,12 @@
 }
 
 
+@pytest.fixture(autouse=True)
+def time_sleep_mock(mocker):
+    time_mock = mocker.patch("time.sleep", lambda x: None)
+    yield time_mock
+
+
 @pytest.mark.parametrize(
     "records_count,page_size,expected_futures_deque_len",
     [

diff --git a/airbyte-integrations/connectors/source-zendesk-support/unit_tests/unit_test.py b/airbyte-integrations/connectors/source-zendesk-support/unit_tests/unit_test.py
@@ -69,6 +69,12 @@
 TEST_STREAM = TicketComments(**STREAM_ARGS)
 
 
+@pytest.fixture(autouse=True)
+def time_sleep_mock(mocker):
+    time_mock = mocker.patch("time.sleep", lambda x: None)
+    yield time_mock
+
+
 def test_str2datetime():
     expected = datetime.strptime(DATETIME_STR, DATETIME_FORMAT)
     output = BaseSourceZendeskSupportStream.str2datetime(DATETIME_STR)

diff --git a/docs/integrations/sources/freshdesk.md b/docs/integrations/sources/freshdesk.md
@@ -51,11 +51,11 @@ Please read [How to find your API key](https://support.freshdesk.com/support/sol
 
 ## Changelog
 
-| Version | Date       | Pull Request | Subject                                                                        |
-|:--------|:-----------| :--- |:-------------------------------------------------------------------------------|
-| 0.2.11  | 2021-12-14 | [8682](https://github.com/airbytehq/airbyte/pull/8682) | Migrate to the CDK                                                             |
-| 0.2.10  | 2021-12-06 | [8524](https://github.com/airbytehq/airbyte/pull/8524) | Update connector fields title/description                                      |
-| 0.2.9   | 2021-11-16 | [8017](https://github.com/airbytehq/airbyte/pull/8017) | Bugfix an issue that caused the connector not to sync more than 50000 contacts |
-| 0.2.8   | 2021-10-28 | [7486](https://github.com/airbytehq/airbyte/pull/7486) | Include "requester" and "stats" fields in "tickets" stream                     |
-| 0.2.7   | 2021-10-13 | [6442](https://github.com/airbytehq/airbyte/pull/6442) | Add start_date parameter to specification from which to start pulling data.    |
+| Version | Date       | Pull Request                                             | Subject                                                                        |
+|:--------|:-----------|:---------------------------------------------------------|:-------------------------------------------------------------------------------|
+| 0.2.11  | 2021-12-14 | [8682](https://github.com/airbytehq/airbyte/pull/8682)   | Migrate to the CDK                                                             |
+| 0.2.10  | 2021-12-06 | [8524](https://github.com/airbytehq/airbyte/pull/8524)   | Update connector fields title/description                                      |
+| 0.2.9   | 2021-11-16 | [8017](https://github.com/airbytehq/airbyte/pull/8017)   | Bugfix an issue that caused the connector not to sync more than 50000 contacts |
+| 0.2.8   | 2021-10-28 | [7486](https://github.com/airbytehq/airbyte/pull/7486)   | Include "requester" and "stats" fields in "tickets" stream                     |
+| 0.2.7   | 2021-10-13 | [6442](https://github.com/airbytehq/airbyte/pull/6442)   | Add start_date parameter to specification from which to start pulling data.    |
 
diff --git a/docs/integrations/sources/mixpanel.md b/docs/integrations/sources/mixpanel.md
@@ -59,6 +59,7 @@ Select the correct region \(EU or US\) for your Mixpanel project. See detail [he
 
 | Version  | Date       | Pull Request                                             | Subject                                                                                              |
 |:---------|:-----------|:---------------------------------------------------------|:-----------------------------------------------------------------------------------------------------|
+| `0.1.13` | 2022-04-27 | [12335](https://github.com/airbytehq/airbyte/pull/12335) | Adding fixtures to mock time.sleep for connectors that explicitly sleep                              |  
 | `0.1.12` | 2022-03-31 | [11633](https://github.com/airbytehq/airbyte/pull/11633) | Increase unit test coverage                                                                          |  
 | `0.1.11` | 2022-04-04 | [11318](https://github.com/airbytehq/airbyte/pull/11318) | Change Response Reading                                                                              |
 | `0.1.10` | 2022-03-31 | [11227](https://github.com/airbytehq/airbyte/pull/11227) | Fix cohort id always null in the cohort_members stream                                               |

diff --git a/docs/integrations/sources/paypal-transaction.md b/docs/integrations/sources/paypal-transaction.md
@@ -55,11 +55,12 @@ Transactions sync is performed with default `stream_slice_period` = 1 day, it me
 
 ## Changelog
 
-| Version | Date | Pull Request | Subject |
-| :--- | :--- | :--- | :--- |
-| 0.1.4 | 2021-12-22 | [9034](https://github.com/airbytehq/airbyte/pull/9034) | Update connector fields title/description |
-| 0.1.3 | 2021-12-16 | [8580](https://github.com/airbytehq/airbyte/pull/8580) | Added more logs during `check connection` stage |
-| 0.1.2 | 2021-11-08 | [7499](https://github.com/airbytehq/airbyte/pull/7499) | Remove base-python dependencies |
-| 0.1.1 | 2021-08-03 | [5155](https://github.com/airbytehq/airbyte/pull/5155) | fix start\_date\_min limit |
-| 0.1.0 | 2021-06-10 | [4240](https://github.com/airbytehq/airbyte/pull/4240) | PayPal Transaction Search API |
+| Version | Date       | Pull Request                                             | Subject                                                                 |
+|:--------|:-----------|:---------------------------------------------------------|:------------------------------------------------------------------------|
+| 0.1.5   | 2022-04-27 | [12335](https://github.com/airbytehq/airbyte/pull/12335) | Adding fixtures to mock time.sleep for connectors that explicitly sleep |
+| 0.1.4   | 2021-12-22 | [9034](https://github.com/airbytehq/airbyte/pull/9034)   | Update connector fields title/description                               |
+| 0.1.3   | 2021-12-16 | [8580](https://github.com/airbytehq/airbyte/pull/8580)   | Added more logs during `check connection` stage                         |
+| 0.1.2   | 2021-11-08 | [7499](https://github.com/airbytehq/airbyte/pull/7499)   | Remove base-python dependencies                                         |
+| 0.1.1   | 2021-08-03 | [5155](https://github.com/airbytehq/airbyte/pull/5155)   | fix start\_date\_min limit                                              |
+| 0.1.0   | 2021-06-10 | [4240](https://github.com/airbytehq/airbyte/pull/4240)   | PayPal Transaction Search API                                           |
 
diff --git a/docs/integrations/sources/salesforce.md b/docs/integrations/sources/salesforce.md
@@ -121,7 +121,8 @@ Now that you have set up the Salesforce source connector, check out the followin
 ## Changelog
 
 | Version | Date       | Pull Request | Subject                                                                                                                          |
-|:--------|:-----------| :--- |:---------------------------------------------------------------------------------------------------------------------------------|
+|:--------|:-----------|:---|:---------------------------------------------------------------------------------------------------------------------------------|
+| 1.0.4 | 2022-04-27 | [12335](https://github.com/airbytehq/airbyte/pull/12335) | Adding fixtures to mock time.sleep for connectors that explicitly sleep |
 | 1.0.3 | 2022-04-04 | [11692](https://github.com/airbytehq/airbyte/pull/11692) | Optimised memory usage for `BULK` API calls |
 | 1.0.2 | 2022-03-01 | [10751](https://github.com/airbytehq/airbyte/pull/10751) | Fix broken link anchor in connector configuration |
 | 1.0.1 | 2022-02-27 | [10679](https://github.com/airbytehq/airbyte/pull/10679) | Reorganize input parameter order on the UI |