Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prepare docs for Nov 1st wave of providers #44011

Merged
merged 3 commits into from
Nov 14, 2024
Merged

Conversation

eladkal
Copy link
Contributor

@eladkal eladkal commented Nov 14, 2024

Summary of prepared documentation:

Success: 45

amazon apache.beam apache.drill apache.druid apache.hive apache.impala apache.pinot apache.spark apprise atlassian.jira celery cloudant cncf.kubernetes common.compat common.sql databricks dbt.cloud
docker elasticsearch exasol fab google http jdbc microsoft.azure microsoft.mssql microsoft.winrm mysql odbc openlineage oracle pagerduty papermill pinecone postgres presto slack smtp snowflake sqlite
standard teradata trino vertica ydb

Docs only: 43

airbyte alibaba apache.cassandra apache.flink apache.hdfs apache.iceberg apache.kafka apache.kylin apache.pig arangodb asana cohere common.io datadog dingding discord facebook ftp github grpc hashicorp
imap influxdb microsoft.psrp neo4j openai openfaas opensearch opsgenie pgvector qdrant redis salesforce samba segment sendgrid sftp singularity tableau telegram weaviate yandex zendesk

Skipped on no changes: 4

apache.livy jenkins mongo ssh


Successfully prepared documentation for packages!

@potiuk
Copy link
Member

potiuk commented Nov 14, 2024

Hmm. Interesting thing - now we have circular dependency: compat -> standard , standard->compat ... which is generally fine but we might have some interesting CI issues because of that :)

@potiuk
Copy link
Member

potiuk commented Nov 14, 2024

That will also mean @eladkal that we MUST release standard provider with this wave - otherwise common won't be installable.

It's currently 0.0.1 - which I think migt be a good idea to keep it at 0.* until we migrate all things out from core?

@eladkal
Copy link
Contributor Author

eladkal commented Nov 14, 2024

That will also mean @eladkal that we MUST release standard provider with this wave - otherwise common won't be installable.

It's currently 0.0.1 - which I think migt be a good idea to keep it at 0.* until we migrate all things out from core?

that is the plan. This wave is pretty much all or nothing.

It's 0.0.1 till we make it "stable" by extracting everything.

@potiuk
Copy link
Member

potiuk commented Nov 14, 2024

Very interesting error. Side effect of another test. Fix is coming.

_______________________ TestAutoMLHook.test_get_dataset ________________________
[gw2] linux -- Python 3.9.20 /usr/local/bin/python
/opt/airflow/providers/tests/google/cloud/hooks/test_automl.py:258: in test_get_dataset
    self.hook.get_dataset(dataset_id=DATASET_ID, location=GCP_LOCATION, project_id=GCP_PROJECT_ID)
/opt/airflow/providers/src/airflow/providers/google/common/hooks/base_google.py:560: in inner_wrapper
    return func(self, *args, **kwargs)
/opt/airflow/providers/src/airflow/providers/google/cloud/hooks/automl.py:669: in get_dataset
    client = self.get_conn()
/opt/airflow/providers/src/airflow/providers/google/cloud/hooks/automl.py:97: in get_conn
    self._client = AutoMlClient(credentials=self.get_credentials(), client_info=CLIENT_INFO)
/usr/local/lib/python3.9/site-packages/google/cloud/automl_v1beta1/services/auto_ml/client.py:793: in __init__
    self._transport = transport_init(
/usr/local/lib/python3.9/site-packages/google/cloud/automl_v1beta1/services/auto_ml/transports/grpc.py:145: in __init__
    if isinstance(channel, grpc.Channel):
E   TypeError: isinstance() arg 2 must be a type or tuple of types

potiuk added a commit to potiuk/airflow that referenced this pull request Nov 14, 2024
The grpc.Channel has been patched but not relased in the test_grpc.py
and it could have caused other tests failing - when they were run
later in the same interpreter. For example it failed in in apache#44011 in the
apache#44011 (comment)

Patching is now fixed via using fixtures.
@potiuk
Copy link
Member

potiuk commented Nov 14, 2024

The failure is fixed in #44029

@eladkal
Copy link
Contributor Author

eladkal commented Nov 14, 2024

The failure is fixed in #44029

Cool so no need to regenerate docs as the fix is only in a test

potiuk added a commit that referenced this pull request Nov 14, 2024
The grpc.Channel has been patched but not relased in the test_grpc.py
and it could have caused other tests failing - when they were run
later in the same interpreter. For example it failed in in #44011 in the
#44011 (comment)

Patching is now fixed via using fixtures.
~~~~

* ``Move bash operator to Standard provider (#42252)``
* ``Purge existing SLA implementation (#42285)``
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm this one is because https://github.com/apache/airflow/pull/42285/files#diff-8fa4fd0e9c359df89fde6c3043b13aa7d36d9f63aebc1744bffc43e00b158886L26 - i.e. removal of sla_miss documentation. This is a small thing and probably not worth reverting, but it looks like the providers affected (pagerduty, slack, smtp) will miss the sla_miss documetnation. Not a big deal, would be better to have a note instead that it wil only work in 2.0 - @ferruzzi -> not sure if this is worth the effort.

Copy link
Member

@potiuk potiuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed in a pretty detailed way. Gernerally approved, but there are few questions/comments

@eladkal eladkal merged commit a53d9f6 into apache:main Nov 14, 2024
67 checks passed
@eladkal eladkal deleted the providersnew branch November 14, 2024 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants