Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split providers out of the main "airflow/" tree into a UV workspace project #42505

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ashb
Copy link
Member

@ashb ashb commented Sep 26, 2024

As discussed https://lists.apache.org/thread/dyv5jhvt65xs6l5o2byc2b67f4wlwf6r, this is the first part of the new layout.

This is only a partial split so far. It moves all the code and tests, but the provider release generation code hasn't moved yet as it's a few weeks until the next batch, and I wan't to try and keep this already huge PR as small as possible. Likewise the creation of core/ will be a follow on PR

In addition to the straight file rename the other changes I had to make here are:

  • Some mypy/typing fixes.

    Mypy can be fragile about what it picks up when, so maybe some of those changes were caused by that. But the typing changes aren't large.

  • Improve typing in common.sql type stub

    Again, likely a mypy file oddity, but the types should be safe

  • Removed the check-providers-init-file-missing check

    This isn't needed now that airflow/providers shouldn't exist at all in the main tree.

In future we should switch how the CI image is built to make better use of UV caching than our own approach as that would remvoe a lot of custom code.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@ashb
Copy link
Member Author

ashb commented Sep 26, 2024

Oh, prepare provider packages is failing for prod image build. I guess that's going to have to be in this PR then

@ashb ashb force-pushed the uv-workspace-for-providers branch 2 times, most recently from 362c123 to 47d7155 Compare September 26, 2024 16:05
…roject

This is only a partial split so far. It moves all the code and tests, but
leaves the creation of `core/` to a separate PR as this is already large
enough.

In addition to the straight file rename the other changes I had to make here
are:

- Some mypy/typing fixes.

  Mypy can be fragile about what it picks up when, so maybe some of those
  changes were caused by that. But the typing changes aren't large.

- Improve typing in common.sql type stub

  Again, likely a mypy file oddity, but the types should be safe

- Removed the `check-providers-init-file-missing` check

  This isn't needed now that airflow/providers shouldn't exist at all in the
  main tree.

In future we should switch how the CI image is built to make better use of UV
caching than our own approach as that would remvoe a lot of custom code.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant