Status of testing of Apache Airflow 2.7.3rc1

[ephraimbuddy]

Body

We are kindly requesting that contributors to Apache Airflow RC 2.7.3rc1 help test the RC.

Please let us know by commenting if the issue is addressed in the latest RC.

• fix(TimeSensorAsync): use DAG timezone (#33406): @SapphicCode
  Linked issues:
  • TimeSensorAsync does not use DAG timezone to convert naive time input (#33256)
• Add read only validation to read only fields (#33413): @ahidalgob
• Bump MariaDB client libraries to latest LTS (#33722): @Taragolis
• Improve modules import in Airflow core by some of them into a type-checking block (#33755): @hussein-awala
• Improve importing the module in Airflow utils package (#33803): @hussein-awala
• Refactor string splitting (#34185): @eumiro
• Fix pre-mature evaluation of tasks in mapped task group (#34337): @ephraimbuddy
• Mark tasks with all_skipped trigger rule as skipped if any task is in upstream_failed state (#34392): @TJaniF
• Add init to params subpackage (#34710): @uranusjr
• Bump azure-mgmt-containerinstance (#34738): @pankajastro
• Remove warning about max_tis per query > parallelism (#34742): @dstandish
• Remove unused set from SchedulerJobRunner (#34810): @jedcunningham
• Fix usage of cron-descriptor for custom TimeTables since BC in v1.3.0 (#34836): @poupounetjoyeux
• REST API: Fix wrong plugin schema (#34858): @ephraimbuddy
• Fix triggerer thread crash in daemon mode (#34931): @Bisk1 @ThuanDoHung
  Linked issues:
  • Airflow 2.7.1 can not start Scheduler & trigger (#34816)
• update moto version (#34938): @eladkal
• Return only the TIs of the readable dags when ~ is provided as a dag_id (#34939): @hussein-awala
• Some improvements/fixes for dag_run and task_instance endpoints (#34942): @hussein-awala
• Limit WTForms to below 3.1.0 (#34943): @potiuk
• Fix the dags count filter in webserver home page (#34944): @hussein-awala
  Linked issues:
  • Improve importing the modules in Airflow www package (#33810)
• Fix tests to respond to Python 3.12 handling of utcnow in sentry-sdk (#34946): @potiuk
• Fix typo in error message when render_content method is not implemented (#34952): @utkarsharma2
• Fix get_plugin_info for class based listeners. (#35022): @JDarDagran
• feat(dockerfile): Add pip caching for faster build (#35026): @V0lantis
• Upgrade pip to 23.3 (#35029): @potiuk
• Allow to install MariaDB client libraries instead of MySQL on x86_64 (#35070): @Taragolis @Sunny-Island
  Linked issues:
  • MYSQL_OPT_RECONNECT is deprecated. When exec airflow db upgrade. (#32708)
• Fix test connection with codemirror and extra (#35122): @vchiapaikeo
• Limit Airflow and breeze to < 3.12 (#35123): @Taragolis @potiuk
  Linked issues:
  • Make core compatible with pendulum 3 (#34744)
  • Use pre-release version of pendulum in CI and breeze (#34746)
• Bump min version of presto client to bring better dbapi compatibility (#35132): @potiuk @yanivsin
  Linked issues:
  • [airflow 2.4.3] presto queries returning none following upgrade to common.sql provider (#31612)
• Fix Scheduler crash looping when dagrun creation fails (#35135): @ephraimbuddy
• Upgrade pip to 23.3.1 (#35150): @potiuk
• Bump docker cli version in Airflow Images (#35153): @potiuk
• Improve testing harness to separate DB and non-DB tests (#35160): @potiuk
• Add TriggerRule missing value in rest API (#35194): @pankajastro
• Add connexion<3.0 upper bound (#35218): @Kache
• Fix subtle bug in mocking processor_agent in our tests (#35221): @potiuk
  Linked issues:
  • Mysterious hanging of the test_retry_handling_job for sqlite on self-hosted/local env (#35204)
• Add pytest db_test markers to our tests (#35264): @potiuk
  Linked issues:
  • Improve testing harness to separate DB and non-DB tests (#35160)
• Limit 1.33.0 broken sentry_sdk released 31st of October 2023 (#35298): @potiuk
• Upper bound pendulum requirement to <3.0 (#35336): @Taragolis

Thanks to all who contributed to the release (probably not a complete list!):
@eumiro @SapphicCode @TJaniF @vchiapaikeo @dstandish @Bisk1 @yanivsin @jedcunningham @pankajastro @Sunny-Island @poupounetjoyeux @JDarDagran @potiuk @ThuanDoHung @eladkal @Kache @utkarsharma2 @hussein-awala @uranusjr @V0lantis @ahidalgob @Taragolis @ephraimbuddy

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.

[JDarDagran]

#35022 working as expected.

[poupounetjoyeux]

It's ok for #34836

[utkarsharma2]

#34952 <#34952> working fine. Thanks, Utkarsh Sharma

…

On Thu, Nov 2, 2023 at 6:07 PM Quentin Bray ***@***.***> wrote: It's ok for #34836 <#34836> — Reply to this email directly, view it on GitHub <#35372 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADDLAHMQN36Z3ESQOL3HN7TYCOHXVAVCNFSM6AAAAAA62ZATRCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJQGY2DSMJZGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[vchiapaikeo]

#35122 working as expected

[Taragolis]

Successfully build custom image with MariaDB client libraries instead of Oracle MySQL from https://github.com/apache/airflow/tree/2.7.3rc1

• Bump MariaDB client libraries to latest LTS #33722
• Allow to install MariaDB client libraries instead of MySQL on x86_64 #35070

Check packages metadata, that limitation for packages/python included in it

• Limit Airflow and breeze to < 3.12 #35123
• Add connexion<3.0 upper bound #35218
• Upper bound pendulum requirement to <3.0 #35336
• Limit WTForms to below 3.1.0 #34943
• Limit 1.33.0 broken sentry_sdk released 31st of October 2023 #35298

[Taragolis]

#35132 also included into the all_dbs package extra

[potiuk]

Checked all my changes - some of them were test only, others work fine 👍

I have however found that pip version in the final PROD image is not the expected one (23.0.1 instead of 23.3.1 - what would be suggested by #35029 and #35150). This is nothing critical or even serious - those are relatively new versions of pip and it's not a regression - it seems that we've always had this problem, simply pip version was not upgraded in the main segment of the image as the installed dependencies were copied from build image and we never upgraded pip in main image.

I opened PR with fix #35387 but I think it's not a blocker for RC1, however if we do have a blocker and prepare RC2 and merge my PR, it should be cherry-picked as well.

[Bisk1]

#34931 works as expected

[hussein-awala]

I checked my changes, they all work as expected.

[pankajastro]

#34738 and #35194 looks good 👍

[ephraimbuddy]

Airflow 2.7.3 has been released. Thank you all for testing this release!

[ThuanDoHung]

So sorry i late,
I try install airflow 2.7.3 release, and check bug start scheduler, trigger on daemon. I show scheduler run daemon fine, but triggerer not run daemon.

Bug of trigger daemon same: #34816
ps:

1. I upgrade airflow 2.7.2 to 27.3
2. start: airflow triggerer -D (not run), airflow triggerer: run fine

[potiuk]

Please open a new issue about it as asked .