Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add default_deferrable config #31712

Merged
merged 51 commits into from
Jul 5, 2023
Merged

Add default_deferrable config #31712

merged 51 commits into from
Jul 5, 2023

Commits on Jul 5, 2023

  1. feat(operator): add deferrable attribute to base operator

    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    76943ee View commit details
    Browse the repository at this point in the history

  2. feat(operators): change the default value of deferrable from false to… 

    … DEFAULT_DEFERRABLE
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    0644d72 View commit details
    Browse the repository at this point in the history

  3. feat(providers): add backward compatitiliby for DEFAULT_DEFERRABLE

    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    862b4a8 View commit details
    Browse the repository at this point in the history

  4. fix(providers.amazon): change the default deferrable value to False f… 

    …or AwsGenericHook.get_client_type

apache#31712 (comment)
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    7fb65b5 View commit details
    Browse the repository at this point in the history

  5. fix(operators): remove unnecessary fallback when gettting DEFAULT_DEF… 

    …ERRABLE
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    ba5bab0 View commit details
    Browse the repository at this point in the history

  6. fix(serialization/schema.json): revert json format

    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    472f53f View commit details
    Browse the repository at this point in the history

  7. docs(providers): add a todo to remove unnecessary try-exception block… 

    … after min airflow version updated to 2.7.0
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    74f114f View commit details
    Browse the repository at this point in the history

  8. Revert "docs(providers): add a todo to remove unnecessary try-excepti… 

    …on block after min airflow version updated to 2.7.0"

This reverts commit da6c89354fe09fcb1d8f06a69fe891c09cc6b110.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    b730aed View commit details
    Browse the repository at this point in the history

  9. Revert "fix(serialization/schema.json): revert json format" 

    This reverts commit 03a27dad2537ab93f83a6a68b93d35e5b015b464.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    3cfa751 View commit details
    Browse the repository at this point in the history

  10. Revert "fix(operators): remove unnecessary fallback when gettting DEF… 

    …AULT_DEFERRABLE"

This reverts commit ef8d7041aa30ade2417ccecefedd56707f516e07.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    81c5924 View commit details
    Browse the repository at this point in the history

  11. Revert "fix(providers.amazon): change the default deferrable value to… 

    … False for AwsGenericHook.get_client_type"

This reverts commit b65f66ec146fcbeff9961211a9caf1a2f2a1f8bc.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    99121a8 View commit details
    Browse the repository at this point in the history

  12. Revert "feat(providers): add backward compatitiliby for DEFAULT_DEFER… 

    …RABLE"

This reverts commit 5ff43ed14d602ae9e8b62fb63813f0dbc762b3b1.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    2725192 View commit details
    Browse the repository at this point in the history

  13. Revert "feat(operators): change the default value of deferrable from … 

    …false to DEFAULT_DEFERRABLE"

This reverts commit d90f531bd951981772cadc9baebf43a709c7d70e.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    0dc2b1c View commit details
    Browse the repository at this point in the history

  14. Revert "feat(operator): add deferrable attribute to base operator" 

    This reverts commit 36344c902eb5b6e41765dda4b7b50ed6521fd624.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    1dc6363 View commit details
    Browse the repository at this point in the history

  15. feat(config_template): add operators.default_deferrable option

    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    e209f9c View commit details
    Browse the repository at this point in the history

  16. feat(models/deferrablemixin): add deferrablemixin to load default_def… 

    …errable and assign it to deferrable
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    75008bb View commit details
    Browse the repository at this point in the history

  17. feat(providers): use DEFAULT_DEFERRABLE from deferrablemixin as the d… 

    …efault value of deferable
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    6d35f83 View commit details
    Browse the repository at this point in the history

  18. feat(operators/trigger_dagrun): use DeferrableMixin for TriggerDagRun… 

    …Operator
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    5c3e2e0 View commit details
    Browse the repository at this point in the history

  19. test(models/deferrablemixin): add test case for deferrablemixin

    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    2806b9e View commit details
    Browse the repository at this point in the history

  20. Revert "test(models/deferrablemixin): add test case for deferrablemixin" 

    This reverts commit 586713012c8951d709be245ff6da8cf20314542f.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    462e69b View commit details
    Browse the repository at this point in the history

  21. Revert "feat(operators/trigger_dagrun): use DeferrableMixin for Trigg… 

    …erDagRunOperator"

This reverts commit fca47aab773cd6bfe32bc88a6fdc0737c32267e3.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    1e4d928 View commit details
    Browse the repository at this point in the history

  22. Revert "feat(models/deferrablemixin): add deferrablemixin to load def… 

    …ault_deferrable and assign it to deferrable"

This reverts commit 9289c089d07425b4d10e1e91f0b41123fcfd4c10.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    b360b1f View commit details
    Browse the repository at this point in the history

  23. Revert "feat(providers): use DEFAULT_DEFERRABLE from deferrablemixin … 

    …as the default value of deferable"

This reverts commit aae2594f16adfcfb0861358267e5ca04b820e09a.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    e2796e3 View commit details
    Browse the repository at this point in the history

  24. feat(models/basedeferrableoperator): add BaseDeferrableOperator class… 

    … for operator with deferrable attribute and load this value from config
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    7b3d743 View commit details
    Browse the repository at this point in the history

  25. feat(providers): use DEFAULT_DEFERRABLE from basedeferrableoperator a… 

    …s the default value of deferabl
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    6baad78 View commit details
    Browse the repository at this point in the history

  26. feat(operators/trigger_dagrun): inherit BaseDeferrableOperator for Tr… 

    …iggerDagRunOperator
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    37f3de8 View commit details
    Browse the repository at this point in the history

  27. test(models/basedeferrableoperator): add test case for BaseDeferrable… 

    …Operator
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    5e986ff View commit details
    Browse the repository at this point in the history

  28. refactor(models/baseoperator): merge basedeferrableoperator to baseop… 

    …erator
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    067e8ff View commit details
    Browse the repository at this point in the history

  29. test(baseoperator): create a dummy operator that inherits BaseDeferra… 

    …bleOperator with empty execute method implementation
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    9ccf7d4 View commit details
    Browse the repository at this point in the history

  30. test(baseoperator): remove reload and only check whether the deferrab… 

    …le attribute in BaseDeferrableOperator is the same as DEFAULT_DEFERRABLE
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    f74fdbe View commit details
    Browse the repository at this point in the history

  31. test(baseoperator): remove TestBaseDeferrableOperator class 

    Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>
    @Lee-W @uranusjr
    Lee-W and uranusjr committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    29069ac View commit details
    Browse the repository at this point in the history

  32. fix(providers/amazon): change deferrable default value back to False… 

    … for waiter and hooks
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    acfa369 View commit details
    Browse the repository at this point in the history

  33. style(airflow/config_templates/default_airflow.cfg): unify false capt… 

    …ialization to False
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    5878f21 View commit details
    Browse the repository at this point in the history

  34. docs(baseoperator): remove abstract in BaseDeferrableOperator docstring

    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    d1dee83 View commit details
    Browse the repository at this point in the history

  35. test(baseoperator): add missing import

    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    0f7921c View commit details
    Browse the repository at this point in the history

  36. Revert "test(baseoperator): add missing import" 

    This reverts commit c38c84921ef240abe073fd38251a4aef570fcabf.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    850b137 View commit details
    Browse the repository at this point in the history

  37. Revert " docs(baseoperator): remove abstract in BaseDeferrableOperato… 

    …r docstring"

This reverts commit 128ef7fe3bbabbc75769339c00d7842486c08146.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    c8ee403 View commit details
    Browse the repository at this point in the history

  38. Revert "style(airflow/config_templates/default_airflow.cfg): unify fa… 

    …lse captialization to False"

This reverts commit adaf5ca52eda4f8a7303dabd150c6a97839a53a9.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    6ff6cb9 View commit details
    Browse the repository at this point in the history

  39. Revert " fix(providers/amazon): change deferrable default value back … 

    …to False for waiter and hooks"

This reverts commit bdac07b635b96ad1d6ebd2113512fb953d622a80.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    bca2621 View commit details
    Browse the repository at this point in the history

  40. Revert "test(baseoperator): remove TestBaseDeferrableOperator class" 

    This reverts commit 0129a0e9606c5e85bf6939386ce910b636b721e9.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    c62be0d View commit details
    Browse the repository at this point in the history

  41. Revert "test(baseoperator): remove reload and only check whether the … 

    …deferrable attribute in BaseDeferrableOperator is the same as DEFAULT_DEFERRABLE"

This reverts commit bb6ca2105659cd6a5107e2913599655f491e6de8.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    154d34b View commit details
    Browse the repository at this point in the history

  42. Revert "test(baseoperator): create a dummy operator that inherits Bas… 

    …eDeferrableOperator with empty execute method implementation"

This reverts commit a2d358f9bd20d3c010d1fa8efdcb7d2bfd546e42.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    02ea257 View commit details
    Browse the repository at this point in the history

  43. Revert "refactor(models/baseoperator): merge basedeferrableoperator t… 

    …o baseoperator"

This reverts commit 3f8c72e27496bb87abb3fa7a6715c63fd3acb2ea.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    bf2174e View commit details
    Browse the repository at this point in the history

  44. Revert "test(models/basedeferrableoperator): add test case for BaseDe… 

    …ferrableOperator"

This reverts commit 9f85338a4eb49289e896f5844fd0d67897dafbd5.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    f291ba3 View commit details
    Browse the repository at this point in the history

  45. Revert "feat(operators/trigger_dagrun): inherit BaseDeferrableOperato… 

    …r for TriggerDagRunOperator"

This reverts commit 838fa28e7888937169d938143ee57fbafa41abf8.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    b78f257 View commit details
    Browse the repository at this point in the history

  46. Revert "feat(providers): use DEFAULT_DEFERRABLE from basedeferrableop… 

    …erator as the default value of deferabl"

This reverts commit a17afa02e5b6fd9bf4046be4188e229e12c72060.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    29908c3 View commit details
    Browse the repository at this point in the history

  47. Revert "feat(models/basedeferrableoperator): add BaseDeferrableOperat… 

    …or class for operator with deferrable attribute and load this value from config"

This reverts commit 26404c231d55efe4466090a29a37a6e85316d690.
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    cc8c6c4 View commit details
    Browse the repository at this point in the history

  48. feat(providers): use "conf.getboolean("operators", "default_deferrabl… 

    …e", fallback=False)" as the default deferrable value
    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    fa2b136 View commit details
    Browse the repository at this point in the history

  49. docs: fix description error

    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    5dee336 View commit details
    Browse the repository at this point in the history

  50. docs(deferring): add instruction on adding deferrable attribute

    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    2506bde View commit details
    Browse the repository at this point in the history

  51. style: fix style error

    @Lee-W
    Lee-W committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    b9f870e View commit details
    Browse the repository at this point in the history