question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Release a 2nd wave of Backport packages

See original GitHub issue

Description

I think we are really close to release the second wave of Backport Packages. We accumulated a number of changes already and few more are about to be completed so that we are, I think, close to release the second wave.

** What will be in those packages **

Everything that is currently merged to the master branch in “providers” package will be included.

Those are open issues that we know we should merge before though:

  • ~Enabling cncf.kubernetes provider after 1.10.11/1.10.12 backported changes and spark-on-kubernetes operator~ - this is deferred to the next backport release as per #11298
  • Implementing Impersonation for all applicable Google operators (@olchas works on it with Hooks already merged)
  • DataProc operator improvements (@turbaszek works on it in #9593 )
  • Improve idempotency of BigQueryInsertJobOperator (@turbaszek works on it in #9590)
  • Release 1.10.12 of Airflow as it fixes some teething problems with Kubernetes refactor from 1.10.11
  • Fix BigQuery operators problem with serialization #10171
  • AWS SES #10004
  • Fix the DLP operators.
  • ~finalize and implement the #10822 allowing to release 1.10.13 with support to auto-dicover provider capabilities such as connections and extra links~ deferred to another release
  • ~Merge #10488 with Kerberos on Presto change.~ deferred to the next release
  • ~Breaking changes: Email operator moved to Core: we are going to address it in 1.10.13 by adding email provider there and releasing provider package via #11178~

We have to make the following adjustments to the process:

  • make sure that we only release the packages that have some changes #back #10375
  • make sure to add changes from the previous releases - not only commits but add mechanism to add backwards-incompatible changes akin to UPDATING.md #10376

Update: The second wave is delayed a bit (but coming soon). The reason is two-fold

  • the cncf.kubernetes package has a problem that we are solving
  • we might try to get the mechanism of separate providers packages and “discovery” mechanism before we build the packages - see the #10822 and #10823

Use case / motivation

We want the users of airflow 1.10 to benefit from fixes and improvements in the latest packages.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:15
  • Comments:18 (16 by maintainers)

github_iconTop GitHub Comments

3reactions
mik-lajcommented, Oct 5, 2020
3reactions
mik-lajcommented, Aug 27, 2020

Yes. It is planned.

Read more comments on GitHub >

github_iconTop Results From Across the Web

[GitHub] [airflow] guptamyr commented on issue ... - The Mail Archive
guptamyr commented on issue #10014: URL: https://github.com/apache/airflow/issues/10014#issuecomment-689769252. Hi, Has the 2nd wave of backport packages ...
Read more >
Package apache-airflow-providers-apache-pinot
This is detailed commit list of changes for versions provider package: apache.pinot . For high-level changelog, see package information including changelog.
Read more >
apache-airflow-backport-providers-apache-pig 2021.3.17 - PyPI
Backport provider package apache-airflow-backport-providers-apache-pig for Apache ... 88bdcfa0d, 2021-02-04, Prepare to release a new wave of providers.
Read more >
Debian Backports
Backports are packages taken from the next Debian release (called "testing"), adjusted and recompiled for usage on Debian stable. Because the package is ......
Read more >
apache-airflow-backport-providers-slack - Python package
Learn more about apache-airflow-backport-providers-slack: package health score, ... 88bdcfa0d, 2021-02-04, Prepare to release a new wave of providers.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found