chore(airflow): Extend providers/extras for 3.0.6

[adwk67]

Description

This PR extends the list of extras/providers that will be installed when building the image for Airflow 3.0.6.
This is the only version that has been changed as

• Airflow 2.x versions are deprecated and will not be supported as of SDP 26.3
• Airflow 3.0.1 is marked as experimental and 3.0.6 should be preferred over it

Fixes stackabletech/airflow-operator#705

🟢 Local tests (all 3.0.6 ones):

--- PASS: kuttl (3451.47s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/orphaned-resources_airflow-latest-3.0.6_openshift-false (151.02s)
        --- PASS: kuttl/harness/ldap_airflow-latest-3.0.6_ldap-authentication-server-verification-tls_openshift-false_executor-celery (344.73s)
        --- PASS: kuttl/harness/opa_airflow-3.0.6_opa-latest-1.8.0_openshift-false (236.56s)
        --- PASS: kuttl/harness/ldap_airflow-latest-3.0.6_ldap-authentication-no-tls_openshift-false_executor-kubernetes (247.23s)
        --- PASS: kuttl/harness/ldap_airflow-latest-3.0.6_ldap-authentication-no-tls_openshift-false_executor-celery (217.06s)
        --- PASS: kuttl/harness/ldap_airflow-latest-3.0.6_ldap-authentication-insecure-tls_openshift-false_executor-kubernetes (203.87s)
        --- PASS: kuttl/harness/ldap_airflow-latest-3.0.6_ldap-authentication-insecure-tls_openshift-false_executor-celery (199.12s)
        --- PASS: kuttl/harness/cluster-operation_airflow-latest-3.0.6_openshift-false (296.97s)
        --- PASS: kuttl/harness/overrides_airflow-latest-3.0.6_openshift-false (332.07s)
        --- PASS: kuttl/harness/mount-dags-configmap_airflow-latest-3.0.6_openshift-false_executor-kubernetes (175.69s)
        --- PASS: kuttl/harness/mount-dags-configmap_airflow-latest-3.0.6_openshift-false_executor-celery (173.23s)
        --- PASS: kuttl/harness/remote-logging_airflow-latest-3.0.6_openshift-false_executor-kubernetes (253.39s)
        --- PASS: kuttl/harness/remote-logging_airflow-latest-3.0.6_openshift-false_executor-celery (282.43s)
        --- PASS: kuttl/harness/triggerer_airflow-latest-3.0.6_openshift-false_executor-kubernetes (198.79s)
        --- PASS: kuttl/harness/triggerer_airflow-latest-3.0.6_openshift-false_executor-celery (194.84s)
        --- PASS: kuttl/harness/mount-dags-gitsync_airflow-latest-3.0.6_openshift-false_executor-celery (540.37s)
        --- PASS: kuttl/harness/smoke_airflow-3.0.6_openshift-false_executor-kubernetes (207.59s)
        --- PASS: kuttl/harness/smoke_airflow-3.0.6_openshift-false_executor-celery (294.79s)
        --- PASS: kuttl/harness/mount-dags-gitsync_airflow-latest-3.0.6_openshift-false_executor-kubernetes (292.83s)
        --- PASS: kuttl/harness/resources_airflow-latest-3.0.6_openshift-false (180.65s)
        --- PASS: kuttl/harness/oidc_airflow-latest-3.0.6_openshift-false (277.30s)
        --- PASS: kuttl/harness/logging_airflow-3.0.6_openshift-false_executor-kubernetes (505.75s)
        --- PASS: kuttl/harness/external-access_airflow-3.0.6_openshift-false_executor-celery (212.47s)
        --- PASS: kuttl/harness/logging_airflow-3.0.6_openshift-false_executor-celery (390.04s)
        --- PASS: kuttl/harness/ldap_airflow-latest-3.0.6_ldap-authentication-server-verification-tls_openshift-false_executor-kubernetes (157.87s)
        --- PASS: kuttl/harness/external-access_airflow-3.0.6_openshift-false_executor-kubernetes (213.32s)
PASS

Connections are now available in the UI:

Definition of Done Checklist

Note

Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant.

Please make sure all these things are done and tick the boxes

• Changes are OpenShift compatible
• All added packages (via microdnf or otherwise) have a comment on why they are added
• Things not downloaded from Red Hat repositories should be mirrored in the Stackable repository and downloaded from there
• All packages should have (if available) signatures/hashes verified
• Add an entry to the CHANGELOG.md file
• Integration tests ran successfully

TIP: Running integration tests with a new product image

The image can be built and uploaded to the kind cluster with the following commands:

boil build <IMAGE> --image-version <RELEASE_VERSION> --strip-architecture --load
kind load docker-image <MANIFEST_URI> --name=<name-of-your-test-cluster>

See the output of boil to retrieve the image manifest URI for <MANIFEST_URI>.

[maltesander]

Tested and works. This is very cumbersome to check and maintain.

Can we split the AIRFLOW_EXTRAS parameters to be closer to the actual docs like:

• AIRFLOW_EXTRA_CORE
• AIRFLOW_EXTRA_META
• AIRFLOW_EXTRA_PROVIDER_APACHE
• AIRFLOW_EXTRA_PROVIDER_EXTERNAL_SERVICES
• AIRFLOW_EXTRA_LOCALLY_INSTALLED_SOFTWARE
• AIRFLOW_EXTRA_OTHER

I think this would already help to make it easier. This could be concatenated in the docker image into AIRFLOW_EXTRA.

[adwk67]

Tested and works. This is very cumbersome to check and maintain.

Can we split the AIRFLOW_EXTRAS parameters to be closer to the actual docs like:

• AIRFLOW_EXTRA_CORE
• AIRFLOW_EXTRA_META
• AIRFLOW_EXTRA_PROVIDER_APACHE
• AIRFLOW_EXTRA_PROVIDER_EXTERNAL_SERVICES
• AIRFLOW_EXTRA_LOCALLY_INSTALLED_SOFTWARE
• AIRFLOW_EXTRA_OTHER

I think this would already help to make it easier. This could be concatenated in the docker image into AIRFLOW_EXTRA.

Done in 4b1d932

Built 3.0.6 and 3.0.1 and tested with smoke tests:

--- PASS: kuttl (443.57s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/external-access_airflow-3.0.1_openshift-false_executor-celery (232.51s)
        --- PASS: kuttl/harness/external-access_airflow-3.0.6_openshift-false_executor-celery (235.56s)
        --- PASS: kuttl/harness/external-access_airflow-3.0.6_openshift-false_executor-kubernetes (201.50s)
        --- PASS: kuttl/harness/external-access_airflow-3.0.1_openshift-false_executor-kubernetes (206.91s)
PASS

I missed one provider - tabular - which has been added :)