Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Migrate Google example DAGs to new design
See original GitHub issueThere is a new design of system tests that was introduced by the AIP-47.
All current example dags need to be migrated and converted into system tests, so they can be run in the CI process automatically before releases.
This is an aggregated issue for all example DAGs related to Google provider. It is created to track progress of their migration.
List of paths to example DAGs:
- airflow/providers/google/leveldb/example_dags/example_leveldb.py
- airflow/providers/google/firebase/example_dags/example_firestore.py
- airflow/providers/google/ads/example_dags/example_ads.py
- airflow/providers/google/suite/example_dags/example_sql_to_sheets.py
- airflow/providers/google/suite/example_dags/example_gcs_to_sheets.py
- airflow/providers/google/suite/example_dags/example_gcs_to_gdrive.py
- airflow/providers/google/suite/example_dags/example_sheets.py
- airflow/providers/google/marketing_platform/example_dags/example_analytics.py
- airflow/providers/google/marketing_platform/example_dags/example_display_video.py
- airflow/providers/google/marketing_platform/example_dags/example_campaign_manager.py
- airflow/providers/google/marketing_platform/example_dags/example_search_ads.py
- airflow/providers/google/cloud/example_dags/example_bigquery_queries.py
- airflow/providers/google/cloud/example_dags/example_gcs_to_sftp.py
- airflow/providers/google/cloud/example_dags/example_oracle_to_gcs.py
- airflow/providers/google/cloud/example_dags/example_compute_igm.py
- airflow/providers/google/cloud/example_dags/example_automl_vision_classification.py
- airflow/providers/google/cloud/example_dags/example_datacatalog.py
- airflow/providers/google/cloud/example_dags/example_bigquery_to_mssql.py
- airflow/providers/google/cloud/example_dags/example_mssql_to_gcs.py
- airflow/providers/google/cloud/example_dags/example_mysql_to_gcs.py
- airflow/providers/google/cloud/example_dags/example_bigquery_to_bigquery.py
- airflow/providers/google/cloud/example_dags/example_natural_language.py
- airflow/providers/google/cloud/example_dags/example_life_sciences.py
- airflow/providers/google/cloud/example_dags/example_automl_nl_text_extraction.py
- airflow/providers/google/cloud/example_dags/example_automl_video_intelligence_classification.py
- airflow/providers/google/cloud/example_dags/example_trino_to_gcs.py
- airflow/providers/google/cloud/example_dags/example_bigquery_transfer.py
- airflow/providers/google/cloud/example_dags/example_cloud_storage_transfer_service_gcp.py
- airflow/providers/google/cloud/example_dags/example_translate.py
- airflow/providers/google/cloud/example_dags/example_local_to_gcs.py
- airflow/providers/google/cloud/example_dags/example_compute_ssh.py
- airflow/providers/google/cloud/example_dags/example_dataflow_sql.py
- airflow/providers/google/cloud/example_dags/example_text_to_speech.py
- airflow/providers/google/cloud/example_dags/example_presto_to_gcs.py
- airflow/providers/google/cloud/example_dags/example_looker.py
- airflow/providers/google/cloud/example_dags/example_sftp_to_gcs.py
- airflow/providers/google/cloud/example_dags/example_facebook_ads_to_gcs.py
- airflow/providers/google/cloud/example_dags/example_gcs_to_bigquery.py
- airflow/providers/google/cloud/example_dags/example_postgres_to_gcs.py
- airflow/providers/google/cloud/example_dags/example_spanner.py
- airflow/providers/google/cloud/example_dags/example_gdrive_to_gcs.py
- airflow/providers/google/cloud/example_dags/example_dlp.py
- airflow/providers/google/cloud/example_dags/example_bigquery_dts.py
- airflow/providers/google/cloud/example_dags/example_datastore.py
- airflow/providers/google/cloud/example_dags/example_bigquery_operations.py
- airflow/providers/google/cloud/example_dags/example_automl_nl_text_classification.py
- airflow/providers/google/cloud/example_dags/example_bigquery_to_gcs.py
- airflow/providers/google/cloud/example_dags/example_automl_vision_object_detection.py
- airflow/providers/google/cloud/example_dags/example_bigtable.py
- airflow/providers/google/cloud/example_dags/example_gcs_to_local.py
- airflow/providers/google/cloud/example_dags/example_vertex_ai.py
- airflow/providers/google/cloud/example_dags/example_automl_video_intelligence_tracking.py
- airflow/providers/google/cloud/example_dags/example_gcs_timespan_file_transform.py
- airflow/providers/google/cloud/example_dags/example_dataproc_metastore.py
- airflow/providers/google/cloud/example_dags/example_salesforce_to_gcs.py
- airflow/providers/google/cloud/example_dags/example_functions.py
- airflow/providers/google/cloud/example_dags/example_compute.py
- airflow/providers/google/cloud/example_dags/example_sheets_to_gcs.py
- airflow/providers/google/cloud/example_dags/example_dataflow.py
- airflow/providers/google/cloud/example_dags/example_datafusion.py
- airflow/providers/google/cloud/example_dags/example_cloud_sql_query.py
- airflow/providers/google/cloud/example_dags/example_translate_speech.py
- airflow/providers/google/cloud/example_dags/example_automl_tables.py
- airflow/providers/google/cloud/example_dags/example_cloud_memorystore.py
- airflow/providers/google/cloud/example_dags/example_bigquery_sensors.py
- airflow/providers/google/cloud/example_dags/example_gcs.py
- airflow/providers/google/cloud/example_dags/example_mlengine.py
- airflow/providers/google/cloud/example_dags/example_dataprep.py
- airflow/providers/google/cloud/example_dags/example_cloud_storage_transfer_service_aws.py
- airflow/providers/google/cloud/example_dags/example_pubsub.py
- airflow/providers/google/cloud/example_dags/example_gcs_to_gcs.py
- airflow/providers/google/cloud/example_dags/example_s3_to_gcs.py
- airflow/providers/google/cloud/example_dags/example_gdrive_to_local.py
- airflow/providers/google/cloud/example_dags/example_azure_fileshare_to_gcs.py
- airflow/providers/google/cloud/example_dags/example_workflows.py
- airflow/providers/google/cloud/example_dags/example_kubernetes_engine.py
- airflow/providers/google/cloud/example_dags/example_dataproc.py
- airflow/providers/google/cloud/example_dags/example_speech_to_text.py
- airflow/providers/google/cloud/example_dags/example_cloud_composer.py
- airflow/providers/google/cloud/example_dags/example_automl_translation.py
- airflow/providers/google/cloud/example_dags/example_stackdriver.py
- airflow/providers/google/cloud/example_dags/example_tasks.py
- airflow/providers/google/cloud/example_dags/example_calendar_to_gcs.py
- airflow/providers/google/cloud/example_dags/example_video_intelligence.py
- airflow/providers/google/cloud/example_dags/example_cloud_build.py
- airflow/providers/google/cloud/example_dags/example_dataflow_flex_template.py
- airflow/providers/google/cloud/example_dags/example_automl_nl_text_sentiment.py
- airflow/providers/google/cloud/example_dags/example_cloud_sql.py
- airflow/providers/google/cloud/example_dags/example_vision.py
In progress:
- airflow/providers/google/cloud/example_dags/example_bigquery_to_bigquery.py
- airflow/providers/google/cloud/example_dags/example_dataflow.py
- airflow/providers/google/cloud/example_dags/example_datafusion.py
- airflow/providers/google/cloud/example_dags/example_dataproc.py
- airflow/providers/google/cloud/example_dags/example_dataproc_metastore.py
- airflow/providers/google/cloud/example_dags/example_datastore.py
- airflow/providers/google/cloud/example_dags/example_gcs.py
- airflow/providers/google/cloud/example_dags/example_gcs_timespan_file_transform.py
- airflow/providers/google/cloud/example_dags/example_gcs_to_gcs.py
- airflow/providers/google/cloud/example_dags/example_gcs_to_local.py
- airflow/providers/google/cloud/example_dags/example_local_to_gcs.py
Issue Analytics
- State:
- Created a year ago
- Comments:5 (5 by maintainers)
Top Results From Across the Web
Migrate Databricks example DAGs to new design · Issue #22442
This is an aggregated issue for all example DAGs related to Databricks provider. It is created to track progress of their migration. List...
Read more >Add and update DAGs | Cloud Composer
Updating DAGs that run on a frequent schedule · Open the Logs tab for your environment in Google Cloud console. · Go to...
Read more >[airflow] branch main updated: Migrate Google example DAG ...
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository ...
Read more >Package apache-airflow-providers-google
Migrate Google example DAG mssql_to_gcs to new design AIP-47 (#24541). 46ac083f7e. 2022-07-01. Align Black and blacken-docs configs (#24785).
Read more >AIP-47 - Migrate google marketing DAGs to new design #22447 ...
`source code <https://github.com/apache/airflow/tree/main/airflow/providers/google/marketing_platform/example_dags/>`_ of the particular example DAGs.
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Updated
Sure. I merge all the green ones and re-run the script to update the checklists