Definitions created using 'az synapse spark-job-definition create' do not show in the Workspace UI with Git Integration enabled
See original GitHub issueRelated command
az synapse spark-job-definition create --workspace-name <workspace> --name <job-name> --file @"folder\sparkjob.json"
Describe the bug When using ‘az synapse spark-job-definition create’ command I am able to create a sparkjob definition, however that definition is not then visible in the Synapse Workspace UI. I am however able to see it when calling ‘az synapse spark-job-definition list’
I have Git integration enabled on that workspace, checked all the branches and can’t find the job definition I’ve created in the UI.
To Reproduce
- Export existing job definition from Synapse UI
- Change jobProperties.name to be unique.
- Run `az synapse spark-job-definition create --workspace-name <workspace> --name <job-name> --file @“folder\sparkjob.json” using the previously exported file.
- Run
az synapse spark-job-definition list --workspace-name <workspace>
and verify newly create job definition is visible. - Check UI.
Expected behavior The created spark job definition is also visible in Synapse Workspace UI
Environment summary
Windows-10-10.0.19044-SP0
Python 3.10.5
Installer: MSI
azure-cli 2.38.0
Extensions:
azure-cli-ml 1.41.0
Dependencies:
msal 1.18.0b1
azure-mgmt-resource 21.1.0b1
Additional context
Issue Analytics
- State:
- Created a year ago
- Comments:16 (8 by maintainers)
Top Results From Across the Web
az synapse spark-job-definition - Microsoft Learn
Manage spark job definitions in a synapse workspace. In this article. Commands; az synapse spark-job-definition create; az synapse spark-job-definition delete ...
Read more >Azure Synapse Analytics - Git Integration and CI/CD
Azure Synapse Analytics is a data services tool that provides data integration, warehousing, and analytical functionality.
Read more >Synapse – Choosing Between Spark Notebook vs Spark Job ...
Introduction Apache Spark applications are used by businesses to perform ... You can integrate spark job definition in the Synapse pipeline.
Read more >azurerm_synapse_workspace | Resources | hashicorp/azurerm
Example Usage - creating a workspace with Customer Managed Key and Azure AD ... the name of the Resource Group where the synapse...
Read more >Access issue with Azure Synapse studio - Stack Overflow
This could be an intermittent issue while opening synapse workspace. Could you please confirm the permission on the Synapse workspace which ...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Hi @alexszym, I tried to use cli to create other artifacts like SQL script with git enable and there is the same issue as you mentioned above. But something like spark pools that are managed by “azure-mgmt-synapse” module are good. So, I think this issue is related to git CI/CD background.
@alexszym Let me engage the synapse engineering team (CLI), before we resort to support.