Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Additional named step automatically added to the end of the pipeline
See original GitHub issueWhen withName("BigQuerySink").saveAsTypedBigQuery(...) transform is used as the last step of the pipeline, an additional step is automatically added at the end of the it. Inspecting the job through Dataflow UI gives named steps that look something like this:
... --> BigQuerySink --> saveAsTypedBigQuery@{FileName.scala:82}1
I asked on gitter whether there is a way to name the last step as well and was asked to open an issue. I’m interested in doing so because every time the withName("BigQuerySink").saveAsTypedBigQuery(...) line is moved, the name of the last step changes and I end up having trouble updating the job on the fly.
Issue Analytics
- State:
- Created 6 years ago
- Reactions:1
- Comments:8 (8 by maintainers)
Top Results From Across the Web
Top Related Medium Post
Top Related StackOverflow Question
Troubleshoot Live Code
Top Related Reddit Thread
Top Related Hackernoon Post
Top Related Tweet
Top Related Dev.to Post
Top Related Hashnode Post
I’m interested in looking into this.
@andrewsmartin I’ve opened a PR (also tested the fix by submitting a dataflow-runner job and it looks alright). Could you explain why another name is not automatically set here? Thanks.