use singer taps and targets programatically
See original GitHub issueHi, I’m participating in development of dataflows which has similar goals to your projects, and we would like to be able to integrate between the libraries - use singer taps / targets inside a data flow, and use a data flow as a singer tap / target (https://github.com/datahq/dataflows/issues/16)
To enable this integration we need to be able to call singer taps / targets from Python code, this is easy to do using subprocess.Popen, see example here
I think it would really useful to have this in a more standard way as part of the singer-python library.
Example: singer.read_tap
Install the tap: pip install tap-exchangeratesapi
Read from the tap:
>>> tap = singer.read_tap('exchangeratesapi', {"base": "ILS", "start_date": "2018-10-01"})
>>> for message in tap:
>>> print(message) # SchemaMessage / RecordMessage / StateMessage
Issue Analytics
- State:
- Created 5 years ago
- Comments:5 (5 by maintainers)
Top Results From Across the Web
Please Dont Build Your Data Pipeline using Singer
Run any Singer tap or target; Provide a UI for configuring and running those taps and targets; Count the number of records synced...
Read more >Anybody have experience creating singer taps and targets?
Hey I do have some experience working with Singer taps and targets, I can try answering if you have any queries!
Read more >How to Use Singer for Your ETL Needs: Step-by-Step Tutorial
I'll demonstrate how you can use Singer for your ETL requirements through a practical ... Singer has two types of scripts—taps and targets....
Read more >Singer SDK: New framework for building Singer taps - Meltano
The goal is to work together on building the framework and supporting tooling and documentation that you will want to use for your...
Read more >Singer Spec - Meltano Hub
Since taps and targets are able to communicate to each other via the ... Within the Singer Spec, JSON schema definitions are used...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
I’m going to close this for now but please feel free to reopen it if you’re interested in exploring it via a PR.
I’ll close this out for now since we don’t have much forward momentum on the PR for now.