Can't use default_replication_method INCREMENTAL with tap-postgres
See original GitHub issueWith meltano 2.0.3, if I have default_replication_method set as INCREMENTAL with tap-postgres then I get an error.
meltano config tap-postgres test
2022-06-24T15:26:15.923809Z [info ] Environment 'dev' is active
Plugin configuration is invalid
AttributeError: 'NoneType' object has no attribute 'get'
Changing default_replication_method to either FULL_TABLE or LOG_BASED gets past the error and returns “Plugin configuration is valid”
meltano.yml
version: 1
default_environment: dev
project_id: fc0cd1d8-5717-4675-9102-c660d25017d5
plugins:
extractors:
- name: tap-postgres
variant: transferwise
pip_url: pipelinewise-tap-postgres
loaders:
- name: target-postgres
variant: transferwise
pip_url: pipelinewise-target-postgres
environments:
- name: dev
config:
plugins:
extractors:
- name: tap-postgres
config:
user: meltano_test_user
dbname: meltano_test
filter_schema: src_sample_store
start_date: '2022-06-24T00:00:00Z'
default_replication_method: INCREMENTAL
metadata:
meltano_test-src_sample_store:
replication-key: updated_at
replication-method: INCREMENTAL
loaders:
- name: target-postgres
config:
user: meltano_test_user
dbname: meltano_test_dw
schema_mapping:
src_sample_store: dest_sample_store
- name: staging
- name: prod
Issue Analytics
- State:
- Created a year ago
- Comments:5 (1 by maintainers)
Top Results From Across the Web
pipelinewise-tap-postgres/README.md at master - GitHub
The replication method can be one of FULL_TABLE , INCREMENTAL or LOG_BASED . Note: Log based replication requires a few adjustments in the...
Read more >Meltano elt tab-postgres target-postgres (full table replication ...
I am using the Full table replication method, do I need to write the ... Default transferwise variants of tap-postgres and tap-mysql are ......
Read more >tap-postgres - Meltano Hub
The prefered replication method. Note: Key-based incremental ( INCREMENTAL ) requires configuring a replication_key column within the catalog's stream ...
Read more >Tap PostgreSQL — PipelineWise documentation - GitHub Pages
ALTER DEFAULT PRIVILEGES IN SCHEMA <schema_name> GRANT SELECT ON TABLES TO ... This step is only required if you use Log Based replication...
Read more >PostgreSQL (v1) | Stitch Documentation
Connect and replicate data from your PostgreSQL database using Stitch's ... This integration is powered by Singer's Postgres tap and certified by Stitch....
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
We found out in Slack that this is a duplicate of https://github.com/meltano/meltano/issues/3322. Moving the
metadata
out ofenvironments
and to the top-leveltap-postgres
fixed the issue!Duplicate of https://github.com/meltano/meltano/issues/3322