Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Can't use default_replication_method INCREMENTAL with tap-postgres

See original GitHub issue

With meltano 2.0.3, if I have default_replication_method set as INCREMENTAL with tap-postgres then I get an error.

meltano config tap-postgres test

2022-06-24T15:26:15.923809Z [info     ] Environment 'dev' is active

Plugin configuration is invalid

AttributeError: 'NoneType' object has no attribute 'get'

Changing default_replication_method to either FULL_TABLE or LOG_BASED gets past the error and returns “Plugin configuration is valid”

meltano.yml

version: 1
default_environment: dev
project_id: fc0cd1d8-5717-4675-9102-c660d25017d5
plugins:
  extractors:
  - name: tap-postgres
    variant: transferwise
    pip_url: pipelinewise-tap-postgres
  loaders:
  - name: target-postgres
    variant: transferwise
    pip_url: pipelinewise-target-postgres
environments:
- name: dev
  config:
    plugins:
      extractors:
      - name: tap-postgres
        config:
          user: meltano_test_user
          dbname: meltano_test
          filter_schema: src_sample_store
          start_date: '2022-06-24T00:00:00Z'
          default_replication_method: INCREMENTAL 
          metadata:
            meltano_test-src_sample_store:
              replication-key: updated_at
              replication-method: INCREMENTAL
      loaders:
      - name: target-postgres
        config:
          user: meltano_test_user
          dbname: meltano_test_dw
          schema_mapping:
            src_sample_store: dest_sample_store
- name: staging
- name: prod

Issue Analytics

State:
Created a year ago
Comments:5 (1 by maintainers)

Top GitHub Comments

2reactions

DouweMcommented, Jun 24, 2022

We found out in Slack that this is a duplicate of https://github.com/meltano/meltano/issues/3322. Moving the metadata out of environments and to the top-level tap-postgres fixed the issue!

0reactions

tayloramurphycommented, Jun 24, 2022

Duplicate of https://github.com/meltano/meltano/issues/3322

Top Results From Across the Web

pipelinewise-tap-postgres/README.md at master - GitHub

The replication method can be one of FULL_TABLE , INCREMENTAL or LOG_BASED . Note: Log based replication requires a few adjustments in the...

Meltano elt tab-postgres target-postgres (full table replication ...

I am using the Full table replication method, do I need to write the ... Default transferwise variants of tap-postgres and tap-mysql are ......

tap-postgres - Meltano Hub

The prefered replication method. Note: Key-based incremental ( INCREMENTAL ) requires configuring a replication_key column within the catalog's stream ...

Tap PostgreSQL — PipelineWise documentation - GitHub Pages

ALTER DEFAULT PRIVILEGES IN SCHEMA <schema_name> GRANT SELECT ON TABLES TO ... This step is only required if you use Log Based replication...

PostgreSQL (v1) | Stitch Documentation

Connect and replicate data from your PostgreSQL database using Stitch's ... This integration is powered by Singer's Postgres tap and certified by Stitch....