Diff between 2 PostgreSQL databases returns nothing

Hi,

I am using the Dockerfile to build and use geodiff with the PostgreSQL driver enabled, with the following steps:

# Clone repository
git clone [e-mail]:MerginMaps/mergin-db-sync.git
cd mergin-db-sync

# Build docker image
docker build -t mergin_image .

# Run the drivers command
docker run -it --rm --network="host" --volume /home/mdouchin/:/home/mdouchin/ --name=mergin_container mergin_image /geodiff/build/geodiff drivers
# returns
sqlite
postgres

NB: I created a bash alias to run the docker command with the simple geodiff

Test data:

# Create the source database
dropdb geodiff_source
createdb geodiff_source
psql -d geodiff_source -c "CREATE EXTENSION postgis"

# Fill it with test data
psql -d geodiff_source -c "CREATE SCHEMA eee"
psql -d geodiff_source -c "CREATE TABLE eee.points (id serial primary key, p_name text, geom geometry(point, 2154))"
psql -d geodiff_source -c "INSERT INTO eee.points (p_name, geom) VALUES ('first', ST_MakePoint(0, 1))"
psql -d geodiff_source -c "INSERT INTO eee.points (p_name, geom) VALUES ('second', ST_MakePoint(5, 10))"

# Create the second database
dropdb geodiff_target
createdb geodiff_target
psql -d geodiff_target -c "CREATE EXTENSION postgis"

I successfully ran PostgreSQL related commands, for example:

# Get the source database schema as JSON
geodiff schema --driver 'postgres' "host=localhost port=5432 user=mdouchin password=**** dbname=geodiff_source" eee schema_pg.json

# create a copy of a PostgreSQL database "geodiff", schema "eee" to another database "geodiff_target", schema "eee"
geodiff copy --driver-1 'postgres' "host=localhost port=5432 user=mdouchin password=**** dbname=geodiff_source" --driver-2 'postgres' "host=localhost port=5432 user=mdouchin password=**** dbname=geodiff_target" eee eee

# This works well: the target database has been populated with the eee schema and the points table
md ~ $ psql -d geodiff_target -c "SELECT * FROM eee.points"
 id | p_name |                        geom                        
----+--------+----------------------------------------------------
  1 | first  | 01010000206A0800000000000000000000000000000000F03F
  2 | second | 01010000206A08000000000000000014400000000000002440
(2 lignes)

Then I inserted one more feature in the unique table eee.points in the geodiff database, and try to create a diff between the two databases.

# Insert one more feature in the source database
psql -d geodiff_source -c "INSERT INTO eee.points (p_name, geom) VALUES ('third', ST_MakePoint(3, 5))"

# Get the diff
geodiff diff --json --driver-1 'postgres' "host=localhost port=5432 user=mdouchin password=**** dbname=geodiff_source" --driver-2 'postgres' "host=localhost port=5432 user=mdouchin password=**** dbname=geodiff_target" eee eee
# Returns
{
  "geodiff": []
}

The returned JSON is “empty”.

I also tried with the same source database, but with a SQLite database as the target database, and it worked: the diff corresponded to the expected differences between the 2 databases:

# Copy the eee schema of the source PostgreSQL database to the GeoPackage geodiff_target.gpkg
geodiff copy --driver-1 'postgres' "host=localhost port=5432 user=mdouchin password=**** dbname=geodiff_source" --driver-2 'sqlite' "" eee /home/mdouchin/geodiff_target.gpkg

# The GPKG data feels good
sqlite3 geodiff_target.gpkg
sqlite> SELECT * FROM points;
1|first|GP
2|second|GP
3|third|GP
exit

# Insert a new line in the source points table
psql -d geodiff_source -c "INSERT INTO eee.points (p_name, geom) VALUES ('fourth', ST_MakePoint(10, 2))"

# Check the diff
geodiff diff --json --driver-1 'postgres' "host=localhost port=5432 user=mdouchin password=**** dbname=geodiff_source" --driver-2 'sqlite' "" eee /home/mdouchin/geodiff_target.gpkg 

which returns

{
  "geodiff": [
    {
      "changes": [
        {
          "column": 0,
          "old": 4
        },
        {
          "column": 1,
          "old": "fourth"
        },
        {
          "column": 2,
          "old": "R1AAAWoIAAABAQAAAAAAAAAAACRAAAAAAAAAAEA="
        }
      ],
      "table": "points",
      "type": "delete"
    }
  ]
}

I was wondering if it was possible to use geodiff for a PostgreSQL/PostgreSQL diff tool, or is it more “field oriented” and designed for GeoPackage as the pivot format ?

Thanks a lot for your great work on this tool !

[PeterPetrik]

Thanks, it is still a bug so lets keep this one open

[wonder-sk]

Do you think geodiff could provide a “conflict” merging system, for example to merge all the conflicts which have happened between 2 dates ? For example, if you have synchronized 10 times between the 2 databases, and once per week/month, you need to have a look at the conflicts, but in a simple way. It would be handy to read one single conflict file instead of possibly 3 or 4. I could propose a PR for this “conflict merging tool”, even if my C++ coding skills are a bit rusty… I could also propose a Python QGIS processing alg which will do the merge and create a vectory layer to visualize the conflicts ?

With this conflict merging system, do you mean a GUI which would show all the recorded conflicts (on a map or a table or both) and it would allow inspection of each conflict, with some options to change the conflict resolution? There are certainly more ways how to do that - probably the easiest would be a QGIS processing alg as you mentioned, or it could be a complete GUI within QGIS (the Mergin plugin already comes with a GUI to compare changes - see https://github.com/MerginMaps/qgis-mergin-plugin/pull/397 - only missing the bits related to conflict resolution).