Upsert support

I’d like to consolidate the discussions in #942 and #1067 as they both concern extensions to basic INSERT and UPDATE queries. In this ticket I’ll describe the various features and syntaxes Peewee will need to support – please let me know if I’ve missed something!

---

SQLite

Docs.

• INSERT OR <REPLACE|ROLLBACK|ABORT|FAIL|IGNORE> INTO ...
• REPLACE INTO ... (as a shortcut for INSERT OR REPLACE).

These extra clauses are referred to in the docs as the ON CONFLICT CLAUSE. We are only interested in the IGNORE and REPLACE strategies. Here’s how REPLACE works:

When a UNIQUE or PRIMARY KEY constraint violation occurs, the REPLACE algorithm deletes pre-existing rows that are causing the constraint violation prior to inserting or updating the current row and the command continues executing normally. If a NOT NULL constraint violation occurs, the REPLACE conflict resolution replaces the NULL value with the default value for that column, or if the column has no default value, then the ABORT algorithm is used. If a CHECK constraint violation occurs, the REPLACE conflict resolution algorithm always works like ABORT.

MySQL

Docs.

• INSERT IGNORE
• REPLACE INTO ...
• INSERT INTO ... ON DUPLICATE KEY UPDATE ... The last one is different from SQLite as it allows the user to specify a different query in the event of a conflict. Here’s an example:

INSERT INTO table (a,b,c) VALUES (1,2,3)
  ON DUPLICATE KEY UPDATE c=c+1;

No special syntax seems to be needed to reference the column value of the conflicting row. To reference the value from the INSERT portion of the query you can use the VALUES() function. I believe the following would be equivalent to a REPLACE if the only constraint on the table was on column a:

INSERT INTO table (a,b,c) VALUES (1, 2, 3)
   ON DUPLICATE KEY UPDATE b = VALUES(b), c = VALUES(c)

Postgresql

Docs.

• INSERT INTO ... ON CONFLICT DO NOTHING
• INSERT INTO ... ON CONFLICT (conflict_target) DO UPDATE SET ...

Postgresql’s upsert looks to be the most sophisticated, as you need to specify a conflict target to do an update. The conflict target description in the docs is very confusing but it seems to work fine if you just pass in the list of columns that have the appropriate constraint. You can also reference the value of the conflicting row using EXCLUDED.

Examples:

INSERT INTO distributors (did, dname)
    VALUES (5, 'Gizmo Transglobal'), (6, 'Associated Computing, Inc')
    ON CONFLICT (did) DO UPDATE SET dname = EXCLUDED.dname;

-- Don't update existing distributors based in a certain ZIP code
INSERT INTO distributors AS d (did, dname) VALUES (8, 'Anvil Distribution')
    ON CONFLICT (did) DO UPDATE
    SET dname = EXCLUDED.dname || ' (formerly ' || d.dname || ')'
    WHERE d.zipcode <> '21201';

-- Name a constraint directly in the statement (uses associated
-- index to arbitrate taking the DO NOTHING action)
INSERT INTO distributors (did, dname) VALUES (9, 'Antwerp Design')
    ON CONFLICT ON CONSTRAINT distributors_pkey DO NOTHING;

-- This statement could infer a partial unique index on "did"
-- with a predicate of "WHERE is_active", but it could also
-- just use a regular unique constraint on "did"
INSERT INTO distributors (did, dname) VALUES (10, 'Conrad International')
    ON CONFLICT (did) WHERE is_active DO NOTHING;

[coleifer]

Peewee would use:

Event.insert_many(event_list).on_conflict('ignore')

Assuming event_list was a list of dictionaries.

[willyhakim]

This project is so amazing! @coleifer Just wanted to say that you’re amazing. One the best API designs I’ve seen. Thanks