TypeError: not enough arguments for format string

I believe there’s some unexpected behavior that happens when using the Database.execute_sql command with an SQL statement but no parameters. The above error occurs in the pymysql project, but it’s actually a problem with peewee (in my opinion).

When you run the following code:

sql = "INSERT INTO some_table (column_a, column_b) VALUES ('hello_my_name_is_inigo', 'odd_%_percent')
database.execute_sql(sql) # or database.execute_sql(sql, params=None)

you receive an error that looks like this:

Traceback (most recent call last):
    ...
    models.main_database.execute_sql(sql)
  File "/home/kyle/.local/lib/python3.7/site-packages/peewee.py", line 3057, in execute_sql
    cursor.execute(sql, params or ())
  File "/home/kyle/.local/lib/python3.7/site-packages/pymysql/cursors.py", line 171, in execute
    query = self.mogrify(query, args)
  File "/home/kyle/.local/lib/python3.7/site-packages/pymysql/cursors.py", line 150, in mogrify
    query = query % self._escape_args(args, conn)
TypeError: not enough arguments for format string

• Quick note: Anyone reading this can fix the error by ‘escaping’ the % sign in your SQL statement, from ‘%’ to ‘%%’

Technically, this happens because there’s a percent sign in the SQL statement that needs to be escaped so the line query = query % self._escape_args(args, conn) in pymysql can run. However, this puts the burden on the user to figure that out.

I think the real culprit is the params argument in database.execute_sql which doesn’t seem to be passed correctly into cursor.execute at line 3057 in peewee.py. In other words, the line cursor.execute(sql, params or ()) seems to be the problem. I think we should consider changing the line to this:

cursor.execute(sql, args=params or None)

which is more in line with pymysql’s documentation, seen here.

To give some more context, the line failing in pymysql (within cursor.execute) looks like this:

# "pymysql/cursors.py", lines 149-150

if args is not None:
    query = query % self._escape_args(args, conn)

So passing in params as an empty tuple really just creates more trouble then it’s worth. It should stay as None.

It would be different if I was passing in an SQL statement with the expectation that it was a string going to be formatted. Then the error TypeError: not enough arguments for format string would make sense, and I could debug it easily. However, I believe passing in only an SQL statement should just execute that SQL statement, without needing to think about formatting or params from cursor.execute.

Maybe I’m missing something, but is there a reason why params is defaulted to an empty tuple in the first place?

Finally, this issue isn’t just prevalent here, but also Django, SQLAlchemy, and although correct behavior for the package, pymysql itself. So I think it’s definitely something that should be fixed as I’m sure more people are having trouble with it.

If anything, we should at least consider updating peewee’s documentation for execute_sql, seen here to better explain how formatting works, like within pymysql’s documentation here. Then the error might make more sense after looking it up.

Thanks

[coleifer]

Passing params=None does not work with psycopg2.

[kelmore5]

Hm, well that’s fair-ish. Really I brought it up because it seemed like other people were having trouble with it.

Passing params=None does not work with psycopg2.

And as long as there’s a reason for it, I’m satisfied. Hopefully laying out the issue will help people in the future.