Cannot store BLOB fields with peewee and PyMySQL

Hi,

I have the problem, that I cannot store Models having a BlobField with binary content since upgrading to the latest version of peewee 5aea8f6fa405c896884c71262e0b2ac04d8cdaf9. I am using a MySQL database with peewee over PyMySQL b2ec8287151f8dc54e71e426b41a1b300fb934bb.

How to reproduce:

1. Install peewee and PyMySQL versions as mentioned above
2. Create a MySQL databse with a table containing a BLOB field
3. Create an appropriate peewee.Model
4. Create a new Model instance
5. Set the BlobField to a non-text binary (e.g. a picture file)
6. Try to store the model with model.save()
7. Get UnicodeEncodeError exceptions like:

'utf-8' codec can't encode character '\udcff' in position 159: surrogates not allowed in <class 'homie.mods.openimmodb.openimmodb.Anhaenge'>
'utf-8' codec can't encode character '\udcff' in position 160: surrogates not allowed in <class 'homie.mods.openimmodb.openimmodb.Anhaenge'>
'utf-8' codec can't encode character '\udcff' in position 161: surrogates not allowed in <class 'homie.mods.openimmodb.openimmodb.Anhaenge'>
'utf-8' codec can't encode character '\udc89' in position 163: surrogates not allowed in <class 'homie.mods.openimmodb.openimmodb.Anhaenge'>
'utf-8' codec can't encode character '\udcc7' in position 185: surrogates not allowed in <class 'homie.mods.openimmodb.openimmodb.Anhaenge'>

Thanks for looking into it.

PS: peewee does not seem compatible with the surrogates escapes introduced in PyMySQL 5157c9b7d058f764695a35684b599e0cbac22d9c : https://github.com/PyMySQL/PyMySQL/commit/5157c9b7d058f764695a35684b599e0cbac22d9c

Since this was a fix in PyMySQL, I assume, that peewee needs to correct for compatibility here, or am I wrong?

[coleifer]

I think the best path forward will be to subclass BlobField in your app and use custom logic for the db_value() function:

from peewee import *
from pymysql import Binary

class MyBlobField(BlobField):
    def db_value(self, value):
        if value is not None:
            return Binary(value)

[coleifer]

I think that peewee handles this datatype correctly. According to PEP 249, the db-api 2.0 spec:

The preferred object type for Binary objects are the buffer types available in standard Python starting with version 1.5.2. Please see the Python documentation for details. For information about the C interface have a look at Include/bufferobject.h and Objects/bufferobject.c in the Python source distribution.

When using a BlobField, peewee uses buffer in Python2 and bytes for Python3.