How to use a separate database for testing?

I am trying to integrate Pony with Flask-Testing (unsuccessfully, so far–see #25 )–and I’m wondering how I can use a separate database for testing. In Flask-SQLAlchemy, for example, this would be achieved by setting the SQLALCHEMY_DATABASE_URI config option in the test setup

from flask.ext.testing import TestCase
from app import app
class TestApp(TestCase):
    TESTING = True
    DEBUG = True
    SQLALCHEMY_DATABASE_URI = 'sqlite:///tmp/test.db'

    def create_app(self):
        app.config.from_object(self)
        return app

I am not sure how to achieve this with Pony, since each Entity class must be declared from a Database that is already configured.

[kozlovsky]

Hi Steven!

At this point we don’t have a preferred way for testing Pony-powered applications. I want to describe the way, which seems most convenient to me. If your have some comments or suggestions, don’t hesitate to tell. It would be good if other developers contribute to the discussion and tell their opinion about the best testing practices and what is missing in Pony in this regard.

1. How to separate database and entity creation from the module import

Although Pony standard examples demonstrate creation of the Database instance as a module variable, it may be convenient to create the Database instance along with the corresponding entities inside a function. Something like this:

def define_database_and_entities(**db_params):
    db = Database(**db_params)

    class Person(db.Entity):
        name = Required(unicode)
        pets = Set('Pet')

    class Pet(dn.Entity):
        name = Required(unicode)
        kind = Required(unicode)
        owner = Required('Person')

    db.generate_mapping(create_tables=True)
    return db

This way you can create Database and entities multiple times with different config parameters. After an entity is defined, it can be accessed as a database attribute with the same name:

db1 = define_database_and_entities(provider='sqlite', filename='/tmp/test.db', create_db=True)
select(p.name for p in db1.Person)

db2 = define_database_and entities(provider='postgres', user='usr', password='***',
                      host='localhost', database='pets')
select(p.name for p in db2.Person)

2. How to separate database parameters from entity definitions

In some situations it may be convenient to separate entity definitions and mapping generation. To do this, just pass the database as a parameter to the function:

def define_entities(db):
    class Person(db.Entity):
        name = Required(unicode)
        pets = Set('Pet')

    class Pet(dn.Entity):
        name = Required(unicode)
        kind = Required(unicode)
        owner = Required('Person')

def define_database(**db_params):
    db = Database(**db_params)
    define_entities(db)
    db.generate_mapping(create_tables=True)
    return db

db = define_database(provider='sqlite', filename='/tmp/test.db', create_db=True)
select(p for p in db.Person)

3. How to fill the database with test data

One of the ways to do this is to define a separate function, like this:

@db_session
def populate_database(db):
    person1 = db.Person(name='John')
    person2 = db.Person(name='Mike')
    db.Pet(name='Napoleon', kind='hamster', owner=person1)
    db.Pet(name='Toby', kind='dog', owner=person1)
    db.Pet(name='Missy', cand='cat', owner=person2)

Then you can call this function any time you want to repopulate your database.

4. How to use Pony with Flask-Testing

Currently Pony doesn’t support database URI syntax. Although it is possible to add the database URI support in the SQLAlchemy-compatible way, right now I don’t see the point in it. It seems that specifying arguments of the Database in a dict is enough. In the next example I add PONY_DB_PARAMS dictionary to the configuration and then use it for passing parameters to the define_database function which was described earlier. If you think that database URIs have some real benefits, please add a comment which describes these benefits.

from flask.ext.testing import TestCase
from app import app
from pony.orm import *
from some_module import define_database
from other_module import populate_database

class TestApp(TestCase):
    TESTING = True
    DEBUG = True
    PONY_DB_PARAMS = dict(provider='sqlite', filename='/tmp/test.db')

    def create_app(self):
        app = app.config.from_object(self)
        return app

    def setUp(self):
        self.db = define_database(**self.PONY_DB_PARAMS)
        populate_database(self.db)

    def tearDown(self):
        db.drop_all_tables(with_all_data=True)

    @db_session  # example of using db_session as a decorator
    def test_1(self):
        dog = self.db.Pets.get(name='Toby')
        self.assertEqual(dog.kind, 'dog')

    def test_2(self):
        with db_session:  # example of using db_session as a context manager
            count_of_cats = count(pet for pet in self.db.Pet if pet.kind == 'cat')
            self.assertEqual(count_of_cats, 1)

It is not necessary to store parameters of the test database inside Flask configuration. You can simplify this by hardcoding these parameters inside the setUp method:

def setUp(self):
        self.db = Database('sqlite', '/tmp/test.db')
        define_entities(self.db)
        self.db.generate_mapping(create_tables=True)
        populate_database(self.db)

---

Please let me know what do you think about it. Is there anything that we can implement in Pony in order to simplify the testing process?

[gaozhidf]

models/brands.py

from config.settings import DB_CONFIG
from pony.orm import *

db = Database()
db.bind(provider='mysql', **DB_CONFIG)
set_sql_debug(False)


class Brands(db.Entity):
    _table = 'brands'
    id = PrimaryKey(int, size=64, unsigned=True)
    country_id = Optional(int, default=0)

db.generate_mapping(create_tables=False)

test_api.py

class ApiTestCase(unittest.TestCase):
    def fake_db(self):
        from models.brands import db
        from pony.orm import Database
        self.db_fd, self.db_fp = tempfile.mkstemp()
        new_db = Database(provider='sqlite', filename=self.db_fp)
        new_db.entities = db.entities
        new_db.generate_mapping(create_tables=True)
        db.provider = new_db.provider
        db.schema = new_db.schema
        self.db = db

    def setUp(self):
        with mock.patch('models.brands.db.bind', new_callable=self.fake_db):
            self.app = api_server.app.test_client()

    def test_get_brands(self):
        rsp = self.app.get('/api/brand/')
        print(str(rsp.json))

    def tearDown(self):
        # self.db.drop_all_tables(with_all_data=True)
        os.close(self.db_fd)
        os.remove(self.db_fp)

it mays be a trick one, and mocks db.bind when it was called