Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Use sqlalchemy Python script to test availability of database.

See original GitHub issue


  • CTFd Version/Commit:

2.0.0 branch

  • Operating System:

MacOS X, but building/deploying container images.

  • Web Browser and Version:


What happened?

The check using mysqladmin in will fail if the target system only has MySQL libs package and doesn’t have MySQL clients package.

Further, the wiki says:

My personal suggestion is MySQL or MariaDB, but PostgreSQL will work just fine.

The script is hard wired to MySQL and expects MySQL clients package to be installed. Would be nice that if also want to use PostgreSQL and only provide alternate Dockerfile and not change anything else, that this would work and didn’t have to rewrite it. Similarly, if use alternate build system like S2I for OpenShift, that can reuse existing scripts unchanged.

What did you expect to happen?

Would hope database check is database agnostic and work with only database libs available.

How to reproduce your issue

In my case hit it as building from source using S2I (Source-to-Image) builder in OpenShift. The Python base images do not include MySQL clients package.

Any associated stack traces or error logs

Waiting for ctfd-mariadb to be ready
  | /opt/app-root/src/ line 33: mysqladmin: command not found
  | /opt/app-root/src/ line 33: mysqladmin: command not found
  | /opt/app-root/src/ line 33: mysqladmin: command not found
  | /opt/app-root/src/ line 33: mysqladmin: command not found


Suggest database agnostic check be used which uses small Python script and sqlalchemy.

import os
from sqlalchemy import create_engine
database = create_engine(os.getenv('DATABASE_URL'))
database.execute('select 1')

This has added benefit that also checks that required Python packages installed correctly and work before starting up the application.

It should also technically work for SQLite if someone wanted to override DATABASE_URL and use SQLite with file system path, to directory in a mounted persistent volume, rather than have database created in default location of source code directories where not possible to mounted a persistent volume. Although in SQLite case the database file would technically need to exist first.

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:11 (6 by maintainers)

github_iconTop GitHub Comments

GrahamDumpletoncommented, Nov 6, 2018

Then can use:

function database_ready() {
    python -c "import os, sys
from sqlalchemy_utils.functions import database_exists
database_exists(os.getenv('DATABASE_URL')) or sys.exit(1)" > /dev/null 2>&1

The docs for database_exists() leave a bit to be desired because it isn’t a simple case of returning True or False. It can still raise an exception such as in the case of not being able to route to the database. So still have to throw away output to avoid the exception tracebacks filling the logs.

GrahamDumpletoncommented, Nov 6, 2018

The script could use:

function database_ready() {
    python -c "import os
from sqlalchemy import create_engine
database = create_engine(os.getenv('DATABASE_URL'))
database.execute('select 1')" > /dev/null 2>&1

# Check that the database is available
if [ -n "$DATABASE_URL" ]; then
    database=`echo $DATABASE_URL | awk -F[@//] '{print $4}'`
    echo "Waiting for $database to be ready"
    while ! database_ready; do
        # Show some progress
        echo -n '.';
        sleep 1;
    echo "$database is ready"
    # Give it another second.
    sleep 1;

There is another existing problem with that loop for checking the database though.

        echo -n '.';

This attempt at showing progress will not necessarily work when run in a container as display of output in log system of the container platform can be line buffered. This means you will only see the dots all at once when it finally succeeds, or possibly if some line buffering limit is reached. Would be better to use:

        echo 'wait 1 sec for database';

and accept that it will display something every second, one per line. If too noisy, increase sleep time to 2 seconds to at least halve it.

Read more comments on GitHub >

github_iconTop Results From Across the Web

how to search for the existence of a database with sqlalchemy
I need to write a script with python sqlalchemy that searchs if a database exist, if the database exists it should query the...
Read more >
SQLAlchemy — Python Tutorial - Towards Data Science
SQLAlchemy can be used to automatically load tables from a database using something called reflection. Reflection is the process of reading the database...
Read more >
4. Testing - Essential SQLAlchemy, 2nd Edition [Book] - O'Reilly
In, you can see that our database is set up via the DataAccessLayer class. We're using this data access class to enable...
Read more >
Connecting to SQL Database using SQLAlchemy in Python
In this article, we will see how to connect to an SQL database using SQLAlchemy in Python. To connect to a SQL database...
Read more >
How to Use Flask-SQLAlchemy to Interact with Databases in a ...
You'll use it to create a database object that connects to your Flask application, allowing you to create and manipulate tables using Python...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found