Database disk image is malformed

Observed behavior

Upgrading kolibri from 0.7.2 to 0.10.0b3 will have this Database disk image is malformed error when running the kolibri start command.

Errors and logs

442 static files copied to '/home/user/.kolibri/static', 3 unmodified.
Operations to perform:
  Apply all migrations: admin, auth, content, contenttypes, device, exams, kolibriauth, lessons, logger, morango, sessions
Running migrations:
  Applying auth.0008_alter_user_username_max_length... OK
  Applying content.0007_auto_20180212_1155... OK
  Applying content.0008_contentnode_coach_content...Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/kolibri/dist/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/python3/dist-packages/kolibri/dist/django/db/backends/sqlite3/base.py", line 328, in execute
    return Database.Cursor.execute(self, query, params)
sqlite3.DatabaseError: database disk image is malformed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/bin/kolibri", line 9, in <module>
    load_entry_point('kolibri==0.10.0b3', 'console_scripts', 'kolibri')()
  File "/usr/lib/python3/dist-packages/kolibri/utils/cli.py", line 646, in main
    start(port, daemon=daemon)
  File "/usr/lib/python3/dist-packages/kolibri/utils/cli.py", line 261, in start
    update()
  File "/usr/lib/python3/dist-packages/kolibri/utils/cli.py", line 239, in update
    _migrate_databases()
  File "/usr/lib/python3/dist-packages/kolibri/utils/cli.py", line 176, in _migrate_databases
    call_command("migrate", interactive=False, database=database)
  File "/usr/lib/python3/dist-packages/kolibri/dist/django/core/management/__init__.py", line 131, in call_command
    return command.execute(*args, **defaults)
  File "/usr/lib/python3/dist-packages/kolibri/dist/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/usr/lib/python3/dist-packages/kolibri/dist/django/core/management/commands/migrate.py", line 204, in handle
    fake_initial=fake_initial,
  File "/usr/lib/python3/dist-packages/kolibri/dist/django/db/migrations/executor.py", line 115, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/usr/lib/python3/dist-packages/kolibri/dist/django/db/migrations/executor.py", line 145, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/usr/lib/python3/dist-packages/kolibri/dist/django/db/migrations/executor.py", line 244, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/lib/python3/dist-packages/kolibri/dist/django/db/migrations/migration.py", line 129, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/usr/lib/python3/dist-packages/kolibri/dist/django/db/migrations/operations/fields.py", line 88, in database_forwards
    field,
  File "/usr/lib/python3/dist-packages/kolibri/dist/django/db/backends/sqlite3/schema.py", line 238, in add_field
    self._remake_table(model, create_field=field)
  File "/usr/lib/python3/dist-packages/kolibri/dist/django/db/backends/sqlite3/schema.py", line 206, in _remake_table
    self.quote_name(model._meta.db_table),
  File "/usr/lib/python3/dist-packages/kolibri/dist/django/db/backends/base/schema.py", line 136, in execute
    cursor.execute(sql, params)
  File "/usr/lib/python3/dist-packages/kolibri/dist/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/python3/dist-packages/kolibri/dist/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/lib/python3/dist-packages/kolibri/dist/django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib/python3/dist-packages/kolibri/dist/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/python3/dist-packages/kolibri/dist/django/db/backends/sqlite3/base.py", line 328, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.DatabaseError: database disk image is malformed

Steps to reproduce

• Install the 0.7.2
• Copy the 0.7.2 Kolibri sample data at the .kolibri
• Upgrade to 0.10.0b3
• Run the Kolibri start command

Context

• Kolibri version: 0.10.0b3
• Operating system: Ubuntu 18.o4 LTS
• Browser:

[benjaoming]

@indirectlylit indeed - and it seems that I might have to go back to 0.7.2 to reproduce malformed databases.

Thanks @mrpau-richard for reproducing this 🎉

I have two favors to ask from you @mrpau-richard 😃

1. The technique for restoring the database as described here did not seem to work on Ubuntu 18.04, but I would appreciate if you can confirm this for your problematic db?

2. What are your simplest steps to reproduce a malformed database? I mean, not by copying the user data directory that you so kindly uploaded - but can you repeat the minimal set of steps to end up with this malformed db in ~/.kolibri?

[benjaoming]

Here is another type of malformed database. Happening on a Raspberry Pi w/ Raspian and Kolibri from PPA:

1. Upgraded Kolibri 0.10.1 on top of 0.10.1-beta1, which was working fine.
2. Observation 1: There are not db schema differences between these two.
3. Observation 2: It was a very heavy db (126 MB), carrying all of Khan Academy.
4. Then got the malformed db. It seems to have happened while stopping the old instance!

INFO 2018-08-03 15:03:16,730 options Option HTTP_PORT in section [Deployment] being overridden by environment variable KOLIBRI_LISTEN_PORT
INFO 2018-08-03 15:03:19,796 apps Running Kolibri with the following settings: kolibri.deployment.default.settings.base
INFO 2018-08-03 15:03:21,587 cli Kolibri server has successfully been stopped.
ERROR 2018-08-03 16:14:07,731 cli 
Your database is corrupted. This is a known issue that is usually fixed by running this command: 

    sqlite3 /mnt/data/.kolibri/db.sqlite3 .dump | sqlite3 fixed.db 
    cp fixed.db /mnt/data/.kolibri/db.sqlite3

Notice that you need the 'sqlite3' command available on your system prior to running this.


ERROR 2018-08-06 16:09:36,350 cli 
Your database is corrupted. This is a known issue that is usually fixed by running this command: 

    sqlite3 /mnt/data/.kolibri/db.sqlite3 .dump | sqlite3 fixed.db 
    cp fixed.db /mnt/data/.kolibri/db.sqlite3

Notice that you need the 'sqlite3' command available on your system prior to running this.