Feature discussion: Auto-sync table fields

Hi. I just noticed that there are currently no feature which allows us to automatically add new fields to a table. For example, if I have this model

class Project extends Model {
  static table = "projects";
  static timestamps = true;

  static fields = {
    id: { primaryKey: true, autoIncrement: true, type: DataTypes.INTEGER },
    name: DataTypes.TEXT,
    date: DataTypes.DATETIME,
  };

  static transactions() {
    return this.hasMany(Transaction);
  }
}

and when I add the newField field

class Project extends Model {
  static table = "projects";
  static timestamps = true;

  static fields = {
    id: { primaryKey: true, autoIncrement: true, type: DataTypes.INTEGER },
    name: DataTypes.TEXT,
    date: DataTypes.DATETIME,
    
    /* This field */

    newField: DataTypes.INTEGER
    
    /* This field */ 
  };

  static transactions() {
    return this.hasMany(Transaction);
  }
}

and then we run

await db.sync();

The newField doesn’t get automatically added. To add new field, we’ll have to pass { drop: true } and remove the whole table. Every ORM I’ve come across like GORM (Golang) or JPA (Java) has this auto-add fields feature without dropping the table and I think it’s crucial.

Are there any plans to implement this yet or is someone already working on it? If not, I’d like to try making this feature.

I don’t think this feature is trivial and implementing this can be tricky especially with relationships, so I’ll need lots of feedback as I’m working on this.

[vmasdani]

Hi, I’ve done some work on my fork vmasdani/denodb. The function is experimentalAutoMigrate. I’ve successfully detected missing fields using select field like this:

// https://github.com/vmasdani/denodb/blob/287a975371ea87185594940ebf884768183041e0/lib/model.ts#L208
this._queryBuilder
  .select(key)
  .table(
    this.table,
  ).get();

If the field does not exist, it will throw an error and will try to create the missing field/column. (Note the [object Object] bug while mapping the primary key, haven’t fixed that yet)

I tried an attempt to run an alter table query. However I learned that QueryType does not support alter table and add column yet so I had to add the query type and description here and here.

After that, I also noticed that this library uses dex for dialect translation, so It’ll take some time for me to learn how to map QueryDescription to dex’s query builder for the alter table.

// https://github.com/vmasdani/denodb/blob/287a975371ea87185594940ebf884768183041e0/lib/model.ts#L221
const alterQuery = this._queryBuilder
  .removeSelect()
  .alterTable(this.table)
  .addColumn(key)
  .columnType(this.fields[key].toString());

I haven’t made tests but you can use this code to try the experimentalAutoMigrate function with this code:

// main.ts
import {
  Database,
  DataTypes,
  Model,
  PostgresConnector,
  SQLite3Connector,
} from "./denodb/mod.ts";

const connection = new SQLite3Connector({
  filepath: "db.sqlite3",
});

const db = new Database(connection);

class Flight extends Model {
  static table = "flights";
  static timestamps = true;

  static fields = {
    id: { primaryKey: true, autoIncrement: true, type: DataTypes.INTEGER },
    departure: DataTypes.STRING,
    destination: DataTypes.STRING,
    flightDuration: DataTypes.FLOAT,
  };
 
}

db.link([Flight]);

await db.experimentalAutoMigrate();

Run with deno run --allow-all main.ts

I’ll give more updates when I finished integrating QueryDescription into dex. Should I open a PR for this progress, by the way?

[stillalivx]

Guys! Sorry to invited my self to the discussion. I’m so excited to see where this functionality will be go! If you need help, count on me.