TIP #1: Migrations as Modular Scripts

  TIP: 1
  Title: Contract Migrations as Modular Scripts
  Status: Draft
  Type: Awesome
  Author: Tim Coulter
  Created: 2016-04-18

Introduction

Truffle’s current deploy process only allows for a very strict type of contract: One that doesn’t take constructor parameters, and one that’s meant to exist as a singleton. For many people these restrictions are limiting and confusing, as deployment for different styles of contracts is needed. Furthermore, no matter the contract, Truffle’s deploy process fails to address the needs of future deployments. This proposal aims to provide a solution that allows for deploying complex contracts in a way that doesn’t introduce that complexity back into Truffle itself; as well, this proposal will outline a method for managing multiple deployments across the lifetime of a project.

Currently, the only recourse for developers who want to perform complex deployments is to run after_deploy scripts. These are scripts that can be run with truffle exec, and will be run directly after deployment and whenever a deployment is needed. The benefit of after_deploy scripts is they can run arbitrary code during deployment, allowing for as complex of a deployment as the user needs. The negatives, on the other hand, is that after_deploy scripts are hard to use and that the addresses of deployed contracts within the scripts are not saved.

after_deploy scripts, as well, don’t address future deployments, as the initial scripts written would assume contracts related to the dapp don’t already exist on the desired network. When a future deployment is needed, when a project’s contracts do exist – which need to be modified or updated as a course of the deployment – the user would be required to manage those scripts depending on the environment in a way that’s tedious and error prone.

Solution: Migrations

To address all of the issues mentioned above, let me introduce the concept of Migrations. This idea is heavily influenced by Rail’s ActiveRecord migrations, but is much lighter weight.

1. Migrations, in general, are a set of scripts identified by their file name and prefixed by a number, that are responsible for initializing and maintaining the state of a set of contracts on a given network. The numbered prefix represents the date and time of the migration, and the rest of the file name is meant to communicate the purpose of the migration. Example: 1461005828324_initial_deploy.js. Here, the numbered prefix was produced by new Date().getTime(), but it can theoretically be any number.
2. Migrations assume that regardless of intentions, over the lifetime of a dapp contract state will need to be modified and contract code will either need to be updated, modified, or destroyed in order to support new features and bug fixes.
3. Migrations are simple Javascripts modules that export the code they want to run and are expected to call a done() callback when finished, outlined below. This structure allows users to execute complex deployment steps without running into the same issues with after_deploy scripts mentioned above.
4. Migrations will be run in the order of their filename, lowest first. Migration state – i.e., a record of which migrations have been run on a given network – will be saved in a special Migrator contract deployed to the network, so on future deployments Truffle can “intuit” which migration to run next. This contract will be added automatically to the user’s project upon creation (truffle init). If the Migrator contract doesn’t exist or is deleted by the user, Truffle will instead run all migrations in order, and will not save the migiration state upon completion.
5. A new top-level directory will be created for migrations, called migrations. Truffle will determine which migrations need to be run based on the network’s saved migration state as well as the files within the migrations directory.
6. Truffle’s current deployment architecture will be completely removed except for automatic library linking. This includes removing the deploy configuration from the project’s truffle.js file. The truffle deploy command will now run the migrations instead of deploying contracts specified within the deploy configuration.

Discussion

One cause of contention is saving the migration state on the blockchain. Theoretically this state could be saved in a file within the target environment. The benefits of having it on the blockchain is that in my experience, it’s easier to manage and the migrator contract could be edited to act differently if the user desired. On the other hand, the largest negative is that deploying and interacting with the Migrator contract would cost real Ether on the public network. I’m open to other’s opinions on this (as well as anything else) and would appreciate your thoughts.

Another cause for contention is that as of now, migrations aren’t atomic. If a migration fails, there’s no way to automatically revert the transactions within the migration that succeeded, as you can with Rails. This likely isn’t a feature that can be provided due to the nature of the Ethereum network, but ideas welcome.

Lastly, there is currently no notion of migrating both “up” and “down” as provided in Rails – currently they only happen in one direction. I would love to hear if this feature would be useful.

Code & Examples

Here are some example migrations as well as proposed solidity code for the migrator contract:

Example Migration: Initial deploy

This will deploy a new contract to the network and save that contract’s address.

File name: 1461005828324_initial_deploy.js

module.exports = function(accounts, done) {
  // Add a new contract to the network
  MyContract.new(function(instance) {
    // Tell Truffle that this new contract represents
    // the deployed version of this contract
    MyContract.address = instance.address;
  }).then(done).catch(done); 
};

Example Migration: Upgrading a Contract

This will expect a currently deployed contract and update a value (Hub & Spoke model).

File name: 1461005828324_update_spoke.js

module.exports = function(accounts, done) {
  // Expect an already deployed Hub.
  var hub = Hub.deployed();

  // Deploy a new spoke. 
  Spoke.new(function(spoke) {
    // Tell the hub about the new spoke.
    return hub.setSpoke(spoke.address);
  })).then(done).catch(done);
};

Proposed Migrator Contract

Effectively pseudo-code. Subject to change.

contract Migrator {
  uint[] public completed_migrations;

  function Migrator() {}

  function migrate(uint migration) {
    completed_migrations[completed_migrations.length++] = migration;
  }
}

Feedback

All feedback is welcome. This TIP will remain active until a sufficient consensys is reached. Please leave all comments on github. Thanks!

[tcoulter]

@thiagodelgado111 Couple things:

Having an up/down concept is a good one, in general, but undefined behavior will occur whenever a migration errors. For instance, imagine a migration that, when migrating up, makes 20 transactions (possibly an exaggeration). Say transaction 11 fails: This means the whole migration fails. Since we’re dealing with a blockchain, you can’t revert those transactions, which means you’re in an unknown state. If you migrate down from there, it’s likely that your down will fail as well since the down expects the environment to be in a specific state, which it is not. This is a similar problem with rollbacks: How do you rollback a migration that can’t be reverted? The only way is human interaction, where you have to determine the best possible rollback yourself - and it’s entirely situationally dependent. Which means - to recap - we could have a down and a rollback, but they’re unlikely to work in most cases, and so in most cases are very useless. The only way around this that I can see is to have one transaction per migration, but this becomes untenable - though I can imagine some people going this route.

Also, in my experience with other migration systems (i.e., Rails) down and rollback were rarely used in production, and were only useful during development. Though I can see the benefit of having these features during development, we there are other ways to solve that problem, like testing/developing from a known blockchain state, which can be reverted. That said, you’ve made me think of a great feature: automatic reverts when using the TestRPC. The TestRPC supports state reverting, so when developing migrations, truffle can take advantage of this and automatically revert if there’s an error during a migration.

Given the above, out of the gate I’m likely not going to add in support for down and rollback, but we can consider those features once more people use the first implementation and we get feedback from users.

Thanks, as always!

[tcoulter]

Just pushed the branch migrations to master. No differences from master yet, but changes will be added there.