Mup 1.5 Plan - Load balancing and zero downtime deploys

Mup 1.5 will add support for load balancing and zero downtime deploys. It will use Docker Swarm and the reverse proxy to implement this. Mup 1.5 does add an experimental swarm integration, but it is not required for load balancing or zero downtime deploys.

• Docker plugin
  • Setup docker swarm. Done in mup 1.4
  • Show swarm status in mup docker status. Done in mup 1.4
  • Discover and fix edge cases
  • Document restrictions. In this initial release, mup will only support one app using swarm per server. Other apps can be deployed to the same servers as long as they do not use swarm. Also, it will only support deploying apps when Prepare Bundle is enabled due to how services rollback versions. It will be recommended to use servers with at least 1gb ram with Docker Swarm.
  • Recover when quorum is lost
  • Show error when servers in config are part of several clusters
  • Show error when servers are part of a cluster, but mup doesn’t have access to a manager in that cluster
  • Decide how to handle nodes and managers that are not in the list of servers in the config
  • Command to destroy the swarm cluster. Done in mup 1.4
• Meteor plugin
  • Use specific numbers for an app’s image tag instead of previous and current to support how services roll back failed deployments
  • Deploy app as a swarm service
  • Explore possibilities of validating deployment
• Plugin API
  • Review swarm api’s that have already been added
  • Document api’s
  • ~Give user more control if desired over what servers are managers~
  • Add general tasks any plugin can use:
    • Create service
    • Update service
    • createOrUpdateService
    • Wait until service is ready
      • Decide how to handle a service that encounters an error starting since it repeatedly retries
      • Allow plugin or Meteor Up to look at output to offer suggestions for fixing errors
• Mongo (optional)
  • Deploy mongo as a service with an overlay network to support the app running on multiple servers
• Proxy
  • ~Add support to nginx-proxy for docker swarm, or run docker flow, dockercloud haproxy, or traefik between nginx-proxy and the app~
  • Run container on proxy’s servers for each service using the proxy (needed for the reverse proxy to know about the service)
  • Create container to generate upstream blocks for services
  • Create fork of nginx-proxy that supports custom upstream blocks
  • Connect apps to same overlay network as proxy
  • Only run the proxy on manager nodes
  • ~Show what servers to use in the dns records~
  • Graceful shutdown
• Default docker image
  • Check compatibility between meteorhacks:meteord and zodern/meteor
  • Consider changing the default docker image to zodern/meteor

[zodern]

I have done some work the past couple months on this, mostly investigating the different options for using Docker swarm with the nginx-proxy container and starting work on a solution for it. I will be working part time on Mup next month.

[zodern]

The second beta is now available. Most of the changes have been around bug fixes and adding missing aspects of swarm support, and general improvements (many implemented by @rolljee). Zero downtime deploys mostly works now. I’ve updated https://github.com/zodern/meteor-up/pull/976#issue-200468540 with the current status.

What is left: https://github.com/zodern/meteor-up/projects/4 The biggest tasks remaining are informing the user and show logs from failed tasks when a deploy fails, and improving rolling deploys to support apps that are slow to start.