Update a secondary resource from a primary

Feature #23

Status: In Progress

• #1677
• #1678
• #1736
• #1679
• #2069

Requirements

• A resource should be able to update properties on another resource before or after its installation process
• The steps need to happen in serial. All ‘pre’ steps must happen in order, before the deployment starts.
• The user should be able to have a view on the overall status of the operation, including the pre + post steps as part of the overall deployment
• The pre + post steps should be able to reference properties from the resource object

Example

A user submits a POST request for a VM user resource. In the resource template, a pipeline: {} block is defined, and contains steps which describe the updating of a firewall. The API reads this array, and creates an Operation document that contains the steps for the pipeline. Each step is executed, using the existing porter runner method, one after the other, until they are all done and the API marks the Operation as complete.

Sequence Diagram

The API layer centrally orchestrates the whole process. When a user submits a request for a resource which has a pipeline: [] array within the template, the API will enqueue the first step, handing the operation_id and step_id to the Resource Processor. When the resource processor returns the status update of a deployment, it will reference the operation_id and a step_id. The API will get the Operation document, check if there are more steps to be carried out, and enqueue the next. When all steps are complete, the overall status will be updated.

sequenceDiagram
    User->>API: POST user_resource {...}
    API->>API: Template has pipeline {}? 
    loop Steps
        API ->> API:Create operations document <br/>containing multiple steps 
    end
    API ->> Cosmos:Operations doc
    API ->> API:Substitute property values for <br>resource defined in step[0]
    API ->> Cosmos:Update resource defined in step[0]
    API ->> Resource Processor: Enqueue upgrade/install action
    API ->> User: Operations Doc
    Note left of User: {steps:[<br>{status:in_progress}, <br>{status:waiting}<br>]}
    Resource Processor --> User: 
    Resource Processor --> User: 
    Resource Processor --> User: 
    Resource Processor ->> Resource Processor: Run bundle <br>upgrade
    Resource Processor ->> API:Enqueue status update message
    API ->> Cosmos: Get Operation doc
    Cosmos ->> API: Operation doc
    loop While Operation has more Steps
        API ->> Cosmos:Get resource template
        Cosmos ->> API: Resource template
        API ->> API:Substitute property values for <br>resource defined in step[N]
        API ->> Cosmos:Update resource defined in step[N]
        API ->> Resource Processor: Enqueue upgrade action
        Resource Processor ->> Resource Processor: Run bundle <br>upgrade
        Resource Processor ->> API:Enqueue status update message
        API ->> Cosmos: Get + update Operation
    end
    API ->> Cosmos: Mark operation as complete

Key Components

The design hinges around 3 key components - the pipeline:[] block in the resource template, the multi-step operations document, and the property substitution mechanism.

Pipeline: [] Block

This is an optional addition to the existing resource template, deployed to /api/<resource_type>-templates. It defines the steps in the workflow. Consider an example of a template updating the firewall before and after a VM deployment:

./templates/workspace_services/guacamole/user_resources/guacamole-azure-windowsvm/template_schema.json:

{
  "$schema": "http://json-schema.org/draft-07/schema",
  "$id": "https://github.com/.../template_schema.json",
  "title": "Windows Virtual Machine",
  "properties": {
   ...
  },
  "pipeline": [ # <- new array
    "upgrade": [
      {
        "step_id": "guid-here",
        "resource_template_name": "tre-shared-service-firewall",
        "resource_type": "shared_service",
        "resource_action": "upgrade",
        "properties": [
        {
          "name": "rules",
          "type": "array", # <- or "string" or "object"
          "substitution_action": "remove" # <- or "append" for arrays, or "overwrite" for other value types
          "value": "{{$resource.properties.ip_address}}" # <- used in substitution. Could use hardcoded vals or something from the resource props
        }]
      },
      {
        "step_id": "main" # <- "main" = "this resource"
      },
      {
        "step_id": "guid-here",
        "resource_template_name": "tre-shared-service-firewall",
        "resource_type": "shared_service",
        "resource_action": "upgrade",
        "properties": [
        {
          "name": "rules",
          "type": "array",
          "substitution_action": "add"
          "value": { "name": "rule_1", "ip_address": "{{$resource.properties.ip_address}}" } # <- the ip would have been updated in the VM props
        }]
     }
    ],
    "install": [ 
      ...
    ],
    "delete": [
      ...
    ]
  }
}

Since this new pipeline: {} block will be present in the template schema json, during schema validation this object will be removed from the schema as the json validation occurs.

Substitutions Mechanism

Aside from orchestration, the core logic and real complexity resides in the substitution mechanism. This will:

• Get the resource document for the resource being updated (firewall in this case)
• Get the resource document for the primary resource (VM in this case)
• Attempt to take placeholder values from the pipeline:{} block, replace them with real values at runtime, and construct an object to the follow the same path as POST and PATCH requests - even being schema validated by the relevant template. For the example above, after substitution we’d end up with the equivalent of:

{
   "properties": {
      "rules": [
        { "name": "rule_0", "ip_address": "172.0.0.0" }, # <-- existing rule
        { "name": "rule_1", "ip_address": "172.0.0.1" } # <-- new rule
       ]
    }
}

Substitution in Array Types

Some properties in a resource document will actually be arrays. The Firewall rules on the Firewall Shared Service is likely to be array. For these types we will support append and remove. If the substitution_action type is one of these, the code will treat the property value as an array and either append or search + remove the value.

Multi-Step Operations Document

Currently each operation in the system relates to a single operations document, with a single status. When we have a deployment that affects multiple resources, we need a way to represent that to the user, and for the API to track what is done and what needs to be done next.

For example, an operation document for the multi-step template outlined above might look like:

{
  "operationId": "guid-here",
  "status": "in_progress",
  "multi_step_template_name": "Update FW before and after VM", # <-- or some identifier
  "resource_id": "id-of-VM-resource",
  "steps": [ 
    {
      "step_id": "guid-here",
      "status": "success",
      "resource_id": "id-of-resource-being-updated", # <-- will we have the resource id, or name?
      "action": "upgrade", # <- potential to support install of supporting resources, not just upgrades
    },
    {
      "step_id": "main",
      "status": "in_progress",
      "resource_id": "id-of-VM-resource",
      "action": "install"
    },
    {
      "step_id": "guid-here",
      "status": "waiting",
      "resource_id": "id-of-resource-to-be-updated",
      "action": "upgrade",
    }
  ]
}

For the MVP:

• We’ll focus on updating (upgrade) secondary resources, but in time support for creating (install) secondary resources would fit into this model.
• The code would only substitute the string “$resource” for the primary resource object, so members of that object could be referenced like $resource.properties.ip_address, or $resource['properties']['ip_address']. In time this might be expanded to allow other types of substitutions.
• We’ll focus on shared_service types of secondary resource. In time ‘my’ workspace would likely need to be included as a secondary resource type.

[damoodamoo]

Ok design updated:

@tanya-borisova / @tamirkamara / @martinpeck :

• Moved from a separate document to defining the steps within the template_schema.json document. This fits in with the current architecture much simpler
• No pre or post steps - but just steps. The step referencing the primary resource is just defined as main. This could change but some way of identifying a step as “deploy/update this resource”.
• Started with an example

@ross-p-smith :

• Included upgrade and install in each step - although for the MVP we’ll focus on upgrade only.
• Moved to calling it pipeline rather than multi-step-thing.

[marrobi]

Great work going through this. Really like the approach, sure it will need refinement but will unlock a lot of the dependency challenges we have.