Workflow Engine & API Changes

For V3, I am thinking of making a number of changes.

These are just thoughts and I am looking for feedback. Nothing is set in stone as of yet.

Activity Data & Handlers

Currently, an activity is both a DTO as well as a service - it contains input & output properties and methods that implement behavior.

If we separate activity data from behavior, things will become easier:

• Serialization of activities simplified
• Ability to construct workflows by composing actual activity objects (this ties into the next section)
• Easier testing.

It would look something like this:

public class WriteLine : Activity
{
   public IActivityInput<string> Text { get; set; }
}

public class WriteLineHandler : IActivityHandler<WriteLine>
{
   public override void ExecuteActivity(WriteLine activity)
   {
      Console.WriteLine(activity.Text);
   }
}

A workflow would be constructed as follows:

public class MyWorkflow : Workflow
{
   public MyWorkflow()
   {
      Activities.Add(
         new Sequence ( // Support for container activities with natural scope.
            new WriteLine { Text = "Line 1" }, // Set a string literal. Implicitly cast to ActivityInput<string>.
            new WriteLine { Text = new CodeInput<string>(context => context.Input) }, // Set a dynamic value using a C# lambda.
            new WriteLine { Text = new LiquidInput<string>("Hello {{ Input }}") } // Set a value using a liquid expression.
         )
      );
   }
}

Connecting Activities

With V2, all activities are connected to one another via connections. Although this is convenient, it also poses a challenge to implement nested scopes and self-scheduling of child activities such as If, ForEach, Switch and others.

It also makes setting up coded workflows inconvenient. Although the Workflow Builder API does a nice job using Then<> and When, the following workflow would be even easier to setup:

public class MyWorkflow : Workflow
{
   public MyWorkflow()
   {
      Activities.Add(
         new Sequence(
            new HttpEndpoint { Path = "/demo", Methods = new[] { "GET" } },
            new If { 
               Condition = JavaScriptInput<bool>("queryString('answer') == '42'"),
               True = new WriteLine { Text = "Correct!" },
               False = new WriteLine { Text = "Wrong..." }
            }
        );
   }
}

This also makes connecting multiple sources to the same destination much more natural:

public class MyWorkflow : Workflow
{
   public MyWorkflow()
   {
      var exit = new WriteLine { Text = "Bye!" };

      Activities.Add(
         new Sequence ( // Support for container activities with natural scope.
            new HttpEndpoint { Path = "/demo", Methods = new[] { "GET" } },
            new If { 
               Condition = JavaScriptInput<bool>("queryString('answer') == '42'"),
               True = new Sequence(WriteLine { Text = "Correct!" }, exit), // Connect to exit node.
               False = new Sequence(new WriteLine { Text = "Wrong..." }, exit) // Connect to same exit node.
            }
         )
      );
   }
}

With this API, containment would be an intrinsic concept, which removes all of the unintuitive quirks we currently have with Composite Activities in V2.

A consequence of this is that we lose the concept of “outcomes”. Instead, an activity now defines “ports”. The If activity for example would look something like this:

public class If : Activity
{
   [Port] public IActivity True { get; set; }
   [Port] public IActivity False { get; set; }
}

public class IfHandler : IActivityHandler<If>
{
   public async ValueTask ExecuteActivityAsync(If activity, ActivityExecutionContext context)
   {
      var result = await context.EvaluateAsync(activity.Condition);
   
      if (result == true)
         context.ScheduleActivity(activity.True);
      else
         context.ScheduleActivity(activity.False);
   }
}

Being able to assign activities to activity properties requires the workflow designer to receive a number of changes:

• When connecting e.g. some WriteLine activity to the True property of an If activity, we want to see a connection drawn. The designer needs to be able to serialize that into an hierarchical object model, instead of maintaining a list of connections between activities.
• When designing a workflow, the user should be able to freely connect one activity to another. The model however should use a Sequence container activity when the user connects activities that don’t have explicit ports like the If activity.

Unwinding

V2 currently relies on “scopes” to enable outcomes such as "Iterate" to automatically re-schedule its owning activity such as ForEach, which then determines if the loop has finished or not. If finished, it schedules the "Done" outcome.

This implementation however is prone to error and is hard to apply to custom activities because there are multiple moving parts that need to cooperate to make this work.

With the proposed API changes, this will become a thing of the past, because it now becomes easy to determine if a child activity (an activity set to a port or within a sequence (which itself is a container activity) has completed. Activities such as ForEach will have two port properties: Loop and Complete, both of which are of type IActivity.

Pseudo code for ForEach:

public class ForEach : Activity
{
   public IActivityInput<bool> Condition { get; set; }
   [Port] public IActivity Loop { get; set; }
   [Port] public IActivity Complete { get; set; }
}

public class ForEachHandler : IActivityHandler<ForEach>
{
   public async ValueTask ExecuteActivityAsync(ForEach activity, ActivityExecutionContext context)
   {
      var result = await context.EvaluateAsync(activity.Condition);
   
      if (result == true)
      {
          context.ScheduleActivity(activity.Loop)
      }
   }

   // This method is invoked by the workflow engine once an activity completes that was scheduled by this activity.
   public void ScheduledActivityCompleted(ForEach activity, ActivityExecutionContext context, IActivity completedActivity)
   {
      if(completedActivity != Loop)
         return;

        var result = await context.EvaluateAsync(activity.Condition);
     
        if (result == true)
        {
            context.ScheduleActivity(activity.Loop)
        }
        else
        {
            context.ScheduleActivity(activity.Complete)
        }
   }
}

TODO: Describe workflow bookmarks for resuming suspended workflows.

[sfmskywalker]

Here’s an updated preview that takes inspiration from WF & ADF:

[sfmskywalker]

Check out this sneak-peek of a POC I’m working on for the Elsa 3 designer with embedded activities (as discussed during last community call)! There’s still a lot of work to be done.

For example, for simple activities, we could keep it embedded as you see now, but for more complex activities such as Sequence and Flowchart, we should probably only render a symbol and a display name, but open a new “tab” with a designer to edit the contents.

I’m also not sure about rendering the embedded activities as-is. Perhaps it needs a slightly different representation. Different color, just icon, no icon at all, etc. Feedback welcome!