[Feature Request] Change Ref approach to expression-based

Describe the solution you’d like

Akin to Hangfire and others, use LINQ expressions to let calls be made. This is more inline with how other .NET libraries work. We need to make sure status still work and we can extract return types in Task vs non-Task. This will also require updating docs and the blog post.

Also, as part of this effort, we need to modernize registration of activities and workflows. We probably need to:

• Make definitions the only collections on worker options instead of the “additional”
• Allow activities to also be registered by entire type/class that crawls looking for activity attributed
• Abstract activity definition a tad to accept a form with arg types, return type, and a Func<object?, Task<object?[]?>> invoker. Put an InvokeAsync on it I guess. This is needed for dependency injection support. Would accept alternative activity class factory approach if we can think of it, though this is probably good enough. Can expose this all the way out to the AddActivity overload in the worker options.

[cretz]

Ok, did some discussion with team. I am going to open a PR all-in on the expression approach (assuming I don’t hit snags with inference). We can evaluate that then. That means you actually can’t use the delegate/ref pattern if you wanted to (sorry!). So basically:

var temp1 = await Workflow.ExecuteActivityAsync(Foo.StaticWithArgAndResult, 123);

var temp2 = await Workflow.ExecuteActivityAsync(Foo.Ref.InstanceWithArgAndResult, 123);

Becomes:

var temp1 = await Workflow.ExecuteActivityAsync(() => Foo.StaticWithArgAndResult(123));

var temp2 = await Workflow.ExecuteActivityAsync((Foo foo) => foo.InstanceWithArgAndResult(123));

With the removal of the dependency, familiarity to .NET devs, and that we (really really) hate two ways of doing things, this just won out. But we’ll reevaluate during PR time if I hit any snags.

[cretz]

Ok, could use some more feedback here (x-posted to #dotnet-sdk on Slack). If I have:

await Workflow.ExecuteActivityAsync((Foo foo) => foo.SomeMethod(123 + 456));

I need to evaluate that argument. From what I read, that means I have to use an approach like https://stackoverflow.com/a/60002882 (basically create a lambda out of each argument and compile at call time) which is really slow. Even Compile(true) available in all but our oldest supported version (framework 4.6) is slow. If you had passed:

await Workflow.ExecuteActivityAsync((Foo foo) => foo.SomeMethod(579));

Then I could just use ConstantExpression.Value, but that is a non-obvious optimization to users to make sure they create their parameter outside the closure. And .NET can’t infer params for something like:

await Workflow.ExecuteActivityAsync((Foo foo) => foo.SomeMethod, 123 + 456);

Looking at hangfire and others, they have elaborate evaluators and will cache some known expression tree patterns resorting to Compile() as needed.

Options (not all are mutually exclusive):

1. Abandon expression trees - pro: good performance, con: not as common in ecosystem
2. Accept the performance hit of compiling non-constant expressions for arguments of workflows/activities on each invocation - pro: easy to read, con: bad performance
3. Put in the effort for matching some common pattern types - pro: better performance in some cases, con: lots of code and probably won’t support common use case of record instantiation (also https://learn.microsoft.com/en-us/dotnet/csharp/advanced-topics/expression-trees/expression-trees-execution#execution-and-lifetimes discourages caching saying equality/identity checks are more expensive than compile)
4. Disallow in-lambda evaluation (i.e. require all arguments be ConstantExpression provided outside the lambda) - pro: good performance, con: runtime only and encourages potentially uglier code
5. Warn on in-lambda evaluation - pro: encourages good performance, con: noisy, non-obvious, encourages potentially uglier code
6. Use something like https://github.com/dadhi/FastExpressionCompiler - pro: better performance, con: new dependency, some expressions don’t work
7. Another option I am unaware of?

Thoughts? I didn’t put “allow expression trees and ref pattern” above because we just can’t reasonably have two approaches. I may do option 2 for now while implementing, but option 1 sure looks tempting and I am hoping I don’t have to maintain two approaches just so I can benchmark differences.