Question: is there a way to plug a D2D1 pixel shader into a composition pipeline?

Background

I’m working on adding D2D1 support to my library ComputeSharp, which allows developers to write HLSL shaders entirely in C#, and then takes care of all the work to transpile them to HLSL, compile them, and offer additional APIs to eg. run them as compute shaders over DX12 APIs. Recently I’ve started working on a new ComputeSharp.D2D1Interop library (see here), with the goal being of allowing Paint.NET to use this to replace all the current infrastructure with separate HLSL files for the pixel shaders, a pre-build MSBuild step to compile them, and then additional logic to load them and extract the constant buffer for dispatch. The new APIs in ComputeSharp instead allow developers to do everything in C#, and very easily. For instance:

[D2DInputCount(2)]
[D2DInputSimple(0)]
[D2DInputSimple(1)]
[D2DEmbeddedBytecode(D2D1ShaderProfile.PixelShader50)]
[AutoConstructor]
public readonly partial struct MyBlendShader : ID2D1PixelShader
{
    public float factor;

    public float4 Execute()
    {
        float4 pixel0 = D2D1.GetInput(0);
        float4 pixel1 = D2D1.GetInput(1);

        return Hlsl.Lerp(pixel0, pixel1, factor);
    }
}

This is then transpiled to an HLSL pixel shader automatically at build-time, and embedded into the assembly.

Authors can then use it like this:

MyBlendShader shader = new(0.6f);

// Get the shader bytecode
ReadOnlyMemory<byte> bytecode = D2D1InteropServices.LoadShaderBytecode<MyBlendShader>();

// Get the constant buffer with all shader properties
ReadOnlyMemory<byte> buffer = D2D1InteropServices.GetPixelShaderConstantBufferForD2D1DrawInfo(in shader);

This provides a major productivity improvement for developers:

• Everything is done in C#, no need for C++ APIs
• No HLSL knowledge is required
• No additional tooling required, no build steps
• Shader code gets IntelliSense support (!)

Additionally, I also have more lowlevel APIs to easily register a pixel shader effect manually from an ID2D1Factory1:

// Register a pixel shader effect for a target factory
D2D1InteropServices.RegisterPixelShaderEffectForD2D1Factory1<MyBlendShader>(d2D1Factory, out Guid effectId);

This effect can then be created normally from a factory using the returned id, to get an ID2D1Effect object. This effect will also automatically handle insertion of the node into the transform graph. Users also have the ability to define a custom draw transform mapping, through additional APIs not shown here.

Now, obviously something that came to mind is: it would be awesome if we could leverage this to also empower developers to easily write pixel shaders to insert into a Win2D/Composition pipeline, to realize all sorts of custom effects. Unfortunately though, I can’t seem to find a way to do this, as the documentation says that the PixelShaderEffect type doesn’t support composition (it’s marked with [NoComposition]). I know it would still be possible to draw stuff with a pixel shader either through a swap chain panel (which I also support, via DX12 APIs), or with a canvas drawing session, but neither of those has the same usability that just having an effect to plug into a composition pipeline provides.

Open questions

I’d like to explore this to ideally augment our APIs in the Microsoft.Toolkit.Uwp.UI.Media package (see docs here) (say, in some hypothetical Microsoft.Toolkit.Uwp.UI.Media.PixelShaders package), but I’m not sure how or if this is actually supported (or, if it is, I don’t see this being mentioned in the docs, along with an explanation of how to set this up).

Essentially:

• Why is PixelShaderEffect marked as [NoComposition]?
• Is there really no way at all to use this effect in a composition pipeline?
• Does the same limitation apply to both UWP and WinUI 3?
• Is there some other way to set this up, even if through additional manual work to set things up? This includes either through PixelShaderEffect, or through some more lowlevel building blocks, such as eg. having to manually query for a D2D1 factory, register the effect, create it, and then insert it into a graph somehow, as long as it’s documented.

I really think this could potentially give developers an incredibly powerful set of new tools to interact with the visual layer.

Pinging @codendone and @marb2000 as a follow up to our past conversations on composition APIs. Feel free to cc. others if you know someone that works in this area in particular that you think would be able to shed some light on this. Thanks! 😄

[pjmlp]

So yet another thing that WPF does better than WinUI, I wonder if the team actually knows what they are trying to replace.

• ShaderEffect Class

• PixelShader

[Sergio0694]

Bumping this (see https://github.com/microsoft/microsoft-ui-xaml/discussions/8638).