Broken custom IDataObject implementations

• .NET Core Version: 5.0

• Have you experienced this same bug with .NET Framework?: No

Problem description:

PR #3388 removed the ComVisible attribute from System.Windows.Forms.IDataObject. That largely breaks the marshaling custom implementation of IDataObject through clipboard within the same application. The concern about marshaling of individual calls is not applicable in that case because the runtime could resolve the IDataObject to the original .NET object implementation.

It likely broke this case in Clipboard.GetDataObject and now all returned objects are wrapped in the default DataObject which doesn’t support all the custom types: https://github.com/dotnet/winforms/blob/2fd2a5840b345ef83e0d15a91747325af27441f1/src/System.Windows.Forms/src/System/Windows/Forms/Clipboard.cs#L170-L173

Expected behavior:

Minimal repro:

TBD

[RussKie]

FWIW, we’re working on making the clipboard API more resilient to address issues like this. I have a very rough draft (thanks to @AaronRobinsonMSFT) on my fork, and @JeremyKuhne is tinkering in this area as well.

[weltkante]

@filipnavara I want to follow up on your original issue, as promised

First here’s what we are doing: just store a MarshalByRefObject (Desktop Framework only) or a serialized COM reference into a data object slot with a custom format. This could even be a back-reference to the custom data object itself if you want it to.

example for putting a COM reference into a data object

Some notes about the code below:

• Stream is the only object that can be transferred safely between frameworks for custom formats, as all other object types are serialized via BinaryFormatter (even strings) which is not available in 3rd party frameworks that may want to talk to the clipboard and even is going away in future .NET Core. Though if you only care about in-process usage that may not matter to you and you can just stick in the moniker as a string.
• any stream you put in is going to come out as a MemoryStream since its transferred through a HGLOBAL memory block, so I’m always working with MemoryStream to simplify the code
• if you want additional protection you can add a private GUID as prefix to the moniker and strip it on reading after using it to verify its actually in your format (but IMHO using a sufficiently unique custom format string should be enough protection against accidental slot colissions)
• to add even more protection you could use a hash or encryption to protect/validate the moniker but at this point it becomes silly, just mentioning it for completeness, if the user (or a program run by the user) wants to break you they have easier ways to do so
• you can restrict effective visibility to your own process by using a non-marshalable interface (i.e. something just defined in your assembly via ComImport, using a private GUID and without TLB registered)
• alternatively you can just use below code to put an arbitrary .NET object into the data object, if the receiving side is in the same process it will unpack as a .NET object, but if you see a COM reference then its in a different process
  • of course if you treat arbitrary objects as COM objects that relies on specific behavior of the framework you are using. It may change in future frameworks and no longer allow getting an IUnknown on an arbitrary managed object, in which case you need to do the boxing yourself. Just put your value in a .NET object you can expose to COM instead of relying on the framework creating an implicit COM object

public static class DataObjectHelper
{
    private static Guid IUnknownIID = new Guid("00000000-0000-0000-C000-000000000046");

    [DllImport("ole32", ExactSpelling = true, CharSet = CharSet.Unicode, PreserveSig = false)]
    public static extern System.Runtime.InteropServices.ComTypes.IBindCtx CreateBindCtx(int reserved = 0);

    [DllImport("ole32", ExactSpelling = true, CharSet = CharSet.Unicode, PreserveSig = false)]
    public static extern System.Runtime.InteropServices.ComTypes.IMoniker CreateObjrefMoniker([MarshalAs(UnmanagedType.IUnknown)] object punk);

    [DllImport("ole32", ExactSpelling = true, CharSet = CharSet.Unicode, PreserveSig = false)]
    public static extern System.Runtime.InteropServices.ComTypes.IMoniker MkParseDisplayName(System.Runtime.InteropServices.ComTypes.IBindCtx pbc, [MarshalAs(UnmanagedType.BStr)] string szUserName, out uint pchEaten);

    public static void SetObjectReference(this IDataObject obj, string format, object reference)
    {
        if (reference == null)
            throw new ArgumentNullException(nameof(reference));

        System.Runtime.InteropServices.ComTypes.IMoniker moniker = null;
        System.Runtime.InteropServices.ComTypes.IBindCtx context = null;
        try
        {
            moniker = CreateObjrefMoniker(reference);
            context = CreateBindCtx();
            moniker.GetDisplayName(context, null, out var id);
            obj.SetData(format, new MemoryStream(Encoding.UTF8.GetBytes(id)));
        }
        finally
        {
            if (context != null)
                Marshal.ReleaseComObject(context);

            if (moniker != null)
                Marshal.ReleaseComObject(moniker);
        }
    }

    public static object GetObjectReference(this IDataObject obj, string format)
    {
        var data = obj?.GetData(format) as MemoryStream;
        if (data is null)
            return null;

        return GetObjectReference(Encoding.UTF8.GetString(data.ToArray()));
    }

    private static object GetObjectReference(string monikerStr)
    {
        if (string.IsNullOrEmpty(monikerStr))
            return null;

        System.Runtime.InteropServices.ComTypes.IMoniker moniker = null;
        System.Runtime.InteropServices.ComTypes.IBindCtx context = null;

        try
        {
            context = CreateBindCtx();
            moniker = MkParseDisplayName(context, monikerStr, out _);
            moniker.BindToObject(context, null, ref IUnknownIID, out var obj);
            return obj;
        }
        finally
        {
            if (moniker != null)
                Marshal.ReleaseComObject(moniker);

            if (context != null)
                Marshal.ReleaseComObject(context);
        }
    }
}

Having given that example of how we do it currently, there is a certain simplicity in your broken solution, especially performance-wise, so I have spent more time to research why it was working for you and if theres a way to make it work reliable.

more technical details of what happens

So lets break down what you were doing again:

• you were calling OleGetClipboard, which returned a COM object instead of your managed object
• you were casting to IDataObject (or later to your custom copy of the interface)

Note that the second step still returns a COM object and not a managed object, regardless of which framework I tried it on, which indicates that you do not have a reference to your original object, but to “something else”.

So I researched what OleGetClipboard is actually doing, here are the results:

• OleGetClipboard constructs an intermediate implementation of the native IDataObject to put between the producer and consumer. The clipboard is a global resource and implementations can have bugs or be incomplete, this intermediate objects tries to fix certain issues, mostly performing TYMED conversion the provider didn’t bother to implement.
• The intermediate data object intentionally violates COM rules by allowing QueryInterface calls it doesn’t understand to be forwarded to the original data object. This is not allowed by COM because you can’t get back to the IUnknown of the intermediate data object (and QueryInterface for IUnknown must always return the same instance, so this is where it violates COM rules)
• the .NET runtime gets incredibly confused by this violation of COM rules, so even though the intermediate implementation forwards QueryInterface for the internal marker interface the runtime can’t manage to unwrap the managed object
• your cast to IDataObject is also broken, the .NET runtime can’t figure out the cast (still doesn’t unwrap the object), and you get weird behavior. The behavior you observe only works for the first interface implemented on the data object, if you change interface order you suddenly get errors that you can’t cast, even if it is ComVisible

So the result of that research is that OleGetClipboard returns a wrapper which will forward QueryInterface calls (violating COM rules) and the .NET runtime is too confused to resolve the reference.

The good news is that you can rely on the behavior of OleGetClipboard (its always been there and won’t go away for compatibility reasons, applications rely on the ability to QI into the underlying object). The only thing you need to solve is help the .NET runtime to not be confused anymore. This is simple, just do the QI manually to move away from the wrapper object, once you are on the inner data object you have a “normal” COM object that plays by the rules and can be understood by the .NET runtime.

• Note that this solution does not depend on IDataObject being ComVisible - it just needs an arbitrary interface implemented on your data object but not implemented by the OLE data object.
• Note that this solution has a performance gain over your previous solution, your implementation did not unpack the managed object so you had a roundtrip through COM (including marshaling and passing through some sort of wrapper) for every function call. By unpacking the managed object properly you can talk directly to it in managed code, including casting to IDataObject regardless of it being ComVisible or not

helper code

[ComImport]
[Guid("59FFC7AA-BE08-4279-AC5E-C734D55429FD")] // random private GUID
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
interface ICustomDataObjectMarker { }

public static class ClipboardHelper
{
    [DllImport("ole32", ExactSpelling = true, CharSet = CharSet.Unicode)]
    private static extern int OleGetClipboard(out IntPtr data);

    public static IDataObject TryGetCustomDataObject()
    {
        IntPtr pWrappedDataObject = IntPtr.Zero;
        IntPtr pInnerDataObject = IntPtr.Zero;
        Guid knownInnerInterface = typeof(ICustomDataObjectMarker).GUID;

        try
        {
            // returns a wrapper object which violates COM rules so we use IntPtr to handle it
            if (OleGetClipboard(out pWrappedDataObject) < 0)
                return null;

            // manually do a QI to an interface not implemented by the wrapper, but implemented by your object
            if (Marshal.QueryInterface(pWrappedDataObject, ref knownInnerInterface, out pInnerDataObject) < 0)
                return null;

            // then let the .NET runtime take over, it unpacks the object and lets you cast to managed interfaces
            return Marshal.GetObjectForIUnknown(pInnerDataObject) as IDataObject;
        }
        finally
        {
            if (pInnerDataObject != IntPtr.Zero)
                Marshal.Release(pInnerDataObject);

            if (pWrappedDataObject != IntPtr.Zero)
                Marshal.Release(pWrappedDataObject);
        }
    }
}