IDirect3DSurface/CanvasBitmap support?

[RandyDavis99]

I'm using Vortice with SwapChainPanels in WinUI 3 - it's working great, thank you so much!

I have a texture that lives as a ID3D11Texture2D and gets manipulated on a per-frame basis by Win2D (as a ID2D1Bitmap). I want that texture to also appear in my XAML UI. So I would like to use CanvasBitmap.CreateFromDirect3D11Surface, which takes a IDirect3DSurface. Then I can use that bitmap to draw into a CanvasControl as part of my XAML tree.

To get the IDirect3DSurface, I believe I need to first get an IDXGISurface from the ID3D11Texture2D. That part works within Vortice and it's fine. But I get confused as to how to get a IDirect3DSurface from the IDXGISurface. The docs says I need to call CreateDirect3DSurface (or possibly CreateDirect3D11SurfaceFromDXGISurface) as defined in windows.graphics.directx.direct3d11.interop.h. These functions don't seem to be available from managed code. Should I be looking at calling those functions directly via DllImport? Or is that something that Vortice could make available?

[amerkoleci]

Hi,
Thanks for your kind words, I've just added interop support for what you request with commit: ff37a81

Tell me if it works so I can bump new nuget beta release.

Thanks!

[RandyDavis99]

When I try that code, that Marshal.GetObjectForIUnknown lines gives me: "System.InvalidCastException: 'Unable to cast COM object of type 'System.__ComObject' to interface type 'Windows.Graphics.DirectX.Direct3D11.IDirect3DSurface'. This operation failed because the QueryInterface call on the COM component for the interface with IID '{0BF4A146-13C1-4694-BEE3-7ABF15EAF586}' failed due to the following error: No such interface supported (0x80004002 (E_NOINTERFACE)).'"

Your code doesn't make use of the returned iInspectable object which was surprising; although like I say I don't know much about how COM interacts with managed code so I won't be a ton of help debugging.

[RandyDavis99]

I can try to zip up my test project that demonstrates the code, if that helps.

[amerkoleci]

Indeed, my bad, you should use iInspectable.NativePointer, can you try?

[RandyDavis99]

iInspectable.NativePointer doesn't compile for me; SharpGen.Runtime.IInspectable has no members for me. Am I missing something there?

[amerkoleci]

This commit adds improved interop logic: 037663c

You can now call:

IDXGISurface surface = editTexture.QueryInterface<IDXGISurface>();
Windows.Graphics.DirectX.Direct3D11.IDirect3DSurface threeDSurface = CreateDirect3D11SurfaceFromDXGISurface<Windows.Graphics.DirectX.Direct3D11.IDirect3DSurface>(surface );

Can you try please?

[amerkoleci]

Share compilable project with me please?

[RandyDavis99]

Thanks for the updated code - I can now get an object that claims to be both a IDirect3DSurface and a IDirect3DDevice. When I call CanvasBitmap.CreateFromDirect3D11Surface, I am getting an error "System.InvalidCastException: 'Specified cast is not valid.'".

If I look at the call stack, it seems like it's trying to marshal something and not succeeding. Attached is a solution file which shows the issue. It sets up a SwapChainPanel and renders a white texture to it. When you click the button, it will call DXMeshTexture.GetCanvasBitmap, and you should wind up with the InvalidCast exception.

VorticeTest.zip

The call stack of the exception:

at System.Runtime.InteropServices.Marshal.ThrowExceptionForHR(Int32 errorCode)
at WinRT.ComWrappersSupport.CreateCCWForObject[T](Object obj, Guid iid)
at WinRT.MarshalInspectable1.CreateMarshaler(T o, Boolean unwrapObject) at WinRT.MarshalInterface1.CreateMarshaler(T value)
at ABI.Microsoft.Graphics.Canvas.ICanvasDeviceStatics.global::Microsoft.Graphics.Canvas.ICanvasDeviceStatics.CreateFromDirect3D11Device(IDirect3DDevice direct3DDevice)

[RandyDavis99]

I should note that even if I replace "CanvasDevice.CreateFromDirect3D11Device(d3dDevice)" with "CanvasDevice.GetSharedDevice()", as in my previous code sample, I get the same error. I am trying to research whether or not the device/surface has to be in some particular configuration for this to work but the documentation is scarce.

[amerkoleci]

Oh, you're using WinUI, which probably is different from UWP, not sure whats wrong, probably interop layer is different between UWP/WinUI, you should open issue/ask WinUI channel.

[RandyDavis99]

Got it, makes sense; it does point to something being incompatible with WinUI. I will try to find out if this scenario is supported yet, and even if it's not yet supported I now have the APIs from Vortice to make it work when it does. Thank you for all your help!

[amerkoleci]

Nope, ask if you have any further questions in future and share your work with me if you want!

[adstep]

@RandyDavis99 did you ever figure this out? Hitting the same issue.