Video Capture with MultiSampling in DirectX for Mortals (C#)

For the last 10 days, I've been tasked with fixing the company graphics engine's video capture component. I have enough DirectX experience to know that it exists, what it is, and what it looks like. But I have enough programming experience to be able to decipher just about any code in a language I can read. Thanks to Google, my former co-worker, Brendon, and my Italian heritage for getting me through this. Now, let me show you what might help you along this path.

There are a few things that should be mentioned before we get started: Fraps is an awesome tool that does this exact thing most likely better than I did it; We could not use Fraps in this case because the requirements for the video were that it have a constant, predefined frame rate; and the Direct X documentation is horrible in some cases. Also, I'm going to assume you can get a frame rendered. If you can't do that much, start somewhere else, then come back once you have that down.

So, Direct X has your RenderTarget and you want to save that info. And all you really want is for the InvalidCallExceptions to quit popping up and ruining your day with the message, "There is an error in your application." The technique I used was a combination of threading and copying the RenderTarget to an OffScreenPlainSurface in system memory.

I created an object called SurfaceCollector to handle collecting copying the surface into system memory and getting the GraphicsStream from its data. For every frame that came into my capture object, I copied the surface to that OffScreenPlainSurface and then queued the operation, SurfaceLoader.SaveToStream to run in its own thread. This allowed for the renderer to start working on the next frame while this was being stored to the video. I had another thread running to take the ready-to-use streams and save them to the video.

Copying proved difficult because I made the mistake of following Direct X's documentation. Since we're multisampling, you can't copy it directly. You have to create another RenderTarget without multisampling. Call StretchRectangle to copy data to the non-multisampled RenderTarget, but you must pass in a Rectangle that is the size of your surface; the documentation says you can pass in null, but C# won't allow structs to be null... Then you can call GetRenderTargetData to get the non-multisampled data into your surface in system memory.

The relevant code (slightly modified from my original to make it more relevant):

Create a Queue of the following object:

private class SurfaceCollector
{
    private        Surface        surface = null;
    private        Boolean        done = false;
    private        GraphicsStream stream = null;
    private static Rectangle      screenRect = Rectangle.Empty;

    public SurfaceCollector(Surface s)
    {
        //Set the screen rectangle if it's not already
        if (screenRect == Rectangle.Empty)
        {
            screenRect = new Rectangle(0, 0, s.Description.Width, s.Description.Height);
        }
        Copy(s);
    }

    /// 

    /// Copies the rendered surface to a non-multisampled surface
    /// then to an OffScreenPlainSurface in system memory.
    /// 
    /// 
    private void Copy(Surface s)
    {
        Surface tempSurface = Device.CreateRenderTarget(
            s.Description.Width,
            s.Description.Height,
            s.Description.Format,
            MultiSampleType.None,
            0,
            false
            );

        Device.StretchRectangle(s, screenRect, tempSurface, screenRect, TextureFilter.None);

        surface = Engine.Instance.Device.CreateOffscreenPlainSurface(
            s.Description.Width,
            s.Description.Height,
            s.Description.Format,
            Pool.SystemMemory
            );

        Device.GetRenderTargetData(tempSurface, surface);

        tempSurface.Dispose();
        tempSurface = null;
    }

    /// 

    /// Retrieves a GraphicsStream from the surface, call this method before accessing the Stream property.
    /// 
    public void GetStream()
    {
        try
        {
            stream = SurfaceLoader.SaveToStream(ImageFileFormat.Bmp, surface);
        }
        catch (Exception e)
        {
            //Handle your errors gracefully
        }
        done = true;
    }

    /// 

    /// Signals if the thread is done retrieving the stream
    /// 
    public Boolean Done
    {
        get { return done; }
    }

    /// 

    /// If a successful call to GetStream() has occured, return that stream,
    /// otherwise return null.
    /// 
    public GraphicsStream Stream
    {
        get
        {
            if (done)
                return stream;
            return null;
        }
    }

    /// 

    /// Frees all resources used by this object.
    /// 
    public void CleanUp()
    {
        if (surface != null && !surface.Disposed)
        {
            surface.Dispose();
            surface = null;
        }
        if (stream != null)
        {
            stream.Close();
            stream.Dispose();
            stream = null;
        }
    }
}
  

And the QueueProcessor Thread

private void ProcessQueue()
{
    while (true)
    {
        //Thread is done, break the loop
        if (waitingToClose && queue.Count == 0)
            break;
        while (queue.Count == 0 || !queue.Peek().Done)
        {
            //To prevent deadlock, check to see if we're waiting to finish
            if (waitingToClose)
                break;
            Thread.Sleep(25);
        }
        //To prevent deadlock, if we've left the previous loop and the queue is empty, we're done.
        if (queue.Count == 0)
            break;

        SurfaceCollector collector = queue.Dequeue();
        try
        {
            CaptureFrameFromGraphicsStream(collector.Stream);
        }
        catch (Direct3D.Direct3DXException ex)
        {
            //Handle your errors gracefully
        }
        finally
        {
            collector.CleanUp();
            collector = null;
            GC.Collect();
        }
    }
}
  

CreateFrameFromGraphicsStream() is a little too tied to the code base for me to put it here, but there are techniques all over the web for that part. It calls another method, AddFrame(). The technique for this is also all over the web.

I really hope this helps you in your endeavors.