Fast Frame Capture

[tolga]

Hi,

I'm trying to find a way to render the current frame to texture. I know there are a lot of ways to do it, but all of them drop FPS to half. I heard it is possible get the current buffer from ID3D11DeviceContext, but before going that route, I just wanted to confirm whether there is a method inside Unigine API.

Thanks.

[silent]

Hi Tolga,

We do have an example for the fast texture transfer using the CPU (with CUDA), but it's available only starting from Engineering editions.

Maybe @rohit.gonsalves can suggest anything related (but without access to the rendering context).

Thanks!

[tolga]

Hi Silent,

We are still using Unigine 2.10 and still have access to ID3D11DeviceContext via CustomApp and therefore, we can use it. Our management is still considering switching to engineering edition which will allow us to upgrade to newer version.

Thanks!

[rohit.gonsalves]

@silent Thanks for considering my long answers.

@tolga 

Quote

I'm trying to find a way to render the current frame to texture.

Please clarify this. You want to have current frame render to texture on GPU? or

You want to render the current frame to an image on CPU?

Do you need every frame to be captured? What processing do you want with captured data? Streaming, Recording, Encoding and sending to other devices?

Answers to these questions will help us to suggest you the best way.

Basically, following things are blocking ones FPS if capturing data to System Memory.

1] One performs everything in render thread. 

Solution: Don't do everything in render thread and send the data to other thread for saving or encoding etc.

2] One try to send big amount of data from GPU (VIDEO MEMORY) to SYSTEM MEMORY using DirectX map calls or UNIGINE API of Image/texture class. The first problem is waiting calls and second one is PCIE transfers of big data chunk.

Solution 1: Generally for video applications people do send GPU to System memory compressed data like YUV buffers which are half the bandwidth of RGBA. But in this case you lose transparency and it is not required in video domain. It is important because the map calls waits till the data transfer is finished. IF data is half the waiting will be over sooner.

Solution 2: NVIDIA Quadro cards provides special features GPU Direct where map calls don't wait and returns immediately for next rendering. But GPU Direct sends the required data to System memory and you can work with this data independently of rendering thread. Never slows down your FPS. This is done by special firmware of NVIDIA Quadro cards which accepts buffer and map calls returns, then another separate device context send the buffer to system memory location which was mapped by map call. Its internal but beneficial.

By the way I work with HD resolution and do 60 frames per second capture of UNIGINE rendering (GPU) to Video I/O cards (Through SYSTEM MEMORY) attached to my system.

With NVIDIA GPU Direct 4K resolutions 60p renders captured smoothly. Only there is one danger. NVIDIA Quadro cards are massively expensive making them not ready for retail consumer adaption.

What is your resolution and what do you want to do?

Regards,

Rohit

[tolga]

Hi Rohit,

Thanks for the comments. I'm trying to render to texture and display it on a big screen in the scene. It does not have to be live, it can be slower than the current FPS. Our output is currently 720p, maybe 1080p in the future. I will try a multi-threaded approach and try to sync with the current thread.

Thanks.

Edited November 2, 2020 by tolga

[rohit.gonsalves]

@tolga,

We thought that you need GPU to System Memory transfer. But this is not what you want. All your work is on GPU only.

1. You must be rendering some camera viewport to texture. (How are you doing it? I suppose using the renderTexture2D method of Viewport, don't you?

ViewportPtr viewport = Viewport::create();

TexturePtr texture = Texture::create();
texture->create2D(512,512,Texture::FORMAT_RGBA8,Texture::USAGE_RENDER); // create 512 x 512 render target

CameraPtr camera = Camera::create();
// set modelview & projection matrices to camera instance

// rendering

// saving current render state and clearing it 
RenderState::saveState();
RenderState::clearStates();
		
// rendering an image from the camera to the texture
viewport->renderTexture2D(camera, texture);

// restoring back render state
RenderState::restoreState();

2. In your second rendering you want to use the above texture onto BIG screen and render the final output.

If this is the things then please try to SKIP some rendering things onto 1 render with your viewport, like

m_objViewport->setSkipFlags(	Viewport::SKIP_POSTEFFECTS | 
									Viewport::SKIP_DYNAMIC_REFLECTIONS | 
									Viewport::SKIP_SHADOWS | 
									Viewport::SKIP_VELOCITY_BUFFER |
									Viewport::SKIP_FORMAT_RG11B10 |
									Viewport::SKIP_SRGB |
									Viewport::SKIP_TRANSPARENT );

Otherwise in single frame output you are rendering two frames with all post effects on.

I think with this strategy you will retain your FPS. But don't skip everything if you really need it in first render to texture.

I hope this helps.

Regards,

Rohit