viewport renderImage2d with Visualizer information

[Gmarquez]

Hi there!

I'm using the viewport::renderImage2d method to render the viewport into a image and later send this image with shared memory to another process.

But the visualizer information is not sended in the image2d, I guess this is because I'm getting the raw image from the camera, before the visualizer prints the additional information.

Is it possible to render an image with the visualizer information included?

Thanks!

Edited August 23, 2023 by Gmarquez

[silent]

Hi Guillermo,

Could you please show us your current code and when do you call renderImage2d()? Also it would be interesting to know the current SDK version that you are using for that task.

Thanks!

[Gmarquez]

Hi Silent,

The bad thing is that the SDK version is the 2.4.1 

The code that is doing the  image capture is the next:

void VideoSender::capture()
{
  if(mCamera.get() == nullptr)
  {
    mCamera = Unigine::Game::get()->getPlayer()->getCamera();
  }
  std::unique_lock<std::mutex> lock(mCaptureImageMutex);
  if(mIsFrozen == false && mIsNucEnabled == false)
  {
    mViewPort->renderImage2D(mCamera, mImage);
  }
  mIsReady = true;
  mCondition.notify_all();
  if(mProcessTexture)
  {
    mImage->flipY();
  }
  mIsWriteSharedMemoryInitialized = true;
}

For the image send we're doing this:

void VideoSender::sendImage()
{
  while(true)
  {
    if (mIsWriteSharedMemoryInitialized)
    {
      std::unique_lock<std::mutex> lock(mCaptureImageMutex);
      mCondition.wait(lock. [&] {return mIsReady.load()});
      if(mImage.get() != nullptr)
      {
        cv::Mat image = cv::Mat(util::UnigineUtils::HEIGHT, util::UnigineUtils::WIDTH,
                                mProcessTexture ? CV_8UC3 : CV_8UC4, mImage->getPixels());
        cv::cvtColor(image, image, CV_RGBA2RGB);
        mIsReady = false;
        mHeight= image.rows;
        mWidth= image.cols;
        mChannels = image.channels();
        std::memcpy(mRegionHeight.get_address(), &mHeight, mRegionHegiht.get_size());
        std::memcpy(mRegionWidth.get_address(), &mWidth, mRegionWidth.get_size());
        std::memset(mRegionChannels.get_address(), mChannels, mRegionChannels.get_size());
        int shmem_size = mHeight * mWidth * mChannels;
        mSharedMemoryVideo.truncate(shmem_size);
        mRegionVideo = boost::interprocess::mapped_region(mSharedMemoryVideo, boost::interprocess::read_write);
        memcpy(mRegionVideo.get_address(), image.ptr(), shmem_size);
      }
    }
  }
}

The send image method is launched in a separated thread at tthe VideoSender class init method

void VideoSender::init()
{
  mImage = Unigine::Image::create();
  mTexture = Unigine::Texture::create();
  mViewport = Unigine::Viewport::create();
  mViewport->setMode(Unigine::Viewport::MODE_DEFAULT);
  mViewport->setSkipflags(0);
  mBufferSize = util::UnigineUtils::WIDTH * util::Unigine::Utils::HEIGHT * mPixelSize;
  mBufferProcessSize = util::UnigineUtils::WIDTH, util::Unigine::Utils::HEIGHT, Unigine::Texture::FORMAT_RGBA8, Unigine::Texture::USAGE:_RENDER);
  initialize();
  mThread = boost::thread(&VideoSender::snedImage, this);
  mImage->create2s(util::UnigineUtils::WIDTH, util::Unigine::Utils::HEIGHT, mProcessTexture ? Unigine::Image::FORMAT_RGB8 : Unigine::Image::FORMAT_RGBA8);
}

 

The VideoSender::capture method is called in the render phase.

 

Sorry if you find any typo in the code snnipets, I must to hand copy the code because I cant copy code fron the IDE to the VM where the Internet browser is running.

Edited August 23, 2023 by Gmarquez

[sweetluna]

Hi,

the only way i could reproduce this behavior is when you are drawing visualizer after you call renderImage2D. Please check ordering of when you call render visualizer functions and when actual render. Our renderer is not deferred to a different thread that might know a lot of stuff post-factum.

The order should be:
1) Visualizer::get()->render ...(...)
2) viewport->renderImage2D(...);

If you swap them places you won't get visualizer into the rendered image.

Hope this helps!

[Gmarquez]

Thanks for your reply but I was assuming that the FPS which are shown through the Unigine.cfg file with the line <item name="show_fps" type="int">1</item> or the profiler information, were showed using the visualizer or something similar.

My mistake.

So now with fresh new information I can reformulate my question:

Is there a way to include the information show by the commands "show_fps" or "show_profiler" from the Unigine.cfg in the image captured from the viewport?

Edited August 23, 2023 by Gmarquez

[silent]

Unfortunately, in 2.4 SDK there is no way to capture texture with the performance profiler data and fps counter overlays using the API.

You should be able to at least get the profiler values as floats and after that draw them somehow on top of the final image if that's your final goal.

[Gmarquez]

Thanks, that could help