Fix FSAA in UI and lower VRAM consumption

[darksylinc]

PR NOTE 1

This fixes a regression from ign-rendering4.
In ign-rendering4 FSAA worked fine because it was relying on compositor scripts. This broke when the compositor script was replaced with C++ code.

The VRAM consumption issue affects both branches, but I think it's just fine to make it ign-rendering5 and onwards only

PR NOTE 2 (important)

How do I test the camera gaussian filter? It's affected directly (i.e. if renderTargetResultsIdx is set to the wrong value then the image won't look like the filter was applied, or some other similar issue) and I had no idea how to test it.

Update: CI says gaussian filter test failed. How shocking. I should check that test locally

Update 2: Found the bug and fixed it. Though DepthGaussianNoise test is still failing and I'm still looking into it

Update 3: DepthGaussianNoise is failing in my machine because the test is buggy. Opened Ticket #315

PR NOTE 3

Affected classes are mostly Ogre2RenderTarget, Ogre2RenderTexture and Ogre2DepthCamera

PR NOTE 4

Currently fixing ABI breakage.

Before vs After

Comments

FSAA was being requested however due to how the compositor was setup,
this effect was not taking effect.

Additionally, the Compositor setup was improved to lower memory
consumption. Originally the setup was taken from Ogre samples which
assume they will ultimately be rendering to a window.

However this is not the case and thus IGN was creating 3 render targets
(two for ping-ponging between postprocess FXs + one for storing the
final result)
This was optimized so that we only create 2 render targets: two for
ping-ponging between postprocess FXs and we pick at runtime which one is
storing the final result via the new variable renderTargetResultsIdx

Further performance optimizations could be made in Ogre 2.2 to improve
unnecessary MSAA resolving when doing postprocess, though considering
there's currently only one postprocessing effect (the Gaussian filter) I
doubt this optimization would make much of a difference

Signed-off-by: Matias N. Goldberg dark_sylinc@yahoo.com.ar

Summary

Checklist

• Signed all commits for DCO
• codecheck passed (See contributing)

Note to maintainers: Remember to use Squash-Merge

[darksylinc]

Converting this PR back to DRAFT since multiple issues were found

[darksylinc]

I fixed the issues I found so it's no longer a draft again.

However I wanted to see if the ABI checker would tell me what I broke if I broke something, but the ABI checker seems to be malfunctioning :(

Are there instructions on how to run it locally?

[iche033]

@osrf-jenkins run tests please

[iche033]

However I wanted to see if the ABI checker would tell me what I broke if I broke something, but the ABI checker seems to be malfunctioning :(

something is wring with this particular build machine. I've triggered the build again on a different machine:
https://build.osrfoundation.org/job/ignition_rendering-abichecker-any_to_any-ubuntu_auto-amd64/569/

Are there instructions on how to run it locally?

We don't have public instructions I think. We use abi-compliance-checker. You can probably extract the cmds that we use by looking at the console output of one of the successful ABI builds:
https://build.osrfoundation.org/job/ignition_rendering-abichecker-any_to_any-ubuntu_auto-amd64/564/console

abi-compliance-checker -lang C++ -lib ign-rendering -old pkg.xml -new devel.xml

The contents of pkg.xml and devel.xml are also in the console output.

[darksylinc]

We don't have public instructions I think. We use abi-compliance-checker. You can probably extract the cmds that we use by looking at the console output of one of the successful ABI builds:

Sigh. I guess I'll have to get my hands dirty. I hope it's easy. This tool is too useful for this project and my computer is vastly faster than the CI machine.

Thanks for highlighting the commands though

[darksylinc]

Apparently isRenderWindow is the only function that broke ABI. So I fixed it.

But I could use a 2nd pair of eyes.

If everything's ok it should be ready for merge

PS. I managed to use abi-compatibility-checker in my machine :) Thanks for the help. It just worked!

[iche033]

can you take a look at the latest ABI report from our CI? It found a few more issues:
https://build.osrfoundation.org/job/ignition_rendering-abichecker-any_to_any-ubuntu_auto-amd64/577/API_5fABI_20report/

Usually the Added Symbols issues are harmless. It's mainly the Removed Symbols and Problems with Data Types that cause ABI incompatibility.

[iche033]

capitalize I, i.e. IsRenderWindow

[iche033]

add doxygen comments for these new params

we'll need to keep the existing function since adding the args here breaks API/ABI.

[darksylinc]

Usually the Added Symbols issues are harmless. It's mainly the Removed Symbols and Problems with Data Types that cause ABI incompatibility.

Mmm....

Ogre2RenderTarget::UpdateRenderPassChain changed signature, so it counts as 1 removal, 1 add.
I can keep two copies, deprecated the old one, and make it throw (since the new code cannot work without the missing arguments). My questions are:

1. If client application is directly calling Ogre2RenderTarget::UpdateRenderPassChain, it will throw. Does this count as ABI breakage? The client app at this point was interfacing directly with a non-virtual function and AFAIK this is not guaranteed (by Gazebo) to stay stable.
2. Or is this just to fix potential unrelated ABI breakage? (i.e. removal of function causes unknown crashes in unrelated functions)

Removal of Ogre2RenderTexture::RenderTarget is controversial, because it is a virtual function. What happened was that this:

class Ogre2RenderTarget
{
  virtual OgrePointer* RenderTarget() = 0; // was pure
};

class Ogre2RenderTexture : public Ogre2RenderTarget
{
  virtual OgrePointer* RenderTarget() override;
};

Became this:

class Ogre2RenderTarget
{
  virtual OgrePointer* RenderTarget(); // no longer pure
};

class Ogre2RenderTexture : public Ogre2RenderTarget
{
  // Nothing here
};

From a vtable point of view nothing relevant changed (calling Ogre2RenderTarget::RenderTarget() should still work), but trying to directly call Ogre2RenderTexture::RenderTarget() (i.e. bypassing the vtable) would fail.

I can play on the safe side here and reimplement Ogre2RenderTexture::RenderTarget.

Btw I'm a bit worried about this error:

1. The layout of v-table has been changed for unknown reason. Call of any method in this class may result in crash or incorrect behavior of applications. is this normal?

[iche033]

If client application is directly calling Ogre2RenderTarget::UpdateRenderPassChain, it will throw. Does this count as ABI breakage? The client app at this point was interfacing directly with a non-virtual function and AFAIK this is not guaranteed (by Gazebo) to stay stable.

We can deprecate old function but instead of making it throw, just print a warning msg to tell users to use the new function (Ignition has a no throw policy which is probably not well documented). We can do something like this to prevent the warning msg being printed out repeatedly: https://github.com/ignitionrobotics/ign-gazebo/blob/ign-gazebo5/include/ignition/gazebo/components/Component.hh#L124-L132

Or is this just to fix potential unrelated ABI breakage? (i.e. removal of function causes unknown crashes in unrelated functions)

In the UpdateRenderPassChain particular case, I think it would be less of a problem. Other issues related to vtable changes and data types found by the ABI build is likely to cause unknown / undefined behavior and crashes.

The layout of v-table has been changed for unknown reason. Call of any method in this class may result in crash or incorrect behavior of applications

We've seen this before and the consensus is that it that we need to fix this issue to avoid unexpected crashes. My guess is that it's related to changes done to the RenderTarget() function.

I can play on the safe side here and reimplement Ogre2RenderTexture::RenderTarget

This is one thing I would try first to see if it helps fix the v-table issue.

The ABI checker also picked up issues with the new ogreTexture[2] var and the change to existing variable materialApplicator which caused changes in memory layout.

Fixing ABI is not fun and usually involves many workarounds :/ I can also take a look at fixing some of these ABI issues later this week if you haven't gotten to it yet.

[darksylinc]

I updated the code to be ABI compatible.

Maaan I'm beat. This is 20% work, 80% figuring out ABI issues.
If this keeps up I'll just target a newer version.

My ABI checker says 100% compatibility. Let's cross fingers CI agrees.

[iche033]

this should get rid of the build warnings:

diff --git a/ogre2/src/Ogre2RenderTarget.cc b/ogre2/src/Ogre2RenderTarget.cc
index a0e8fda6..59b66397 100644
--- a/ogre2/src/Ogre2RenderTarget.cc
+++ b/ogre2/src/Ogre2RenderTarget.cc
@@ -647,10 +647,11 @@ void Ogre2RenderTarget::UpdateRenderPassChain()
 
 //////////////////////////////////////////////////
 void Ogre2RenderTarget::UpdateRenderPassChain(
-    Ogre::CompositorWorkspace *_workspace, const std::string &_workspaceDefName,
-    const std::string &_baseNode, const std::string &_finalNode,
-    const std::vector<RenderPassPtr> &_renderPasses,
-    bool _recreateNodes)
+    Ogre::CompositorWorkspace * /*_workspace*/,
+    const std::string & /*_workspaceDefName*/,
+    const std::string & /*_baseNode*/, const std::string & /*_finalNode*/,
+    const std::vector<RenderPassPtr> & /*_renderPasses*/,
+    bool /*_recreateNodes*/)
 {
   ignwarn << "Warning: This Ogre2RenderTarget::UpdateRenderPassChain "
           << "overload is deprecated" << std::endl;
@@ -856,9 +857,12 @@ void Ogre2RenderTarget::SyncOgreTextureVars()
 //////////////////////////////////////////////////
 // Ogre2RenderTexture
 //////////////////////////////////////////////////
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
 Ogre2RenderTexture::Ogre2RenderTexture()
 {
 }
+#pragma GCC diagnostic pop
 
 //////////////////////////////////////////////////
 Ogre2RenderTexture::~Ogre2RenderTexture()
@@ -917,7 +921,10 @@ Ogre::RenderTarget *Ogre2RenderTexture::RenderTarget() const
 //////////////////////////////////////////////////
 void Ogre2RenderTexture::SetOgreTexture(Ogre::Texture *_ogreTexture)
 {
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
   this->ogreTexture = _ogreTexture;
+#pragma GCC diagnostic pop
 }
 
 /////////////////////////////////////////////////

[iche033]

ABI report: https://build.osrfoundation.org/job/ignition_rendering-abichecker-any_to_any-ubuntu_auto-amd64/584/API_5fABI_20report/

looking at the data type problem description and the vtable, I think the issue could be due to Ogre2RenderTarget::RenderTarget() no longer being pure virtual. I don't think there's a behavior change since the logic is just moved to the base class and we are not re-implementing a virtual function. This may be ok.

[iche033]

Confirmed that the FSAA issue is fixed. Thanks for iterating!

Maaan I'm beat. This is 20% work, 80% figuring out ABI issues.
If this keeps up I'll just target a newer version.

sure, if the changes are looking like they are not trivial to make to a release version without breaking ABI, you can target main first. We would then evaluate afterwards to see if it's worth backporting them.

[darksylinc]

looking at the data type problem description and the vtable, I think the issue could be due to Ogre2RenderTarget::RenderTarget() no longer being pure virtual. I don't think there's a behavior change since the logic is just moved to the base class and we are not re-implementing a virtual function. This may be ok.

I suspect the same. It passed on my machine, but it seems it's not consistent. Just to be sure I made it pure virtual again and let's see what the abi checker says this time.

[iche033]

latest ABI build is green :)

the INTEGRATION_render_pass test failed in the latest ubuntu build. I haven't seen that before. I just retriggered another build and it failed again. I haven't been able to reproduce it locally - could be a timing issue. This led me to try out running the ogre2_demo in examples again and noticed that the mouse movement is laggy / jumpy. See gif below. I don't see this happen on ign-rendering5 branch. If I enable the noise render pass (post processing) effect by pressing P, the lag is gone, not sure why

[darksylinc]

the INTEGRATION_render_pass test failed in the latest ubuntu build
It's been failing randomly (but quite frequently) on my machine. I was gonna take a look at some point as I feared it could be a problem with the PostRender branch, but interesting to see that it repros here instead!

It may mean that this patch accidentally tries to show the wrong texture, containing uninitialized or old results, which would explain why the error fails (i.e. it contains whatever was in memory)

This led me to try out running the ogre2_demo in examples again and noticed that the mouse movement is laggy / jumpy

This further fuels this theory, if the same texture is shown twice. Definitely worth taking a look. Thanks!

[darksylinc]

I'm afraid I couldn't repro any of your problems:

1. INTEGRATION_render_pass passes
2. ogre2_demo does not lag

I tried ogre2_demo on the following machines:

1. AMD Ryzen 5900X + AMD Radeon RX 560 2GB (mesa drivers) + Ubuntu 18.04 LTS (5.11.0-051100-generic)
2. Intel i7 7700 + AMD Radeon HD 7770 (mesa drivers) + Ubuntu 20.04 LTS (5.8.0-50-generic)
3. Intel i7 7700 + NVIDIA GeForce 1060 3GB (proprietary drivers) + Ubuntu 20.04 LTS (5.8.0-50-generic)

I noticed the Intel i7 machine keeps randomly complaining about a double free / mem. corruption (sometimes it complains at startup and crashes before the windows comes up, other times it complains during shutdown, other times it doesn't complain at all) but that may have been some build mistake which I am going to make a clean build now to be sure. If problem persists it may be worth looking into for me.

But overall I could not repro none of the two problems :(

Update: i7 machine still compiling, ran Valgrind on the AMD machine and it found 2 things worth looking into (may not be related with my changes):

Invalid write of size 8
  in initCamera(std::shared_ptr<ignition::rendering::v5::Camera>) in /home/matias/Projects/ign/src/ign-rendering/examples/ogre2_demo/GlutWindow.cc:331
  1: ignition::rendering::v5::Ogre2RenderTarget::UpdateRenderPassChain(Ogre::CompositorWorkspace*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::shared_ptr<ignition::rendering::v5::RenderPass>, std::allocator<std::shared_ptr<ignition::rendering::v5::RenderPass> > > const&, bool, Ogre::Texture* (*) [2], bool) in /home/matias/Projects/ign/src/ign-rendering/ogre2/src/Ogre2RenderTarget.cc:767
  2: ignition::rendering::v5::Ogre2RenderTarget::UpdateRenderPassChain() in /home/matias/Projects/ign/src/ign-rendering/ogre2/src/Ogre2RenderTarget.cc:622
  3: ignition::rendering::v5::Ogre2RenderTarget::PreRender() in /home/matias/Projects/ign/src/ign-rendering/ogre2/src/Ogre2RenderTarget.cc:416
  4: ignition::rendering::v5::Ogre2RenderTexture::PreRender() in /home/matias/Projects/ign/src/ign-rendering/ogre2/src/Ogre2RenderTarget.cc:906
  5: ignition::rendering::v5::BaseCamera<ignition::rendering::v5::Ogre2Sensor>::PreRender() in /home/matias/Projects/ign/src/ign-rendering/include/ignition/rendering/base/BaseCamera.hh:311
  6: ignition::rendering::v5::BaseVisual<ignition::rendering::v5::Ogre2Node>::PreRenderChildren() in /home/matias/Projects/ign/src/ign-rendering/include/ignition/rendering/base/BaseVisual.hh:335
  7: ignition::rendering::v5::BaseNode<ignition::rendering::v5::Ogre2Object>::PreRender() in /home/matias/Projects/ign/src/ign-rendering/include/ignition/rendering/base/BaseNode.hh:285
  8: ignition::rendering::v5::BaseVisual<ignition::rendering::v5::Ogre2Node>::PreRender() in /home/matias/Projects/ign/src/ign-rendering/include/ignition/rendering/base/BaseVisual.hh:306
  9: ignition::rendering::v5::BaseScene::PreRender() in /home/matias/Projects/ign/src/ign-rendering/src/base/BaseScene.cc:1235
  10: ignition::rendering::v5::Ogre2Scene::PreRender() in /home/matias/Projects/ign/src/ign-rendering/ogre2/src/Ogre2Scene.cc:168
  11: ignition::rendering::v5::BaseCamera<ignition::rendering::v5::Ogre2Sensor>::Update() in /home/matias/Projects/ign/src/ign-rendering/include/ignition/rendering/base/BaseCamera.hh:388
  12: ignition::rendering::v5::BaseCamera<ignition::rendering::v5::Ogre2Sensor>::Capture(ignition::rendering::v5::Image&) in /home/matias/Projects/ign/src/ign-rendering/include/ignition/rendering/base/BaseCamera.hh:397
  13: initCamera(std::shared_ptr<ignition::rendering::v5::Camera>) in /home/matias/Projects/ign/src/ign-rendering/examples/ogre2_demo/GlutWindow.cc:331
  14: run(std::vector<std::shared_ptr<ignition::rendering::v5::Camera>, std::allocator<std::shared_ptr<ignition::rendering::v5::Camera> > >) in /home/matias/Projects/ign/src/ign-rendering/examples/ogre2_demo/GlutWindow.cc:379
  15: main in /home/matias/Projects/ign/src/ign-rendering/examples/ogre2_demo/Main.cc:334
Address 0xe42fb38 is 0 bytes after a block of size 184 alloc'd  1: operator new(unsigned long) in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so
  2: ignition::rendering::v5::Ogre2RenderTarget::Ogre2RenderTarget() in /home/matias/Projects/ign/src/ign-rendering/ogre2/src/Ogre2RenderTarget.cc:123
  3: ignition::rendering::v5::Ogre2RenderTexture::Ogre2RenderTexture() in /home/matias/Projects/ign/src/ign-rendering/ogre2/src/Ogre2RenderTarget.cc:862
  4: ignition::rendering::v5::Ogre2Scene::CreateRenderTextureImpl(unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) in /home/matias/Projects/ign/src/ign-rendering/ogre2/src/Ogre2Scene.cc:885
  5: ignition::rendering::v5::BaseScene::CreateRenderTexture() in /home/matias/Projects/ign/src/ign-rendering/src/base/BaseScene.cc:1165
  6: ignition::rendering::v5::Ogre2Camera::CreateRenderTexture() in /home/matias/Projects/ign/src/ign-rendering/ogre2/src/Ogre2Camera.cc:185
  7: ignition::rendering::v5::Ogre2Camera::Init() in /home/matias/Projects/ign/src/ign-rendering/ogre2/src/Ogre2Camera.cc:156
  8: ignition::rendering::v5::Ogre2Scene::InitObject(std::shared_ptr<ignition::rendering::v5::Ogre2Object>, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) in /home/matias/Projects/ign/src/ign-rendering/ogre2/src/Ogre2Scene.cc:928
  9: ignition::rendering::v5::Ogre2Scene::CreateCameraImpl(unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) in /home/matias/Projects/ign/src/ign-rendering/ogre2/src/Ogre2Scene.cc:675
  10: ignition::rendering::v5::BaseScene::CreateCamera(unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) in /home/matias/Projects/ign/src/ign-rendering/src/base/BaseScene.cc:755
  11: ignition::rendering::v5::BaseScene::CreateCamera(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) in /home/matias/Projects/ign/src/ign-rendering/src/base/BaseScene.cc:750
  12: buildScene(std::shared_ptr<ignition::rendering::v5::Scene>) in /home/matias/Projects/ign/src/ign-rendering/examples/ogre2_demo/Main.cc:263
  13: createCamera(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) in /home/matias/Projects/ign/src/ign-rendering/examples/ogre2_demo/Main.cc:286
  14: main in /home/matias/Projects/ign/src/ign-rendering/examples/ogre2_demo/Main.cc:322

Mismatched free() / delete / delete []
  in run(std::vector<std::shared_ptr<ignition::rendering::v5::Camera>, std::allocator<std::shared_ptr<ignition::rendering::v5::Camera> > >) in /home/matias/Projects/ign/src/ign-rendering/examples/ogre2_demo/GlutWindow.cc:392
  1: operator delete(void*) in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so
  2: std::_Sp_counted_ptr<unsigned char*, (__gnu_cxx::_Lock_policy)2>::_M_dispose() in /usr/include/c++/8/bits/shared_ptr_base.h:377
  3: std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() in /usr/include/c++/8/bits/shared_ptr_base.h:155
  4: std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() in /usr/include/c++/8/bits/shared_ptr_base.h:728
  5: std::__shared_ptr<unsigned char, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() in /usr/include/c++/8/bits/shared_ptr_base.h:1167
  6: std::shared_ptr<unsigned char>::~shared_ptr() in /usr/include/c++/8/bits/shared_ptr.h:103
  7: ignition::rendering::v5::Image::~Image() in /home/matias/Projects/ign/src/ign-rendering/src/Image.cc:34
  8: void __gnu_cxx::new_allocator<ignition::rendering::v5::Image>::destroy<ignition::rendering::v5::Image>(ignition::rendering::v5::Image*) in /usr/include/c++/8/ext/new_allocator.h:140
  9: void std::allocator_traits<std::allocator<ignition::rendering::v5::Image> >::destroy<ignition::rendering::v5::Image>(std::allocator<ignition::rendering::v5::Image>&, ignition::rendering::v5::Image*) in /usr/include/c++/8/bits/alloc_traits.h:487
  10: std::_Sp_counted_ptr_inplace<ignition::rendering::v5::Image, std::allocator<ignition::rendering::v5::Image>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() in /usr/include/c++/8/bits/shared_ptr_base.h:554
  11: std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() in /usr/include/c++/8/bits/shared_ptr_base.h:155
  12: std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() in /usr/include/c++/8/bits/shared_ptr_base.h:728
  13: std::__shared_ptr<ignition::rendering::v5::Image, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() in /usr/include/c++/8/bits/shared_ptr_base.h:1167
  14: std::shared_ptr<ignition::rendering::v5::Image>::~shared_ptr() in /usr/include/c++/8/bits/shared_ptr.h:103
  15: __run_exit_handlers in /build/glibc-S9d2JN/glibc-2.27/stdlib/exit.c:108
  16: exit in /build/glibc-S9d2JN/glibc-2.27/stdlib/exit.c:139
  17: glutMainLoopEvent in /usr/lib/x86_64-linux-gnu/libglut.so.3.9.0
  18: glutMainLoop in /usr/lib/x86_64-linux-gnu/libglut.so.3.9.0
  19: run(std::vector<std::shared_ptr<ignition::rendering::v5::Camera>, std::allocator<std::shared_ptr<ignition::rendering::v5::Camera> > >) in /home/matias/Projects/ign/src/ign-rendering/examples/ogre2_demo/GlutWindow.cc:392
  20: main in /home/matias/Projects/ign/src/ign-rendering/examples/ogre2_demo/Main.cc:334
Address 0x3aff8080 is 0 bytes inside a block of size 1,440,000 alloc'd  1: operator new[](unsigned long) in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so
  2: ignition::rendering::v5::Image::Image(unsigned int, unsigned int, ignition::rendering::v5::PixelFormat) in /home/matias/Projects/ign/src/ign-rendering/src/Image.cc:30
  3: ignition::rendering::v5::BaseCamera<ignition::rendering::v5::Ogre2Sensor>::CreateImage() const in /home/matias/Projects/ign/src/ign-rendering/include/ignition/rendering/base/BaseCamera.hh:381
  4: initCamera(std::shared_ptr<ignition::rendering::v5::Camera>) in /home/matias/Projects/ign/src/ign-rendering/examples/ogre2_demo/GlutWindow.cc:329
  5: run(std::vector<std::shared_ptr<ignition::rendering::v5::Camera>, std::allocator<std::shared_ptr<ignition::rendering::v5::Camera> > >) in /home/matias/Projects/ign/src/ign-rendering/examples/ogre2_demo/GlutWindow.cc:379
  6: main in /home/matias/Projects/ign/src/ign-rendering/examples/ogre2_demo/Main.cc:334

Update 2:

The invalid write is a new bug introduced by me.
The new [] / delete operator mismatch is a pre-existing bug.

Update 3:

ASAN discovered another bug. in Ogre2SelectionBuffer::CreateRTTBuffer this->dataPtr->buffer = new uint8_t[bufferSize]; has bufferSize = 3, but later on we read 4 bytes from it (PF_RGB strikes again).
This is not caused by this PR, so I'll create another PR for that

[darksylinc]

@iche033 Although I could not repro the bug you mentioned, I fixed a heap corruption that could definitely cause the behavior you were seeing on your machine. Could you try again?

As for the other two bugs I found, they're unrelated to my fixes so I'll create a PR. I think I'll split it into two more PRs because both are simple, but one of them may break ABI (I'm not sure how we can avoid ABI breakage there, the root of the problem is in the syntax declaration)

[codecov]

Codecov Report

Merging #313 (81a1263) into ign-rendering5 (e452c39) will increase coverage by 5.28%.
The diff coverage is 71.01%.

❗ Current head 81a1263 differs from pull request most recent head 2e49d36. Consider uploading reports for the commit 2e49d36 to get more accurate results

@@                Coverage Diff                 @@
##           ign-rendering5     #313      +/-   ##
==================================================
+ Coverage           52.52%   57.80%   +5.28%     
==================================================
  Files                 143      161      +18     
  Lines               13274    15851    +2577     
==================================================
+ Hits                 6972     9163    +2191     
- Misses               6302     6688     +386     

Impacted Files	Coverage Δ
include/ignition/rendering/ArrowVisual.hh	100.00% <ø> (ø)
include/ignition/rendering/AxisVisual.hh	100.00% <ø> (ø)
include/ignition/rendering/Camera.hh	100.00% <ø> (ø)
include/ignition/rendering/CompositeVisual.hh	100.00% <ø> (ø)
include/ignition/rendering/Geometry.hh	100.00% <ø> (ø)
include/ignition/rendering/GpuRays.hh	0.00% <ø> (ø)
include/ignition/rendering/Image.hh	100.00% <ø> (ø)
include/ignition/rendering/Light.hh	100.00% <ø> (ø)
include/ignition/rendering/Material.hh	100.00% <ø> (ø)
include/ignition/rendering/Mesh.hh	100.00% <ø> (ø)
... and 131 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1421901...2e49d36. Read the comment docs.

[iche033]

ok thanks for looking into the ogre2_demo problem and fixing the memory issues.

I did more testing and found that I was also able to reproduce the problem in latest ign-rendering4 branch so it's not introduced by this PR, sorry for the noise! I tested with the 2 memory fixes but the problem is still there. I ticketed #341 with more info added there. I think it came down to high GPU usage that's causing the lag. Surprisingly, the GPU usage drops when post-processing effect enabled - I would have thought that it would go up instead of down.

In any case, since it's not introduced by this PR, the changes look good here. The INTEGRATION_render_pass test is also passing in the last two builds. I think this is good to go.

[darksylinc]

Yyyyeeeeessss!!

This PR was starting to drive me a little nuts.

I think it came down to high GPU usage that's causing the lag

Ahh!!! This reminds me that I noticed Ogre2RenderTarget::UpdateRenderPassChain ended up calling recreateAllNodes every frame. It didn't seem like that's intended behavior because it should notice nothing changed and hence early out.

That could be causing the lag. I was going to investigate that, then forgot.

Apparently it would always hit this:

// check if we need to create all nodes or just update the connections.
// if node does not exist then it means it has not been added to the
// chain yet, in which case, we need to recreate the nodes and
// connections
if (!node)
{
  _recreateNodes = true; // --> Always reaches here
}

[iche033]

Ahh!!! This reminds me that I noticed Ogre2RenderTarget::UpdateRenderPassChain ended up calling recreateAllNodes every frame.

that's it! I've created a PR to fix it: #342

[darksylinc]

that's it! I've created a PR to fix it: #342

Awesome! I would've probably taken longer to figure that out.

And yeah, it makes sense because shadow node's textures kept being recreated. Resource creation is expensive