Cool VL Viewer forum http://sldev.free.fr/forum/ |
|
Discard bias yo-yo (was: Perf regression in populated areas) http://sldev.free.fr/forum/viewtopic.php?f=6&t=2343 |
Page 1 of 1 |
Author: | GismoZ [ 2023-02-13 22:46:46 ] |
Post subject: | Discard bias yo-yo (was: Perf regression in populated areas) |
Performance regression after 1.30.0.34 in heavy populated areas, seemingly seems re-loading textures after initial loading Description: I seen a performance regression in all viewers later than the mentioned version in heavy populated sites (eg Peak Club, best to see > 50 characters). The initial loading of texture (ctrl+shift+3) seems same. For me that is 2-3 minutes. After this the behavior is changing - while 1.30.0.34 is very "quiet" in textures loading, only new arrivals / particles seems, the later versions seems periodically (for me 1-3 minutes) also to reload already successful loaded avatars / textures. I can also visibly see characters quickly vanish partially and reappearing (because reloading its textures?). I use cpu affinity settting and see all non dedicated core continue running 100% texture loading, instead 1.30.0.34 load is then according to new arrivals it seems. Settings are the same for all versions tested. I use light & shadows. I will investigate that further, to get a better descreption to reproduce, or helpful details. Until now - How to reproduce: go into a heavy populated area like PEAK club, best 50+ characters, [EDIT] cam out so you see many characters [/EDIT] wait initial loading all textures etc. is finished. Wait more, and see a periodically reloading (ctrl+shift+3) and fps drop during reloading. Not seen that with 1.30.0.34. I suspect this change, but my data basis is very weak: "Improved the texture discard bias change algorithm to minimize the occurrences of freezes in extreme bound GL textures usage conditions while turning your avatar/camera around." [EDIT] And VRAM is always less than 90% as i set texture usage of memory to moderate 2,4 GB[/EDIT] Cheers & sorry for the poor information, but i think better to report than wait more CPU: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz (4120.81 MHz) Memory: 32038MB OS version: Linux-x86_64 v5.14.21-150400.24.41-default #1 SMP PREEMPT_DYNAMIC Fri Jan 13 08:55:22 UTC 2023 (1d4442d) Memory manager: jemalloc v5.3.1-20221122 Graphics card vendor: NVIDIA Corporation Graphics card: NVIDIA GeForce GTX 1070/PCIe/SSE2 OpenGL version: 4.6.0 NVIDIA 525.78.01 Detected VRAM: 8192MB J2C decoder: OpenJPEG: 1.4.0.635f Audio driver: FMOD Studio v2.02.11 (ALSA) |
Author: | Henri Beauchamp [ 2023-02-13 23:33:39 ] |
Post subject: | Re: Discard bias yo-yo (was: Perf regression in populated ar |
This is not a performance regression, but on the contrary an improvement. Go to the same club with v1.30.0.34. Wait for everything to load. Turn your avatar around and observe the HUGE hiccup with single-digit frame rates for several seconds. Do the same with the current release: no hiccup, or a moderate one only. This issue is particularly annoying when walking around in highly textured environments (scenic sims, sims with a sh*tload of materials, etc), and it will only get worst with the future PBR renderer (up to 5 textures per face: diffuse texture (for compatibility with old renderers) + 4 PBR channels). What happened ?... When the GL bound textures overshoots 3GB or so, NVIDIA cards choke big time, until the usage of this type of memory regresses below 2.5GB, and when turning your avatar around, you increase suddenly the bound textures number as new objects come into your camera FOV, while others that are no more in it have not yet been removed from memory and their bound textures freed. What I did in v1.30.0.35 ? I changed the texture discard bias algorithm to properly keep the bias at the same level as long as the bound texture memory usage is high (the "soft limit" is set to 2.5GB and can be adjusted via the "MaxBoundTexMem" debug setting), and I smoothed out the decrease in the bias so to avoid sudden changes, e.g. form 4.0 to 0.0 and then suddenly to 5.0 when you turn around, with large overshoots way above 3GB of bound textures. In the texture console, observe the "Bias" number as well as the "Bound" figure, and you will see what I mean. Now, the side effect is that the viewer takes more time to settle at an average bias that suits the textures usage, thus the reloading of textures after a few minutes or when people arrive/leave in your FOV. When the bias changes, the LOD of textures change as well, and the said textures must be re-decoded at the new LOD, thus the CPU usage spike. Of course, the algorithm is not perfect, but I find it much smoother/better than what we previously had... |
Author: | Henri Beauchamp [ 2023-02-15 10:48:45 ] |
Post subject: | Re: Discard bias yo-yo (was: Perf regression in populated ar |
I made some experiments, in crowded areas, zooming out like you did. In these conditions, one of the causes for constant re-decoding of textures is the avatar "impostoring" feature: when zoomed out, the variation in the distance to the camera of the various avatars is much smaller, and causes "random" changes in avatars impostoring (i.e. for each avatar, the impostor status could change every few seconds), incurring attachments textures re-decoding at various LoD levels as well as the corresponding bound/unbound GL textures changes; since, nowadays, almost everything in an avatar is an attachment (and most of the time, with materials, i.e. with up to 3 textures per face), things can get messy... One potential workaround for such cases is to turn impostors off (move the slider to the extreme left in the graphics settings); of course, this means more avatars to fully render and thus lower frame rates... I also found out that it might be worth changing your "Texture memory" setting to lower it a bit, in such conditions (I know, it sounds counterintuitive)... Finally, I further improved the discard bias algorithm to reduce spikes, and I am currently exploring a new, more reliable method for the bound textures accounting (the one used by the PBR project viewer), that seems also to improve things quite a bit... So, stay tuned for next release. ![]() |
Author: | GismoZ [ 2023-02-15 23:40:14 ] |
Post subject: | Re: Discard bias yo-yo (was: Perf regression in populated ar |
I did some more testing and if i understood you right, to simplify it - newer viewer versions just need more time to to balance all. I really stayed a long time in a heavy populated area,using camera carefully while looking at textures loads too. I still have the "better" behavior what you described as intended with 1.30.0.34. its less load on the cores used for texture loading, and stable fps after initial loading (ofc peaks dropping fps when start to cam around but normalizes soon). Even latest version (last Saturday) then after a while stays average circa 50% fps of 1.30.0.34. I will test more try to find out what influences it in my setup. Btw - would it be possible maybe get the old code inside again and make it selectable to be used by a debug setting? I am looking forward to see what you found out and will incooperate into next viewer version. Thank You so much for always keeping CoolVL Viewer ahead. |
Author: | Henri Beauchamp [ 2023-02-18 14:03:07 ] |
Post subject: | Re: Discard bias yo-yo (was: Perf regression in populated ar |
Today's release should largely solve the yo-yo effect (that, by the way, v1.30.0.34 was not immune against, by far: it simply happened under different conditions). In this release, I reworked the texture bias change and related algorithms as follow:
|
Author: | GismoZ [ 2023-02-22 22:30:42 ] |
Post subject: | Re: Discard bias yo-yo (was: Perf regression in populated ar |
I just make it short: great work done! It combines the performance gains already made also in heavy populated areas. I did test for some days, prefect. Even more potential to fine tune as parameters get available by debug settings (I won't do now its just working fine with the defaults). P.S. a short description what exactly the Bias setting does would be useful as i didn't found any good reference by now (just an idea). Big Thank's ! |
Author: | Henri Beauchamp [ 2023-02-22 23:07:24 ] | |||||||||
Post subject: | Re: Discard bias yo-yo (was: Perf regression in populated ar | |||||||||
|
Author: | Henri Beauchamp [ 2023-03-11 10:54:22 ] | |||||||||
Post subject: | Re: Discard bias yo-yo (was: Perf regression in populated ar | |||||||||
In today's v1.30.2.4 release I added:
|
Author: | Henri Beauchamp [ 2023-04-01 09:55:28 ] |
Post subject: | Re: Discard bias yo-yo (was: Perf regression in populated ar |
In today's release, I improved again the texture discard bias algorithm by using the last memory usage to decide whether the bias should be increased or decreased, in excess of the current memory usage, i.e. extrapolating figures for next update. This helps a great deal to avoid entering a yo-yo by avoiding (sometimes large) overshoots while decreasing or increasing the discard bias. I also added a new "TextureUpdateBoostRatioPerDiscard" debug setting to adjust how much more textures we update per frame proportionally to the discard bias value (this setting is used to re-decode more textures at the new bias and free memory faster), but reduced its former (hard-coded) value from 1.0 to 0.25 (*) (the higher the value, the larger the impact on the frame rate), since with last weeks' changes, we also now force-free memory on discard bias changes, reducing the need for this feature. I also added an "Upd/frame" (texture decode priority updates per frame) stat to the texture console. (*) I.e. at discard 5.0 we decode 2.25 times more textures instead of 6.0 times more (the multiplier equals to 1.0 + discard_bias * TextureUpdateBoostRatioPerDiscard). |
Author: | Henri Beauchamp [ 2024-10-26 09:28:12 ] |
Post subject: | Re: Discard bias yo-yo (was: Perf regression in populated ar |
I yet again improved the texture discard bias algorithm in v1.32.2.20. The goal is to try and avoid as much as possible sudden bias jumps to 5.0, when hitting the low free VRAM limit, and to further reduce the yo-yo effect in low free VRAM conditions. The discard bias is now influenced by three, distinct mechanisms:
Note that, in v1.32.2.20, any change to the various settings impacting these mechanisms (VRAM spinner, Texture memory slider, and various debug settings) are taken into account in real time, allowing you to adjust them to best fit your needs. For example, if you want to run simultaneously two sessions of the viewer and not see texture trashing happening, you may want to artificially limit the VRAM amount in the corresponding spinner to half of your actual VRAM... The texture console was also changed: the frame buffers memory usage was added ("FB:" number, in MB), and the "Free VRAM" number is now only shown when the corresponding OpenGL call exists and you did not disable the VRAM checks from the settings; the free VRAM number also sees an exclamation mark printed next to it when the minimum free VRAM margin is reached. |
Page 1 of 1 | All times are UTC |
Powered by phpBB® Forum Software © phpBB Group https://www.phpbb.com/ |