LL is crazy to do this: all but powerful PCs will be bared from using Second Life…
Dave Parksauthor
This is true with the things we tend to enable implicitly with
the ALM checkbox like shadows, AO, and an infinite number of lights, but
it’s not true generally. We’ll be much better off if we divert
resources that would have gone to maintaining the forward render shaders
to making sure the deferred rendering shaders perform well on low end
hardware.
Edit to note: This move has been discussed extensively with third
party viewer developers and content creators and is generally seen as a
good thing, particularly among content creators. Having to support both
render modes is a significant burden both on developers and content
creators, and the only argument for keeping forward rendering is based
on a fallacy. Deferred rendering is not inherently slower than forward
rendering, we just treat “deferred” as synonmous with “high detail” in
the viewer.
Henri Beauchamp
This move has been discussed extensively with third party viewer developers and content creators
Sadly in Voice and thus excluding me, a TPV developers who has been
dealing with his own viewer for the past 15 years, who has seen the move
to Windlinght, then the one to Extended Environment (and who produced
and maintained a dual renderer viewer (WL+EE) until, at last the EE
shaders and render pipeline got fixed and matched WL speed, with the
performances viewer changes).
the only argument for keeping forward rendering is based on a fallacy
The fallacy is to deny FACTS: forward rendering is up to twice as
fast as deferred rendering with iGPUs and APUs, for example. Denying it
is lying to people !
I am right out FURIOUS to see that much BAD FAITH and, sorry, but
that much stupid of a stance too (you are going to make any non-gamer PC
owners experience miserable in SL) !
Dave Parksauthor
I'm happy to talk to you via text (Discord is my preferred chat client, and
that's where most of the discussions are taking place outside the user
groups). You may be correct, and I value the feedback and appreciate your
perspective and contributions, but I think there's some information you're
missing. It is true that forward rendering can produce better
performance than deferred, and historically the forward renderer in SL has
performed as well or better than the deferred renderer, but with a little
work on optimizing the deferred renderer, I believe it can be made to
perform better than the forward renderer does today, and those efforts will
benefit all users, not just those on low end hardware, and will give
everyone the best experience possible. Examples of optimizations here
involve reducing the number of lighting passes required (doing something of
a hybrid between deferred and forward+), lowering the precision of some
render targets on low end hardware, better sorting of render passes and
combining of some passes to reduce overdraw, etc.
I assure you the commitment to low end hardware remains, and the move to
deprecate the forward renderer is predecated on achieving acceptable
performance. Note that the shaders are #if 0'd out, not removed. I'm
absolutely leaving the door open to keeping forward rendering if the
experiment doesn't pan out because I'm absolutely aware that I might be
wrong here.
I've been working on the Second Life viewer since 2005 and took a some
years away to work on Sansar and uplift (and frankly, I wish I hadn't,
although I learned a lot in that time). I completely understand how
important low end hardware support is, and when I say "low end" I don't
mean integrated GPUs like Iris and Intel UHD (what I'd call mainstream). I
mean Intel HD. The phrase I flippantly throw around to describe a typical
PC running SL is "your grandma's crappy laptop." I've been making sure
Iris Xe can run reflection probes at 20fps+ and there's already a fallback
for PBR rendering that works on OpenGL 3.2 and drops reflection probes and
anti-aliasing for performance and backwards compatibility reasons (OpenGL
3.2 doesn't support cubemap arrays, for example, which the reflection probe
shaders need). More optimization passes are coming.
Henri Beauchamp
I am glad you are open to discussion. :-)
I do not use Discord but if you wish, you can send me your email
address (mine is sldev at free dot fr) so that we do not “pollute” this
git.
Please, be sure that I am in no way trying to “hold back progress”: I’m an eager early adopter
(like demonstrates the speed at which I integrate new features in my
viewer: it took me under a week to integrate puppetry, for example, even
though I won’t use it myself), but I am also extremely vigilant and totally unforgiving towards regressions,
especially when they could deter new SLers to join (by pushing too high
the required entry level) or make old SLers give up (by “deprecating” a
hardware they cannot afford to renew and yet was working just fine for
SL so far).
There are some other arguments (power consumption, heating,
throttling, anti-aliasing quality, memory consumption, porting on arm64
(OpenGL 2.1: eeek !) platforms, you know, for mobile devices…) in favour
of keeping a forward rendering mode as it exists today (and not a
degraded deferred rendering which would regress the 3D rendering quality
when compared with what we have, like you suggest above), and not only
linked to pure performances, but also to specific SL usages which could
become impaired or degraded without it.
I would encourage you to hear these arguments of mine (via email) and
take them into consideration, for I would hate to (yet again) tell you
in the future “Hey LL, I told you so…” after the damages are done.
Thank you !
Dave Parksauthor
I'd prefer the discussion to remain public so others might read it, and
commit comments are as good a place as any to publish the discussion. The
main advantage of the SL Discord server is it enables live discussion among
a diverse group -- not just developers, but content creators as well.
I'll start off by saying that the endeavor to remove forward rendering is
first and foremost an experiment, and I'm not prescient enough to know how
an experiment will turn out ahead of time. The hypothesis is that the
forward renderer as it stands today creates more problems than it solves,
and that the problems it solves would be better solved by a fast deferred
renderer.
As for the problems the forward renderer creates, there are three:
1. It consumes a disproportionate amount of developer resources to maintain
(meaning it takes developer resources from the majority of users and spends
those resources on a minority of users)
2. It increases the cost of making content for SL. Content creators often
find themselves having to make "ALM" and "Non-ALM" versions of their
content, and they have to educate their customers on which mode to use.
3. It breaks the shared experience. People with ALM on see a very
different Second Life than people with ALM off.
As for problems it solves:
1. It generally performs better
2. It provides support for some hardware that is incapable of deferred
rendering (OpenGL 2.1, as you say).
As for hardware support. It would be disingenuous to claim that everyone
in SL today either has or could acquire hardware that is capable of
deferred rendering. That will simply never be true as long as we support
OpenGL 2.1. However, from the evidence in front of me in our hardware
reports, the OpenGL 2.1 hardware doesn't actually exist in any quantity in
our existing user base sufficient to justify the expense of maintaining
OpenGL 2.1. Looking at the numbers, the prudent thing for our
existing residents seems to be to draw the compatibility line at OpenGL 3.0.
For future work, the solution for ARM and others is not GLES support, it's
Vulkan. Part of the transition to Vulkan involves reducing our render code
footprint, of which the forward renderer makes up a significant portion.
So I get dragged to the conclusion that hardware support is not a
sufficient enough cause to maintain forward rendering. This truly sucks
for the few people who are actually stuck on hardware that can't do
deferred rendering, but there just aren't enough of them to warrant taking
resources away from the 99.8% of our residents that can do deferred
rendering. While this might be a bad example (I personally think the
removal of Linux support was a bad move), when you compare Linux support to
the numbers we're talking about here in terms of developer hours to
really maintain the forward renderer (which hasn't truly been maintained
since the release of the performance viewer) vs the residents retained by
those efforts, the numbers are an order of magnitude tilted in favor of
dropping forward rendering.
As for performance, when doing traces on the render pipe it is apparent
that the deferred renderer is much slower than it needs to be. The thing
that kills low end hardware is memory bandwidth (aka fill rate), of which
overdraw is the main waste generator. It should be noted that the
performance viewer improvements did nothing to address fill rate related
render performance. The deferred render pipe is throwing cycles away like
candy just by not having various render targets and render passes arranged
optimally (the forward renderer at least attempts to do this, but fails in
several cases). For example, the sky is drawn over the entire screen, not
just where the sky is visible -- TWICE. At least once per frame, the
entire contents of the screen are copied from one buffer to another instead
of just modifying render targets to point at different buffers. Much of
the post effect chain (which deferred rendering relies on for gamma
correction and anti-aliasing and the forward renderer relies on for glow)
could be collapsed into a single shader that touches every pixel on the
screen once instead of three times. There are also ways to do deferred
lighting that involve touching every pixel once instead of the way we do
now, which is to touch every pixel N times depending on how many lights
affect that pixel. Indeed, the only reason the forward renderer holds a
candle to the deferred renderer in terms of performance is because it drops
all but a handful of lights in the scene.
I could go on, but this post is already pretty long. At least for right
now, I'd rather focus on making the deferred renderer as fast and robust as
it can be, and if that turns out to just not be good enough as a
replacement for the forward renderer, then reevaluate.
Nicky Dasmijn
I'd prefer the discussion to remain public so others might read it, and commit comments are as good a place as any to publish the discussion. The main advantage of the SL Discord server is it enables live discussion among a diverse group -- not just developers, but content creators as well.
Take it to the mail list maybe? The audience there is at least a little bigger than here.
Henri Beauchamp
I agree, because commits may vanish with their comments (when
reverted, for example), and this git is not exactly “public” while the
mailing list got a wider audience and a truly public archive…
Should I reply there (quoting the whole conversation, for the context and archives) ? Dave, any objection ?
Henri Beauchamp
Sorry for the late reply, but the week has been very busy for me in RL...
I was also awaiting (good excuse(TM)) for Dave's approval about a
re-posting in the SL devel mailing list, but since it never came, I will
reply here.
I'll start off by saying that the endeavor to remove forward rendering is first
and foremost an experiment .../...
Great !... I'm all for experiments, optimizations and progress: it's been 15 years I have been doing this with my viewer.
But then why *removing* stuff (like in this very commit, where it is
not just about #if-0-ing out some code) that would allow the forward
renderer to run "as is" (i.e. without any reflection probe, PBR and
stuff you add to ALM) ?... :-(
What I did for my dual-renderer viewer (Cool VL Viewer v1.28.2) with
the EE renderer (*) has been to preserve the code and shaders for WL, by
using a separate set of shaders (but here, you could simply change the
names of the shaders for use with PBR/ALM and keep the old ones for the
forward renderer, which would be even easier) and branching, where
needed, in the render pipeline with a global gUseNewShaders boolean
(true for EE, false for WL); this part of the dual-rendering code was
*very easy* to “implement” (actually almost no code to write); the
difficult part was getting both EE and WL environment settings to get
translated flawlessly between each other in real time: something you do
not have to bother with here.
The advantage of keeping a functional forward renderer in your
experimental viewer is also that you can then compare, by just
(un)checking a box in the graphics prefs, the hardware compatibility,
performances and rendering quality of both modes (no need to fire a
different viewer to compare).
(*) EE got released way too soon and had a lot of problems such as a
totally busted culling, etc, that made it *much* slower than the WL
renderer, up until the performance viewer fixed it all. I do really hope
the same mistake will not be done with the future PBR viewer, i.e. do
take your time !!!
As for the problems the forward renderer creates, there are three:
1. It consumes a disproportionate amount of developer resources to maintain
Thing is, all you have to do is to "freeze" it as it is !... No
maintenance needed at all, but just some branching in the code via a new
gUsePbr boolean, where needed, to preserve the old code path when
incompatible with the new one.
It increases the cost of making content for SL. Content creators often find
themselves having to make "ALM" and "Non-ALM" versions of their content, and
they have to educate their customers on which mode to use.
A totally moot point: there's plenty of contents in SL already
(especially for avatar apparel) that renders differently with ALM on and
off. So what ?... It's the *user choice* to buy such stuff or not
and/or to run with ALM on or off, and not only depending on their
hardware, but also on what *usage* they are *currently* making of SL (if
clubbing, you want ALM on and adjust your draw distance for acceptable
frame rates, if flying over main land, you want it off and have high
frame rates with large draw distances and lower memory usage since less
textures loaded).
The problem is that all this PBR stuff got almost exclusively debated
in contents-creator groups/meetings, meaning that the *actual users*
have been largely excluded from these debates and will be faced with
dire consequences as a result ! This is BAAAAAD !
It breaks the shared experience. People with ALM on see a very different
Second Life than people with ALM off.
A “classical” misconception/misinterpretation of rule 2.k in the SL
TPV Policy, and a typical *abusive and fallacious interpretation* by
contents creators (and I'm not saying contents creators are bad people:
I'm a contents creator too in SL !).
The "shared experience" rule has been put in place because LL got
faced with TPVs (well, one TPV in particular) which implemented a
feature (double attachment point on joints) that was not supported by
other viewers (and not supported by LL's viewer, of course) at that
time: this caused a total havoc for people looking (with LL's viewer and
most TPVs) at an avatar pertaining to a user running that specific TPV,
and literally broke the "shared experience" (you could see that
avatar's attachments floating around).
Now, LL *never* forbid to change any aspect of how you render a scene
in *your own* viewer. E.g. via custom environments, Restrained Love
restrictions (blinding, blurring, limiting view distance, etc). Plus
such differences are anyway unavoidable, due to hardware limitations of
the various usable PCs running SL.
ALM on/off is such a choice left to the *user* (and that should be left to them).
This #3 argument is therefore totally *invalid*, *specious*, and
right out *dangerous* (it would mean banning Restrained Love from SL,
for example and I know a sh*tload of contents creators who would be
pissed off by such a ban, counting myself among them) !
However, from the evidence in front of me in our hardware reports, the OpenGL 2.1
hardware doesn't actually exist in any quantity in our existing user base sufficient
to justify the expense of maintaining OpenGL 2.1.
BIG problem: your stats only account for the *current user base*
of SLers. It does not account for putative SLers (or one-day-SLers)
that could not join or immediately gave up on SL because their hardware
could not run it decently enough... The stats do not account either for
"old" (or not so old) time SLers who *gave up* because their PC hardware
became "deprecated" and they cannot afford updating it (thankfully for
LL, there is the Cool VL Viewer that is often known by old time SLers;
quite a few of them end up using it just to keep SLing at decent frame
rates).
With the hardware prices skyrocketing (and especially for GPUs), be
it because of the COVID consequences, the crypto-currencies mining, or
the inflation, a move away from "old" hardware would be on an especially
BAD timing !
As for performance, when doing traces on the render pipe it is apparent that the deferred
renderer is much slower than it needs to be. The thing that kills low end hardware is
memory bandwidth (aka fill rate) ..../...
I am glad you see that many opportunities to improve the deferred
renderer performances, and I am *eager* to see them at play. However, I
am still highly skeptical about the only "fill rate" bottleneck: iGPUs
(and "old" GPUs) are also limited by their low number of computing
units: they just cannot run many complex shaders, and the forward
rendering mode spares them this burden !
But I would *totally love* to be proven wrong about the performances issue for this class of "weak" GPUs !
Yet there are other issues with deferred rendering than just
performances on lower end hardware, which brings me to speak a bit about
my own SL usage: I have a very capable PC (i7-9700K OCEd @ 5.0GHz on
all cores, OCed GTX 1070Ti, 64GB RAM, and of course running under Linux)
able to run the current deferred rendering as fast or sometimes even
faster than forward rendering, and yet, I still use forward rendering at
least 80% of the time in SL (and more likely 90%). Why the Hell is that
?
Well, forward rendering provides me with a *less noisy* system (GPU fans
spin really fast when I enable ALM, even with frame limiting on and set
anywhere between 60 and 120 fps), it provides me with a *much sharper*
image (especially in the distance, meaning all outdoor scenes look
waaaay better), and this despite the fact that my viewer got Alchemy's
SMAA and CAS shaders backported (the first avoiding the blur seen in
detailed textures with LL's viewer FXAA shader and the second sharpening
(artificially) the image even beside what 4xSSAA gives in forward
rendering), it provides more saturated colors (with much less "gloomy"
interior scenes than with ALM on), it eats up less memory (especially
when I push the draw distance to 512m while sailing near coasts, or
flying over main land, for example)...
So I would *HATE* being left with just deferred rendering and any of the
following consequences: a noisy system, blurry scenes, desaturated
colors, too much memory used for large draw distances.
I could go on,
Me too ! :-D
But, if you did read this all, and take it all into account, I will already feel much warmer inside... ;-)
Dave Parksauthor
Wanted to drop a quick note that I did read your comments and you make some good points.
I think one of the things you’re missing WRT the cost of maintaining
both is the bug whack-a-mole we engage in fielding bug reports from
content creators, and the struggle for content creators when deciding
whether their content should require normal and specular maps and
dynamic lighting, or should use baked lighting. It doubles the number
of edge cases and creates a lot of load for QA, content creators, and
developers. Yes, it’s relatively easy to freeze the shaders as they are
and preserve a legacy fork, but that’s a much lower bar than what I’m
setting. The bar I’m setting is to take low end hardware seriously
enough to actually backport support for the entire material system and
lighting model to that hardware. This is what content creators want. I
think a high performance deferred render pipe is the answer. There will
be trade offs, of course. Water reflection will change, anti-aliasing
won’t be as nice, etc. If we see the end results of a high performance
deferred renderer and people aren’t satisfied, the remedy will likely be
the creation of a new forward renderer that uses the same shader code
for lighting and materials as the deferred renderer, but limits the
number of lights, but before resorting to that there are still many
avenues to pursue.
And yes, I misused “shared experience.” If a TPV wants to provide
forward rendering, use a “better” lighting model, etc, that’s fine, so
long as it doesn’t turn into a situation where certain content only
works on that TPV. What I was referring to is the lack of materials
support in the forward renderer causing objects to look very different
depending on your graphics settings. This is untenable from the point
of view of content creators.
As for hardware stats – the 500k samples are every unique login in a
given time period. I’m not much a fan of valuing imaginary potential
residents over existing real residents, but if I find time I’ll try to
get a sample that only includes new logins.
Re: the email list – my attention budget can’t afford yet another
comms channel, so Discord, user groups, commit comments, and Jira are
going to be it for me.
Henri Beauchamp
First and foremost, thank you for your reply !
The bar I’m setting is to take low end hardware seriously enough to
actually backport support for the entire material system and lighting
model to that hardware.
Good. One other point I did not make in my former post (well, I made
it indirectly, when mentioning GPU fans): beware of power consumption
and heating for notebook computers (even “gaming” ones): too many
shaders also lead to a lowered autonomy and sometimes overheating
causing throttling (currently seen with M1/M2 Macs, for example). In
such cases, the current forward renderer is a life savior…
I’m not much a fan of valuing imaginary potential
I’m afraid they are in no way imaginary, but simply not part of your stats. That’s the problem of such stats…
For “giving up old timers”, for example, you would have to use per
avatar stats, extending over months or years, and see whether an avatar
suddenly stopped using SL and why (did they stop after a hardware
deprecation move in SL viewer, did they migrate to another,
lighter/compatible viewer such as mine, etc). This would be an extremely
tedious task to properly analyze such stats and would require a
professional statistician to draw proper conclusions.
All I want to point out is that what you believe the stats are
showing by just looking at crude numbers is likely entirely misleading…
What I am worried about with the PBR renderer would be the recurrence
of the same scheme as what occurred with Windlight, which “killed” a
lot of (not so) old hardware when it got implemented, and lead me to
fork the v1.19.0.5 viewer to make it the Cool VL Viewer v1.19.2,
maintaining a non-Windlight viewer branch for 88 releases (with a lot of thanks received from people with “old” hardware) !
We need old timers to stay in SL and newcomers to find a
renderer mode in the viewer that works with their computer, even if it
is a little too weak for “comfortable SLing” !!!
There will be trade offs, of course. Water reflection will change, anti-aliasing won’t be as nice
Water reflections are currently a catastrophe (way too expensive and
thus most often turned off or to just “terrain and trees”), so you
certainly won’t see me advocate for keeping them “as is” ! :-P
However, anti-aliasing is a big problem with LL’s current deferred renderer, and, to me, right out ugly and intolerable (1).
May I suggest that you, at the very least, adopt the
SMAA shader to replace (even if only as an option in the graphics
prefs) the sh*tty FXAA shader ?… The CAS shader might also be of some
use (even if it may sometimes cause artifacts by “inventing” texture
details).
Yet, even with these shaders, things such as grillage meshes,
windows, brick walls, rails, etc still look especially bad in the
distance (and even worst when moving around too) when compared with the
native GPU 4x SSAA (8x does not bring anything, and 16x is not true
SSAA) of the forward renderer…
I would encourage you to give a try to the Cool VL Viewer or Alchemy
(my viewer reuses Alchemy’s implementation but got an easier way to
switch between AA shaders, and got less aggressive default CAS
settings), and see how AA is so much different in the various modes (try
both indoor and outdoor scenes, with detailed textures for the former
and large draw distances for the latter)… Oh, of course, do try with a
non-HDPI monitor (else, the blur gets partly “hidden” from your eyes by
the excessively high DPI figure): a 24” full HD monitor (92 DPI) is
ideal for such tests.
(1) This is to a point that, before I found out about Rye Mutt’s
excellent work in Alchemy with SMAA and backported the said work to my
viewer, I was pretty much never using (but for short test sessions) the
ALM mode in the viewer !
Dave Parksauthor
There are two arguments for keeping forward rendering – one is GL
2 support, and one is quality/performance. Hopefully this puts the GL 2
argument to bed.
I checked your hypothesis about the historical trend and that
grouping by unique account id would yield different results (good
hypothesis, definitely worth checking). I can’t share raw numbers (not
my story to tell), but I can show you an image that will show you the
trend lines, and the conclusion is the same (although GL 2
representation was slightly higher with that grouping). There’s no
cliff that would correspond with a release that caused people to leave
in large numbers, and overall trend lines on agent counts match other
plots which account for TPVs. Bottom line is accounts that depend on GL
2 are vanishingly small in number to the point where it is
irresponsible to continue investing in supporting them at the expense of
everyone else. The numbers in 2018 tell a compelling (but debatable)
story for continuing to invest in GL 2, but the numbers in 2022 compel
me to abandon GL 2, maintain compatibility with GL 3, and that it’s ok
to require GL 4 where necessary for render quality features like
reflection probes, SSR, and SMAA. Keep in mind this is Windows only,
since the Mac numbers are artificially skewed toward GL 2 because of the
way we used to handle OpenGL on Mac, but Linux numbers show a similar
trend and ratio.
The Y-axis here is unique account id, the x-axis is GL version by
month, going back to January 2018. So the first cluster of 4 bars shows
the counts of GL major verison 1-4 for as they were in January 2018,
last 4 bars are September 2022. Note that this image is cropped to not
show the trend in overall agent numbers (there’s a place to talk about
that trend, and this isn’t it)… the ‘4' count is actually higher than
shown in the image. I understand that there are still hundreds of
people on GL 2, but there are hundreds of thousands on GL 4, and I just
can’t justify taking resources away from the latter group to serve the
former.
There might still be a good argument for maintaining a forward
renderer if performance and quality aren’t good enough on low powered
hardware under an optimized deferred renderer, but I see no evidence
that we should continue investing in GL 2 support. The current plan is
to get the PBR viewer into a public alpha and gather more feedback, and
spend the next six weeks optimizing, fixing regressions, and responding
to feedback.
Henri Beauchamp
Hopefully this puts the GL 2 argument to bed
It would be the case, if there was not the ARM future support to
consider (and I am seriously considering it for my viewer)… The Cool VL
Viewer can already be compiled for ARM64 platforms
(e.g. RockPro64, PI4B, etc), sadly these only support OpenGL (ES) v2.1…
Other potential targets for such ports could be Chrome-books, for
example…
Not saying OpenGL v2.1 is a must have for PCs or Macs, however… But
it would really be “nice too keep”, even if only in a simplified forward
renderer.
LL is crazy to do this: all but powerful PCs will be bared from using Second Life…
This is true with the things we tend to enable implicitly with the ALM checkbox like shadows, AO, and an infinite number of lights, but it’s not true generally. We’ll be much better off if we divert resources that would have gone to maintaining the forward render shaders to making sure the deferred rendering shaders perform well on low end hardware.
Edit to note: This move has been discussed extensively with third party viewer developers and content creators and is generally seen as a good thing, particularly among content creators. Having to support both render modes is a significant burden both on developers and content creators, and the only argument for keeping forward rendering is based on a fallacy. Deferred rendering is not inherently slower than forward rendering, we just treat “deferred” as synonmous with “high detail” in the viewer.
Sadly in Voice and thus excluding me, a TPV developers who has been dealing with his own viewer for the past 15 years, who has seen the move to Windlinght, then the one to Extended Environment (and who produced and maintained a dual renderer viewer (WL+EE) until, at last the EE shaders and render pipeline got fixed and matched WL speed, with the performances viewer changes).
The fallacy is to deny FACTS: forward rendering is up to twice as fast as deferred rendering with iGPUs and APUs, for example. Denying it is lying to people !
I am right out FURIOUS to see that much BAD FAITH and, sorry, but that much stupid of a stance too (you are going to make any non-gamer PC owners experience miserable in SL) !
I'm happy to talk to you via text (Discord is my preferred chat client, and that's where most of the discussions are taking place outside the user groups). You may be correct, and I value the feedback and appreciate your perspective and contributions, but I think there's some information you're missing. It is true that forward rendering can produce better performance than deferred, and historically the forward renderer in SL has performed as well or better than the deferred renderer, but with a little work on optimizing the deferred renderer, I believe it can be made to perform better than the forward renderer does today, and those efforts will benefit all users, not just those on low end hardware, and will give everyone the best experience possible. Examples of optimizations here involve reducing the number of lighting passes required (doing something of a hybrid between deferred and forward+), lowering the precision of some render targets on low end hardware, better sorting of render passes and combining of some passes to reduce overdraw, etc.
I assure you the commitment to low end hardware remains, and the move to deprecate the forward renderer is predecated on achieving acceptable performance. Note that the shaders are #if 0'd out, not removed. I'm absolutely leaving the door open to keeping forward rendering if the experiment doesn't pan out because I'm absolutely aware that I might be wrong here.
I've been working on the Second Life viewer since 2005 and took a some years away to work on Sansar and uplift (and frankly, I wish I hadn't, although I learned a lot in that time). I completely understand how important low end hardware support is, and when I say "low end" I don't mean integrated GPUs like Iris and Intel UHD (what I'd call mainstream). I mean Intel HD. The phrase I flippantly throw around to describe a typical PC running SL is "your grandma's crappy laptop." I've been making sure Iris Xe can run reflection probes at 20fps+ and there's already a fallback for PBR rendering that works on OpenGL 3.2 and drops reflection probes and anti-aliasing for performance and backwards compatibility reasons (OpenGL 3.2 doesn't support cubemap arrays, for example, which the reflection probe shaders need). More optimization passes are coming.
I am glad you are open to discussion. :-)
I do not use Discord but if you wish, you can send me your email address (mine is sldev at free dot fr) so that we do not “pollute” this git.
Please, be sure that I am in no way trying to “hold back progress”: I’m an eager early adopter (like demonstrates the speed at which I integrate new features in my viewer: it took me under a week to integrate puppetry, for example, even though I won’t use it myself), but I am also extremely vigilant and totally unforgiving towards regressions, especially when they could deter new SLers to join (by pushing too high the required entry level) or make old SLers give up (by “deprecating” a hardware they cannot afford to renew and yet was working just fine for SL so far).
There are some other arguments (power consumption, heating, throttling, anti-aliasing quality, memory consumption, porting on arm64 (OpenGL 2.1: eeek !) platforms, you know, for mobile devices…) in favour of keeping a forward rendering mode as it exists today (and not a degraded deferred rendering which would regress the 3D rendering quality when compared with what we have, like you suggest above), and not only linked to pure performances, but also to specific SL usages which could become impaired or degraded without it.
I would encourage you to hear these arguments of mine (via email) and take them into consideration, for I would hate to (yet again) tell you in the future “Hey LL, I told you so…” after the damages are done.
Thank you !
I'd prefer the discussion to remain public so others might read it, and commit comments are as good a place as any to publish the discussion. The main advantage of the SL Discord server is it enables live discussion among a diverse group -- not just developers, but content creators as well.
I'll start off by saying that the endeavor to remove forward rendering is first and foremost an experiment, and I'm not prescient enough to know how an experiment will turn out ahead of time. The hypothesis is that the forward renderer as it stands today creates more problems than it solves, and that the problems it solves would be better solved by a fast deferred renderer.
As for the problems the forward renderer creates, there are three: 1. It consumes a disproportionate amount of developer resources to maintain (meaning it takes developer resources from the majority of users and spends those resources on a minority of users) 2. It increases the cost of making content for SL. Content creators often find themselves having to make "ALM" and "Non-ALM" versions of their content, and they have to educate their customers on which mode to use. 3. It breaks the shared experience. People with ALM on see a very different Second Life than people with ALM off.
As for problems it solves: 1. It generally performs better 2. It provides support for some hardware that is incapable of deferred rendering (OpenGL 2.1, as you say).
As for hardware support. It would be disingenuous to claim that everyone in SL today either has or could acquire hardware that is capable of deferred rendering. That will simply never be true as long as we support OpenGL 2.1. However, from the evidence in front of me in our hardware reports, the OpenGL 2.1 hardware doesn't actually exist in any quantity in our existing user base sufficient to justify the expense of maintaining OpenGL 2.1. Looking at the numbers, the prudent thing for our existing residents seems to be to draw the compatibility line at OpenGL 3.0.
For future work, the solution for ARM and others is not GLES support, it's Vulkan. Part of the transition to Vulkan involves reducing our render code footprint, of which the forward renderer makes up a significant portion.
So I get dragged to the conclusion that hardware support is not a sufficient enough cause to maintain forward rendering. This truly sucks for the few people who are actually stuck on hardware that can't do deferred rendering, but there just aren't enough of them to warrant taking resources away from the 99.8% of our residents that can do deferred rendering. While this might be a bad example (I personally think the removal of Linux support was a bad move), when you compare Linux support to the numbers we're talking about here in terms of developer hours to really maintain the forward renderer (which hasn't truly been maintained since the release of the performance viewer) vs the residents retained by those efforts, the numbers are an order of magnitude tilted in favor of dropping forward rendering.
As for performance, when doing traces on the render pipe it is apparent that the deferred renderer is much slower than it needs to be. The thing that kills low end hardware is memory bandwidth (aka fill rate), of which overdraw is the main waste generator. It should be noted that the performance viewer improvements did nothing to address fill rate related render performance. The deferred render pipe is throwing cycles away like candy just by not having various render targets and render passes arranged optimally (the forward renderer at least attempts to do this, but fails in several cases). For example, the sky is drawn over the entire screen, not just where the sky is visible -- TWICE. At least once per frame, the entire contents of the screen are copied from one buffer to another instead of just modifying render targets to point at different buffers. Much of the post effect chain (which deferred rendering relies on for gamma correction and anti-aliasing and the forward renderer relies on for glow) could be collapsed into a single shader that touches every pixel on the screen once instead of three times. There are also ways to do deferred lighting that involve touching every pixel once instead of the way we do now, which is to touch every pixel N times depending on how many lights affect that pixel. Indeed, the only reason the forward renderer holds a candle to the deferred renderer in terms of performance is because it drops all but a handful of lights in the scene.
I could go on, but this post is already pretty long. At least for right now, I'd rather focus on making the deferred renderer as fast and robust as it can be, and if that turns out to just not be good enough as a replacement for the forward renderer, then reevaluate.
I'd prefer the discussion to remain public so others might read it, and commit comments are as good a place as any to publish the discussion. The main advantage of the SL Discord server is it enables live discussion among a diverse group -- not just developers, but content creators as well.Take it to the mail list maybe? The audience there is at least a little bigger than here.
I agree, because commits may vanish with their comments (when reverted, for example), and this git is not exactly “public” while the mailing list got a wider audience and a truly public archive…
Should I reply there (quoting the whole conversation, for the context and archives) ? Dave, any objection ?
Sorry for the late reply, but the week has been very busy for me in RL...
I was also awaiting (good excuse(TM)) for Dave's approval about a re-posting in the SL devel mailing list, but since it never came, I will reply here.
Great !... I'm all for experiments, optimizations and progress: it's been 15 years I have been doing this with my viewer.
But then why *removing* stuff (like in this very commit, where it is not just about #if-0-ing out some code) that would allow the forward renderer to run "as is" (i.e. without any reflection probe, PBR and stuff you add to ALM) ?... :-(
What I did for my dual-renderer viewer (Cool VL Viewer v1.28.2) with the EE renderer (*) has been to preserve the code and shaders for WL, by using a separate set of shaders (but here, you could simply change the names of the shaders for use with PBR/ALM and keep the old ones for the forward renderer, which would be even easier) and branching, where needed, in the render pipeline with a global gUseNewShaders boolean (true for EE, false for WL); this part of the dual-rendering code was *very easy* to “implement” (actually almost no code to write); the difficult part was getting both EE and WL environment settings to get translated flawlessly between each other in real time: something you do not have to bother with here.
The advantage of keeping a functional forward renderer in your experimental viewer is also that you can then compare, by just (un)checking a box in the graphics prefs, the hardware compatibility, performances and rendering quality of both modes (no need to fire a different viewer to compare).
(*) EE got released way too soon and had a lot of problems such as a totally busted culling, etc, that made it *much* slower than the WL renderer, up until the performance viewer fixed it all. I do really hope the same mistake will not be done with the future PBR viewer, i.e. do take your time !!!
Thing is, all you have to do is to "freeze" it as it is !... No maintenance needed at all, but just some branching in the code via a new gUsePbr boolean, where needed, to preserve the old code path when incompatible with the new one.
A totally moot point: there's plenty of contents in SL already (especially for avatar apparel) that renders differently with ALM on and off. So what ?... It's the *user choice* to buy such stuff or not and/or to run with ALM on or off, and not only depending on their hardware, but also on what *usage* they are *currently* making of SL (if clubbing, you want ALM on and adjust your draw distance for acceptable frame rates, if flying over main land, you want it off and have high frame rates with large draw distances and lower memory usage since less textures loaded).
The problem is that all this PBR stuff got almost exclusively debated in contents-creator groups/meetings, meaning that the *actual users* have been largely excluded from these debates and will be faced with dire consequences as a result ! This is BAAAAAD !
A “classical” misconception/misinterpretation of rule 2.k in the SL TPV Policy, and a typical *abusive and fallacious interpretation* by contents creators (and I'm not saying contents creators are bad people: I'm a contents creator too in SL !).
The "shared experience" rule has been put in place because LL got faced with TPVs (well, one TPV in particular) which implemented a feature (double attachment point on joints) that was not supported by other viewers (and not supported by LL's viewer, of course) at that time: this caused a total havoc for people looking (with LL's viewer and most TPVs) at an avatar pertaining to a user running that specific TPV, and literally broke the "shared experience" (you could see that avatar's attachments floating around).
Now, LL *never* forbid to change any aspect of how you render a scene in *your own* viewer. E.g. via custom environments, Restrained Love restrictions (blinding, blurring, limiting view distance, etc). Plus such differences are anyway unavoidable, due to hardware limitations of the various usable PCs running SL.
ALM on/off is such a choice left to the *user* (and that should be left to them).
This #3 argument is therefore totally *invalid*, *specious*, and right out *dangerous* (it would mean banning Restrained Love from SL, for example and I know a sh*tload of contents creators who would be pissed off by such a ban, counting myself among them) !
BIG problem: your stats only account for the *current user base* of SLers. It does not account for putative SLers (or one-day-SLers) that could not join or immediately gave up on SL because their hardware could not run it decently enough... The stats do not account either for "old" (or not so old) time SLers who *gave up* because their PC hardware became "deprecated" and they cannot afford updating it (thankfully for LL, there is the Cool VL Viewer that is often known by old time SLers; quite a few of them end up using it just to keep SLing at decent frame rates).
With the hardware prices skyrocketing (and especially for GPUs), be it because of the COVID consequences, the crypto-currencies mining, or the inflation, a move away from "old" hardware would be on an especially BAD timing !
I am glad you see that many opportunities to improve the deferred renderer performances, and I am *eager* to see them at play. However, I am still highly skeptical about the only "fill rate" bottleneck: iGPUs (and "old" GPUs) are also limited by their low number of computing units: they just cannot run many complex shaders, and the forward rendering mode spares them this burden !
But I would *totally love* to be proven wrong about the performances issue for this class of "weak" GPUs !
Yet there are other issues with deferred rendering than just performances on lower end hardware, which brings me to speak a bit about my own SL usage: I have a very capable PC (i7-9700K OCEd @ 5.0GHz on all cores, OCed GTX 1070Ti, 64GB RAM, and of course running under Linux) able to run the current deferred rendering as fast or sometimes even faster than forward rendering, and yet, I still use forward rendering at least 80% of the time in SL (and more likely 90%). Why the Hell is that ?
Well, forward rendering provides me with a *less noisy* system (GPU fans spin really fast when I enable ALM, even with frame limiting on and set anywhere between 60 and 120 fps), it provides me with a *much sharper* image (especially in the distance, meaning all outdoor scenes look waaaay better), and this despite the fact that my viewer got Alchemy's SMAA and CAS shaders backported (the first avoiding the blur seen in detailed textures with LL's viewer FXAA shader and the second sharpening (artificially) the image even beside what 4xSSAA gives in forward rendering), it provides more saturated colors (with much less "gloomy" interior scenes than with ALM on), it eats up less memory (especially when I push the draw distance to 512m while sailing near coasts, or flying over main land, for example)...
So I would *HATE* being left with just deferred rendering and any of the following consequences: a noisy system, blurry scenes, desaturated colors, too much memory used for large draw distances.
Me too ! :-D
But, if you did read this all, and take it all into account, I will already feel much warmer inside... ;-)
Wanted to drop a quick note that I did read your comments and you make some good points.
I think one of the things you’re missing WRT the cost of maintaining both is the bug whack-a-mole we engage in fielding bug reports from content creators, and the struggle for content creators when deciding whether their content should require normal and specular maps and dynamic lighting, or should use baked lighting. It doubles the number of edge cases and creates a lot of load for QA, content creators, and developers. Yes, it’s relatively easy to freeze the shaders as they are and preserve a legacy fork, but that’s a much lower bar than what I’m setting. The bar I’m setting is to take low end hardware seriously enough to actually backport support for the entire material system and lighting model to that hardware. This is what content creators want. I think a high performance deferred render pipe is the answer. There will be trade offs, of course. Water reflection will change, anti-aliasing won’t be as nice, etc. If we see the end results of a high performance deferred renderer and people aren’t satisfied, the remedy will likely be the creation of a new forward renderer that uses the same shader code for lighting and materials as the deferred renderer, but limits the number of lights, but before resorting to that there are still many avenues to pursue.
And yes, I misused “shared experience.” If a TPV wants to provide forward rendering, use a “better” lighting model, etc, that’s fine, so long as it doesn’t turn into a situation where certain content only works on that TPV. What I was referring to is the lack of materials support in the forward renderer causing objects to look very different depending on your graphics settings. This is untenable from the point of view of content creators.
As for hardware stats – the 500k samples are every unique login in a given time period. I’m not much a fan of valuing imaginary potential residents over existing real residents, but if I find time I’ll try to get a sample that only includes new logins.
Re: the email list – my attention budget can’t afford yet another comms channel, so Discord, user groups, commit comments, and Jira are going to be it for me.
First and foremost, thank you for your reply !
Good. One other point I did not make in my former post (well, I made it indirectly, when mentioning GPU fans): beware of power consumption and heating for notebook computers (even “gaming” ones): too many shaders also lead to a lowered autonomy and sometimes overheating causing throttling (currently seen with M1/M2 Macs, for example). In such cases, the current forward renderer is a life savior…
I’m afraid they are in no way imaginary, but simply not part of your stats. That’s the problem of such stats…
For “giving up old timers”, for example, you would have to use per avatar stats, extending over months or years, and see whether an avatar suddenly stopped using SL and why (did they stop after a hardware deprecation move in SL viewer, did they migrate to another, lighter/compatible viewer such as mine, etc). This would be an extremely tedious task to properly analyze such stats and would require a professional statistician to draw proper conclusions.
All I want to point out is that what you believe the stats are showing by just looking at crude numbers is likely entirely misleading…
What I am worried about with the PBR renderer would be the recurrence of the same scheme as what occurred with Windlight, which “killed” a lot of (not so) old hardware when it got implemented, and lead me to fork the v1.19.0.5 viewer to make it the Cool VL Viewer v1.19.2, maintaining a non-Windlight viewer branch for 88 releases (with a lot of thanks received from people with “old” hardware) !
We need old timers to stay in SL and newcomers to find a renderer mode in the viewer that works with their computer, even if it is a little too weak for “comfortable SLing” !!!
Water reflections are currently a catastrophe (way too expensive and thus most often turned off or to just “terrain and trees”), so you certainly won’t see me advocate for keeping them “as is” ! :-P
However, anti-aliasing is a big problem with LL’s current deferred renderer, and, to me, right out ugly and intolerable (1).
May I suggest that you, at the very least, adopt the SMAA shader to replace (even if only as an option in the graphics prefs) the sh*tty FXAA shader ?… The CAS shader might also be of some use (even if it may sometimes cause artifacts by “inventing” texture details).
Yet, even with these shaders, things such as grillage meshes, windows, brick walls, rails, etc still look especially bad in the distance (and even worst when moving around too) when compared with the native GPU 4x SSAA (8x does not bring anything, and 16x is not true SSAA) of the forward renderer…
I would encourage you to give a try to the Cool VL Viewer or Alchemy (my viewer reuses Alchemy’s implementation but got an easier way to switch between AA shaders, and got less aggressive default CAS settings), and see how AA is so much different in the various modes (try both indoor and outdoor scenes, with detailed textures for the former and large draw distances for the latter)… Oh, of course, do try with a non-HDPI monitor (else, the blur gets partly “hidden” from your eyes by the excessively high DPI figure): a 24” full HD monitor (92 DPI) is ideal for such tests.
(1) This is to a point that, before I found out about Rye Mutt’s excellent work in Alchemy with SMAA and backported the said work to my viewer, I was pretty much never using (but for short test sessions) the ALM mode in the viewer !
There are two arguments for keeping forward rendering – one is GL 2 support, and one is quality/performance. Hopefully this puts the GL 2 argument to bed.
I checked your hypothesis about the historical trend and that grouping by unique account id would yield different results (good hypothesis, definitely worth checking). I can’t share raw numbers (not my story to tell), but I can show you an image that will show you the trend lines, and the conclusion is the same (although GL 2 representation was slightly higher with that grouping). There’s no cliff that would correspond with a release that caused people to leave in large numbers, and overall trend lines on agent counts match other plots which account for TPVs. Bottom line is accounts that depend on GL 2 are vanishingly small in number to the point where it is irresponsible to continue investing in supporting them at the expense of everyone else. The numbers in 2018 tell a compelling (but debatable) story for continuing to invest in GL 2, but the numbers in 2022 compel me to abandon GL 2, maintain compatibility with GL 3, and that it’s ok to require GL 4 where necessary for render quality features like reflection probes, SSR, and SMAA. Keep in mind this is Windows only, since the Mac numbers are artificially skewed toward GL 2 because of the way we used to handle OpenGL on Mac, but Linux numbers show a similar trend and ratio.
The Y-axis here is unique account id, the x-axis is GL version by month, going back to January 2018. So the first cluster of 4 bars shows the counts of GL major verison 1-4 for as they were in January 2018, last 4 bars are September 2022. Note that this image is cropped to not show the trend in overall agent numbers (there’s a place to talk about that trend, and this isn’t it)… the ‘4' count is actually higher than shown in the image. I understand that there are still hundreds of people on GL 2, but there are hundreds of thousands on GL 4, and I just can’t justify taking resources away from the latter group to serve the former.
There might still be a good argument for maintaining a forward renderer if performance and quality aren’t good enough on low powered hardware under an optimized deferred renderer, but I see no evidence that we should continue investing in GL 2 support. The current plan is to get the PBR viewer into a public alpha and gather more feedback, and spend the next six weeks optimizing, fixing regressions, and responding to feedback.
It would be the case, if there was not the ARM future support to consider (and I am seriously considering it for my viewer)… The Cool VL Viewer can already be compiled for ARM64 platforms (e.g. RockPro64, PI4B, etc), sadly these only support OpenGL (ES) v2.1… Other potential targets for such ports could be Chrome-books, for example…
Not saying OpenGL v2.1 is a must have for PCs or Macs, however… But it would really be “nice too keep”, even if only in a simplified forward renderer.