Cool VL Viewer forum

View unanswered posts | View active topics It is currently 2024-03-29 14:31:37



Reply to topic  [ 14 posts ]  Go to page 1, 2  Next
Cool VL Viewer v1.26.12.2: Linux users, please read this ! 
Author Message

Joined: 2009-03-17 18:42:51
Posts: 5523
Reply with quote
In today's release of the Linux builds, I added a package (CoolVLViewer-1.26.12.2-no_jemalloc-Linux-x86-Setup) that installs a binary built without jemalloc (which is an advanced/optimized memory allocator), i.e. which uses the standard glibc memory allocator instead. Note that you can perfectly install both builds (with and without jemalloc) together on your system: simply install them in different directories.

I'm interested in knowing which one works best for you (i.e. which one is the most resilient to virtual memory address space exhaustion/fragmentation and which one crashes less because of such issues).
With the recent memory leaks fixes that went into v1.26.12.2 (thanks to LL's viewer memplugs new branch), the glibc allocator might well do a better job than jemalloc, after all (albeit it's much slower, especially during forced memory releases back to the system which are triggered once every second in the Cool VL Viewer)...

Note that the no_jemalloc build might benefit (or not... this is also an interesting info I'm seeking for) from the "Private Memory Pools" feature (see in "Advanced" -> "Memory Management"), which could help reducing memory fragmentation (the jemalloc build uses the jemalloc arenas to achieve the same result and albeit the "Private Memory Pools" may also be enabled in it, I doubt very much it would bring any benefit while it would definitely cost some more (moderate) memory consumption overhead).

Note also that memory usage and virtual address space exhaustion/fragmentation can vary immensely from one session to another, even when ran in the same conditions (same sim/avatars/etc): the order in which the structures are built and destroyed by the viewer, the order in which the textures get loaded and discarded, and many other factors all contribute to make it a very chaotic process, thus my indecision and doubts about which allocator is best. Only statistics can help, and only you, the users, can help me gathering enough such statistics in a large enough amount of different situations...


2014-05-24 12:50:56
Profile WWW

Joined: 2009-03-17 18:42:51
Posts: 5523
Reply with quote
*Listens to the chirping crickets* Any Linux user, here, or should I stop publishing Linux builds altogether ?...


2014-05-31 08:09:46
Profile WWW

Joined: 2011-08-27 17:31:05
Posts: 98
Reply with quote
While usually I mostly use Linux, for SL I almost always use my Windows-Machine. It has the *much* better graphics card in it. Linux is mostly for work and has only Intel HD4600 integrated graphics. SL barely works with that.


2014-05-31 15:37:54
Profile

Joined: 2012-08-08 17:51:35
Posts: 84
Reply with quote
I'm here, but found the thread today :oops:

jemalloc has worked ok this far, I'll test the two versions coming week if time allows.


2014-05-31 15:55:16
Profile

Joined: 2009-03-17 18:42:51
Posts: 5523
Reply with quote
Nicolette Lefevre wrote:
While usually I mostly use Linux, for SL I almost always use my Windows-Machine. It has the *much* better graphics card in it. Linux is mostly for work and has only Intel HD4600 integrated graphics. SL barely works with that.
Install a dual-boot with Linux as an alternative to your Windows machine: you will be surprised at how much faster Linux is with an OpenGL application such as the viewer...


2014-05-31 17:15:34
Profile WWW

Joined: 2010-04-07 08:23:18
Posts: 210
Reply with quote
Hello Henri,

I've been running the version without jemalloc and with private memory pools enabled ever since you released it. It's been at least as rock solid as its immediate predecessors, and it feels more efficient (when watching VIRT vs. RES in top output). I did make a point of going to Bondage Ranch (ugh!) and camming, which, as you know, is my personal viewer memory stress test :lol: . I did manage to crash it, albeit "gracefully", i.e. with stack trace, after it failed to allocate memory for Vertex data. (I'd upload the log, alas, it's around 100kB even compressed -- it was a looooong session)

Again, subjectively it felt like it took a lot longer than before to achieve that state. But of course it could be all in my mind. But from my point of view, jemalloc can go to /dev/null -- for me the viewer is as rock solid without it as before, and, as stated, seems a bit more efficient.

No less, I can't wait for LL to finally release a 64 bit version of the viewer in hope that Cool VL Viewer will then in turn get its 64 bit version eventually as well. ;)

Love,
Lia


2014-06-01 04:39:44
Profile

Joined: 2009-03-17 18:42:51
Posts: 5523
Reply with quote
Amalia Illios wrote:
Hello Henri,

I've been running the version without jemalloc and with private memory pools enabled ever since you released it. It's been at least as rock solid as its immediate predecessors, and it feels more efficient (when watching VIRT vs. RES in top output). I did make a point of going to Bondage Ranch (ugh!) and camming, which, as you know, is my personal viewer memory stress test :lol: . I did manage to crash it, albeit "gracefully", i.e. with stack trace, after it failed to allocate memory for Vertex data. (I'd upload the log, alas, it's around 100kB even compressed -- it was a looooong session)
The stack trace (together with the last few lines of the log, starting just before the crash manager gets called) would have been interesting to see... It could help pin-pointing the use of the NULL vertex data buffer pointer from that failed allocation (something LL's code is literally riddled with, and that I tried to fix everywhere I could spot it in the Cool VL Viewer) and thus help plugging one more crash bug case on memory allocation failure conditions...

Quote:
Again, subjectively it felt like it took a lot longer than before to achieve that state. But of course it could be all in my mind.
Yes, one must be wary of the placebo effect. I got the same impression myself, but I still need to gather more data/statistics to tell for sure.

Quote:
But from my point of view, jemalloc can go to /dev/null -- for me the viewer is as rock solid without it as before, and, as stated, seems a bit more efficient.
If by "more efficient" you mean that jemalloc used more memory, then yes: jemalloc uses 16 bytes aligned memory allocations against 8 bytes for glibc's malloc(). On the other hand, jemalloc is much faster than glibc's allocator. The third characteristic of a memory allocator that I'm trying to measure here is the resilience to memory fragmentation: it's the most difficult to measure reliably (because of the chaotic process I described in my first post).

Quote:
No less, I can't wait for LL to finally release a 64 bit version of the viewer in hope that Cool VL Viewer will then in turn get its 64 bit version eventually as well. ;)
Eventually and for Linux, it will, and yes, it will depend on LL's own actions towards 64bits viewers support.


2014-06-01 08:45:04
Profile WWW

Joined: 2012-08-08 17:51:35
Posts: 84
Reply with quote
After a quick stress test jumping around on busy sims with lots of mesh (that's my stress test :lol: ) I don't have any real data on it, but I got the feeling that jemalloc version of the viewer is more responsive somehow but uses one or two % more memory than the glib version.

I'll alternate for a couple of days to see if I can get any grip of the reliability over time ;)


2014-06-01 09:50:24
Profile

Joined: 2010-04-07 08:23:18
Posts: 210
Reply with quote
Hello again Henri,

Henri Beauchamp wrote:
The stack trace (together with the last few lines of the log, starting just before the crash manager gets called) would have been interesting to see... It could help pin-pointing the use of the NULL vertex data buffer pointer from that failed allocation (something LL's code is literally riddled with, and that I tried to fix everywhere I could spot it in the Cool VL Viewer) and thus help plugging one more crash bug case on memory allocation failure conditions...


That can be arranged. :D

Code:
2014-05-29T13:15:49Z WARNING: LLCore::HttpPolicy::stageAfterCompletion: HTTP request 0xe2d92638 failed after 0 retries. Reason: Not Found (Http_404)
2014-05-29T13:15:49Z WARNING: LLTextureFetchWorker::onCompleted: Texture: 3899284e-6c06-cf6e-9a12-950cade4dd1b CURL GET FAILED, status: Http_404 - reason: Not Found
2014-05-29T13:15:50Z WARNING: LLViewerTexture::setIsMissingAsset: 3899284e-6c06-cf6e-9a12-950cade4dd1b: Marking image as missing
2014-05-29T13:16:17Z INFO: display_stats: FPS: 35.00
2014-05-29T13:16:30Z WARNING: LLMemory::allocationFailed: Memory allocation failure for size: 4888576
2014-05-29T13:16:30Z INFO: LLMemory::logMemoryInfo: System memory information: Max physical memory: 32865876KB - Allocated physical memory: 32328KB - Available physical memory: 32865876KB - Allocated virtual memory: 3642446KB
2014-05-29T13:16:30Z INFO: LLMemory::logMemoryInfo: Private memory pool information: Reserved memory: 135216KB - Allocated memory: 118246KB
2014-05-29T13:16:30Z WARNING: LLVertexBuffer::mapVertexBuffer: Memory allocation for Vertex data failed. Ignoring, but do expect a crash soon...
2014-05-29T13:16:30Z INFO: do_elfio_glibc_backtrace: Opening stack trace file /home/amalia/.secondlife/logs/stack_trace.log
2014-05-29T13:16:30Z INFO: do_elfio_glibc_backtrace: Finished generating stack trace.
2014-05-29T13:16:30Z INFO: LLAppViewer::handleViewerCrash: Handle viewer crash entry.
2014-05-29T13:16:30Z INFO: LLMemory::logMemoryInfo: System memory information: Max physical memory: 32865876KB - Allocated physical memory: 32328KB - Available physical memory: 32865876KB - Allocated virtual memory: 3643176KB
2014-05-29T13:16:30Z INFO: LLMemory::logMemoryInfo: Private memory pool information: Reserved memory: 135216KB - Allocated memory: 118246KB
2014-05-29T13:16:30Z INFO: LLAppViewer::handleViewerCrash: Creating crash marker file /home/amalia/.secondlife/logs/CoolVLViewer.error_marker
2014-05-29T13:16:30Z INFO: LLAppViewer::handleViewerCrash: Created marker file /home/amalia/.secondlife/logs/CoolVLViewer.error_marker
2014-05-29T13:16:30Z INFO: LLAppViewer::handleViewerCrash: Handle viewer crash generating stats log.
2014-05-29T13:16:30Z INFO: LLAppViewer::writeDebugInfo: Opening debug file /home/amalia/.secondlife/logs/debug_info.log


Code:
0   com.secondlife.indra.viewer         0x8ef1a77 LLAppViewerLinux::handleSyncCrashTrace() + 243
1   com.secondlife.indra.viewer         0x819f098 LLAppViewer::handleSyncViewerCrash() + 16
2   unknown                             0xf70c84f1 /home/CoolVLViewer-1.26.12.2/lib/libllcommon.so(_ZN5LLApp19runSyncErrorHandlerEv+0x1d) [0xf70c84f1]
3   unknown                             0xf70c85bd /home/CoolVLViewer-1.26.12.2/lib/libllcommon.so(_ZN5LLApp8setErrorEv+0x1d) [0xf70c85bd]
4   unknown                             0xf70c9320 /home/CoolVLViewer-1.26.12.2/lib/libllcommon.so(+0x41320) [0xf70c9320]
5   unknown                             0xf7737410 [0xf7737410]
6   com.secondlife.indra.viewer         0x9380a79 LLVector4a::memcpyNonAliased16(float*, float const*, unsigned int) + 161
7   com.secondlife.indra.viewer         0x8270b1c LLFace::getGeometryVolume(LLVolume const&, int const&, LLMatrix4 const&, LLMatrix3 const&, unsigned short const&, bool) + 18146
8   com.secondlife.indra.viewer         0x8e077e1 LLVolumeGeometryManager::genDrawInfo(LLSpatialGroup*, unsigned int, LLFace**, unsigned int, bool, bool, bool) + 2699
9   com.secondlife.indra.viewer         0x8e0a009 LLVolumeGeometryManager::rebuildGeom(LLSpatialGroup*) + 1943
10  com.secondlife.indra.viewer         0x89a870a LLSpatialGroup::rebuildGeom() + 40
11  com.secondlife.indra.viewer         0x8e7148e LLPipeline::rebuildPriorityGroups() + 118
12  com.secondlife.indra.viewer         0x8e91bbb LLPipeline::postSort(LLCamera&) + 153
13  com.secondlife.indra.viewer         0x8e92d32 LLPipeline::stateSort(LLCamera&, LLCullResult&) + 804
14  com.secondlife.indra.viewer         0x8e96be9 LLPipeline::renderShadow(glh::ns_float::matrix4&, glh::ns_float::matrix4&, LLCamera&, LLCullResult&, bool, bool, unsigned int) + 327
15  com.secondlife.indra.viewer         0x8e9b29b LLPipeline::generateSunShadow(LLCamera&) + 14943
16  com.secondlife.indra.viewer         0x8ad4f22 display(bool, float, int, bool) + 4619
17  com.secondlife.indra.viewer         0x81aea42 LLAppViewer::mainLoop() + 1232
18  com.secondlife.indra.viewer         0x8ef0e73 main + 312
19  unknown                             0xf63394d3 /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0xf63394d3]
20  unknown                             0x810d181 ./bin/cool_vl_viewer-bin() [0x810d181]


Hope this helps!

Love,
Lia


2014-06-01 10:02:13
Profile

Joined: 2009-03-17 18:42:51
Posts: 5523
Reply with quote
Amalia Illios wrote:
Code:
.../...
5   unknown                             0xf7737410 [0xf7737410]
6   com.secondlife.indra.viewer         0x9380a79 LLVector4a::memcpyNonAliased16(float*, float const*, unsigned int) + 161
7   com.secondlife.indra.viewer         0x8270b1c LLFace::getGeometryVolume(LLVolume const&, int const&, LLMatrix4 const&, LLMatrix3 const&, unsigned short const&, bool) + 18146
.../...
Hope this helps!
Lovely ! One more crash bug plugged for next releases !


2014-06-01 12:15:35
Profile WWW
Display posts from previous:  Sort by  
Reply to topic   [ 14 posts ]  Go to page 1, 2  Next

Who is online

Users browsing this forum: No registered users and 31 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group
Designed by ST Software.