Cool VL Viewer forum http://sldev.free.fr/forum/ |
|
IO Completion Port based LLLFSThread for windows & Linux http://sldev.free.fr/forum/viewtopic.php?f=10&t=2222 |
Page 3 of 5 |
Author: | Henri Beauchamp [ 2021-10-01 20:56:53 ] | |||||||||
Post subject: | Re: IO Completion Port based LLLFSThread for windows & Linux | |||||||||
Here is the actual v6 patch I am currently using (with no joy):
|
Author: | ZaneZimer [ 2021-10-01 21:25:08 ] | ||||||||||||||||||
Post subject: | Re: IO Completion Port based LLLFSThread for windows & Linux | ||||||||||||||||||
|
Author: | Henri Beauchamp [ 2021-10-01 21:28:20 ] | |||||||||
Post subject: | Re: IO Completion Port based LLLFSThread for windows & Linux | |||||||||
|
Author: | ZaneZimer [ 2021-10-01 21:33:23 ] | |||||||||
Post subject: | Re: IO Completion Port based LLLFSThread for windows & Linux | |||||||||
|
Author: | Henri Beauchamp [ 2021-10-01 21:57:22 ] |
Post subject: | Re: IO Completion Port based LLLFSThread for windows & Linux |
It does not make any difference for me if I use gcc 5.5 or 11.2, or clang 12.0. I also see a few .dsf files created for the rare sounds that do play but that's it. Also getting "WARNING: LLQueuedThread::shutdown: Called with active requests: 1" on exit, meaning the LFS thread got somehow stuck... |
Author: | ZaneZimer [ 2021-10-01 22:17:06 ] | |||||||||
Post subject: | Re: IO Completion Port based LLLFSThread for windows & Linux | |||||||||
*Edit: Building without those did not allow me to repro the issue. |
Author: | Henri Beauchamp [ 2021-10-02 10:24:54 ] |
Post subject: | Re: IO Completion Port based LLLFSThread for windows & Linux |
I added the code to today's v1.28.2.42, but it is disabled in 00-BuildOptions.cmake since the race condition keeps ruining sounds (and probably other assets) on my system. I slightly improved the Linux code for LLLFSThread::threadedUpdate(), but it does not fix the race condition (which is likely a completed request that does not properly self-delete). I believe this is the same issue I encountered already (and worked around) for the texture cache (see the comment I wrote in indra/newview/lltexturecache.cpp starting at line 1004). This could also be an underlying bug of the multi-threaded (FLAG_ASYNC) queue thread code... Finally there is some code in the image decode worker (which works fine in this respect) that does not exist in LLLFSThread for dealing with FLAG_ASYNC requests. |
Author: | kathrine [ 2021-10-02 12:11:25 ] |
Post subject: | Re: IO Completion Port based LLLFSThread for windows & Linux |
Thanks Henri. Will try to figure out what is really going on there. Something just isn't quite right yet with fetching the completions and triggering the Responders. The LLQueuedThread stuff is kind of strange with the pause()/runCondition() code, where typical threaded code would just use something like WaitForMultipleObjects() or other standard signaling instead of yielding/ms_sleep() in there. Cramming the other stuff inside makes it a bit tricky. If one wrote the LLLFSThread code from scratch for Io_uring & IOCP, it would probably look a little different overall. The write()/read() calls would probably directly queue up the sqe/OVERLAPPED calls in the main thread, instead of handing that off via m_queue to the extra thread. And then the extra thread would only handle the completions and block on io_uring_wait_cqe or the equivalent GetQueuedCompletionStatus() and fire off the Responders instead of busy looping on that run condition. And the whole priority thing gets moot, if you just queue up ALL of the waiting requests to the OS to handle in parallel (assuming the I/O-bandwith of a current NVMe SSD is more than enough to satisfy all the request in one go anyway). I might give that a try if the complexity of the current scheme stays too high for shaking out the races. Kathrine Edit: Yes, your comment about lltexturecache looks pretty similar. The same issues happens with sounds basically, but the other way round, it gets signaled as completed(failed) instead of just waiting. |
Author: | kathrine [ 2021-10-05 23:41:10 ] | ||
Post subject: | Re: IO Completion Port based LLLFSThread for windows & Linux | ||
Hi Henri, i added a little patch on top of the .43 code. It adds a counter for not yet completed async calls and runCondition that keeps the thread awake while it is still waiting for completions. Hopefully it works for you and fixes the sound issues/other problems. Kathrine
|
Author: | Henri Beauchamp [ 2021-10-06 09:41:12 ] | |||||||||
Post subject: | Re: IO Completion Port based LLLFSThread for windows & Linux | |||||||||
Thank you Kathrine ! Note: there are two small bugs in your patch: mPendingResults should be initialized to 0 in the constructor and "setRequestResult(req, true);" should be used in place of "setRequestResult(req, false);" for the Windows code path in case of failure (else the request might not auto-complete; the error condition is flagged by req->setBytesRead(0) and not by the false flag in the request result method). EDIT: I found the problem. LLLFSThread::runCondition() must return true whenever mPendingResults is non-zero (it was tested as !(mPendingResults > 0), i.e. the other way around)... This fixed patch seems to work fine (still testing it):
|
Page 3 of 5 | All times are UTC |
Powered by phpBB® Forum Software © phpBB Group https://www.phpbb.com/ |