Cool VL Viewer forum

View unanswered posts | View active topics It is currently 2022-09-24 18:58:24



Reply to topic  [ 46 posts ]  Go to page Previous  1, 2, 3, 4, 5  Next
IO Completion Port based LLLFSThread for windows & Linux 
Author Message

Joined: 2009-03-17 18:42:51
Posts: 4992
Reply with quote
Here is the actual v6 patch I am currently using (with no joy):
Attachment:
io_uring-v6b.patch.gz [8.35 KiB]
Downloaded 48 times


2021-10-01 20:56:53
Profile WWW

Joined: 2016-06-19 21:33:37
Posts: 280
Location: Columbus area, OH, USA
Reply with quote
Henri Beauchamp wrote:
Here is the actual v6 patch I am currently using (with no joy):
Attachment:
io_uring-v6b.patch.gz
I downloaded this and did a completely fresh build (still using new curl though). I have cleared disk cache (ramdisk), for good measure. Sounds still play fine for me and, like Kathrine, I see .dsf files created in cache_coolvlviewer.


2021-10-01 21:25:08
Profile

Joined: 2009-03-17 18:42:51
Posts: 4992
Reply with quote
ZaneZimer wrote:
I downloaded this and did a completely fresh build (still using new curl though). I have cleared disk cache (ramdisk), for good measure. Sounds still play fine for me and, like Kathrine, I see .dsf files created in cache_coolvlviewer.
This might be a race condition issue, then... Meaning the code is far from reliable/ready as is, sadly.


2021-10-01 21:28:20
Profile WWW

Joined: 2016-06-19 21:33:37
Posts: 280
Location: Columbus area, OH, USA
Reply with quote
Henri Beauchamp wrote:
This might be a race condition issue, then... Meaning the code is far from reliable/ready as is, sadly.
That could be. My newer system is pretty robust. I am using the affinity settings too. I even wondered if it might be related to a gcc version difference. I'm using gcc version 11.2.1 20210728 (Red Hat 11.2.1-1) (GCC).


2021-10-01 21:33:23
Profile

Joined: 2009-03-17 18:42:51
Posts: 4992
Reply with quote
It does not make any difference for me if I use gcc 5.5 or 11.2, or clang 12.0. I also see a few .dsf files created for the rare sounds that do play but that's it. Also getting "WARNING: LLQueuedThread::shutdown: Called with active requests: 1" on exit, meaning the LFS thread got somehow stuck...


2021-10-01 21:57:22
Profile WWW

Joined: 2016-06-19 21:33:37
Posts: 280
Location: Columbus area, OH, USA
Reply with quote
Henri Beauchamp wrote:
Also getting "WARNING: LLQueuedThread::shutdown: Called with active requests: 1" on exit, meaning the LFS thread got somehow stuck...
I just checked my log file for the last session and I don't have any of those warns. It really does seem 'racy' if threads are getting 'stuck'. Another difference that just occurred to me is that I build with ' --tune --usesystemlibs'. Not sure how those would come into play but I can try to build without those and test.

*Edit: Building without those did not allow me to repro the issue.


2021-10-01 22:17:06
Profile

Joined: 2009-03-17 18:42:51
Posts: 4992
Reply with quote
I added the code to today's v1.28.2.42, but it is disabled in 00-BuildOptions.cmake since the race condition keeps ruining sounds (and probably other assets) on my system. I slightly improved the Linux code for LLLFSThread::threadedUpdate(), but it does not fix the race condition (which is likely a completed request that does not properly self-delete).

I believe this is the same issue I encountered already (and worked around) for the texture cache (see the comment I wrote in indra/newview/lltexturecache.cpp starting at line 1004). This could also be an underlying bug of the multi-threaded (FLAG_ASYNC) queue thread code... Finally there is some code in the image decode worker (which works fine in this respect) that does not exist in LLLFSThread for dealing with FLAG_ASYNC requests.


2021-10-02 10:24:54
Profile WWW

Joined: 2011-10-07 10:39:20
Posts: 155
Reply with quote
Thanks Henri.

Will try to figure out what is really going on there. Something just isn't quite right yet with fetching the completions and triggering the Responders.

The LLQueuedThread stuff is kind of strange with the pause()/runCondition() code, where typical threaded code would just use something like WaitForMultipleObjects() or other standard signaling instead of yielding/ms_sleep() in there. Cramming the other stuff inside makes it a bit tricky.

If one wrote the LLLFSThread code from scratch for Io_uring & IOCP, it would probably look a little different overall. The write()/read() calls would probably directly queue up the sqe/OVERLAPPED calls in the main thread, instead of handing that off via m_queue to the extra thread. And then the extra thread would only handle the completions and block on io_uring_wait_cqe or the equivalent GetQueuedCompletionStatus() and fire off the Responders instead of busy looping on that run condition. And the whole priority thing gets moot, if you just queue up ALL of the waiting requests to the OS to handle in parallel (assuming the I/O-bandwith of a current NVMe SSD is more than enough to satisfy all the request in one go anyway). I might give that a try if the complexity of the current scheme stays too high for shaking out the races.

Kathrine

Edit:
Yes, your comment about lltexturecache looks pretty similar. The same issues happens with sounds basically, but the other way round, it gets signaled as completed(failed) instead of just waiting.


2021-10-02 12:11:25
Profile

Joined: 2011-10-07 10:39:20
Posts: 155
Reply with quote
Hi Henri,

i added a little patch on top of the .43 code.

It adds a counter for not yet completed async calls and runCondition that keeps the thread awake while it is still waiting for completions.

Hopefully it works for you and fixes the sound issues/other problems.

Kathrine


Attachments:
io_uring_extra.patch.gz [1.31 KiB]
Downloaded 42 times
2021-10-05 23:41:10
Profile

Joined: 2009-03-17 18:42:51
Posts: 4992
Reply with quote
Thank you Kathrine !

Sadly, this does not solve the sounds issue for me: basically, the first sound gets properly decoded as a .dsf file (but not played), then the sLocal thread stays stuck forever and I get "WARNING: LLQueuedThread::shutdown: Called with active requests: 1" in the log at viewer shutdown...

Note: there are two small bugs in your patch: mPendingResults should be initialized to 0 in the constructor and "setRequestResult(req, true);" should be used in place of "setRequestResult(req, false);" for the Windows code path in case of failure (else the request might not auto-complete; the error condition is flagged by req->setBytesRead(0) and not by the false flag in the request result method).

EDIT: I found the problem. LLLFSThread::runCondition() must return true whenever mPendingResults is non-zero (it was tested as !(mPendingResults > 0), i.e. the other way around)... This fixed patch seems to work fine (still testing it):
Attachment:
io_uring_extra-v3.patch.gz [1.56 KiB]
Downloaded 47 times


2021-10-06 09:41:12
Profile WWW
Display posts from previous:  Sort by  
Reply to topic   [ 46 posts ]  Go to page Previous  1, 2, 3, 4, 5  Next

Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group
Designed by ST Software.