Cool VL Viewer forum
http://sldev.free.fr/forum/

IO Completion Port based LLLFSThread for windows & Linux
http://sldev.free.fr/forum/viewtopic.php?f=10&t=2222
Page 1 of 5

Author:  kathrine [ 2021-09-19 15:46:39 ]
Post subject:  IO Completion Port based LLLFSThread for windows & Linux

Similar transformation as the texture processing thread patch.

Pushing the actual file reads to a thread pool (Kernel pool) via the Windows IOCP functions and fetching the completions later.
A very similar patch can be done for Linux io_uring, just needs switching a few functions and structures around (see https://unixism.net/loti/tutorial/register_eventfd.html for the basic idea).
(on my list of todos, when i get my Linux build environ going again).

This makes a bit better use of the huge command queues on modern SSDs/systems. Just let the OS figure out the best I/O sequence and do not bother to block the thread with a blocking read()/write() call.

It seems to work nicely, been running the viewer with the patch for a few days now without issues, and it feels a little more snappy when having a large draw distance and a lot of texture loading.

Attachments:
iocp.zip [2.61 KiB]
Downloaded 158 times

Author:  Henri Beauchamp [ 2021-09-19 17:53:37 ]
Post subject:  Re: IO Completion Port based LLLFSThread for windows

Many thanks !

I'll have a look, though probably not before a couple of weeks, due to RL constraints next week... With some luck, by that time, you'll have a Linux version ready... :D

Author:  kathrine [ 2021-09-29 23:43:26 ]
Post subject:  Re: IO Completion Port based LLLFSThread

I added a Linux io_uring based version in addition to the Windows IOCP based one.

The Linux version needs at least Kernel version 5.6 and liburing lib and headers installed, the Windows Version should work with any NT based version.

The patch is pretty basic, but seems to work without crashes or bigger oddities i noticed. It felt a little snappier, but thats probably subjective.
Currently it handles results right away, it might be better to first queue all pending events before processing the results in batches, but that would have needed a bit more work with the QueuedThread baseclass that i didn't want to do (yet).

Attachments:
async_io_lllfsthread.patch.gz [4.72 KiB]
Downloaded 181 times

Author:  ZaneZimer [ 2021-09-30 13:46:49 ]
Post subject:  Re: IO Completion Port based LLLFSThread

kathrine wrote:
It felt a little snappier, but thats probably subjective.
I have applied and been testing this patch. For me, it does seem to improve scene loading immediately on login/teleport/region crossing, even though I'm also using a ramdisk for the viewer cache. Thanks for continuing to investigate improvements for modern hardware/OSes, Kathrine.

Author:  Henri Beauchamp [ 2021-09-30 16:14:35 ]
Post subject:  Re: IO Completion Port based LLLFSThread for windows & Linux

I'm back and could have a try at io_uring under Linux (the Windows build compiles fine but is untested so far).

Beside adding an UseIOUring debug setting to allow (de)activating it on demand (after a restart), I also added a runtime Linux kernel version detection (so to deactivate it automatically when running on kernels older than v5.6).

However, as is, the code crashed almost immediately on login for me.

I already spotted and fixed the following issues/bugs:
  • The processNextRequest() method should be made virtual in the LLQueuedThread class and declared as an override in the LLLFSThread class.
  • For liburing/Linux, the opening for writing should also have the O_CREAT flag added ("int outfile = open(mFileName.c_str(), O_CREAT | O_WRONLY);"), else the viewer cannot create cache files.
  • For liburing/Linux, in LLLFSThread::processNextRequest(), cqe and req must be checked against NULL return values and cause the while() loop exit when NULL.
  • For both Linux and Windows, when hitting the "Unable to do async read/write" warning, an exit from the method should be done with a failed read/write condition (mBytesRead = 0; return true;).

Despite the above fixes (which seem to have solved the crashes), I'm still getting issues with sound file decoding ("cannot write file", despite the O_CREAT in the open() call)... Perhaps because I am using links to a ramdisk directory for cache directories...

Author:  kathrine [ 2021-09-30 16:50:31 ]
Post subject:  Re: IO Completion Port based LLLFSThread for windows & Linux

Hi Henri,

thx for taking a look. Never had a crash with it when trying (but only merged the linux/windows parts with ifdefs pretty late, so might have missed some parts there), but well, your fixes are obviously necessary and my oversights. Probably worked because i had a populated cache.

I noticed some sound issues after looking closer, yes. The background sounds like wind etc. are there, but other sounds and UI sounds seem to fail.

I'll see if i can spot the reason for the sound problems.

Kathrine

Author:  Henri Beauchamp [ 2021-09-30 17:55:10 ]
Post subject:  Re: IO Completion Port based LLLFSThread for windows & Linux

kathrine wrote:
Hi Henri,I noticed some sound issues after looking closer, yes.
This might be due to the fact you are using low level file operations with open() while LLLFSThread is normally using APR files... If the file gets opened for read via APR before LLLFSThread attempts to write to it and opens it with open(), there might be a file lock preventing it to do so. Just guess work from my part, since I did not investigate properly so far.

EDIT: in the Linux version, when mOffset is negative, seeking to the end of file is missing for reads and writes. Here is a fixed patch (hopefully, I did not break the Windows code when simplifying/factorizing it), that sadly still not resolves the sound issue:
Attachment:
io_uring-v3.patch.gz [8.64 KiB]
Downloaded 161 times

Author:  kathrine [ 2021-09-30 21:31:08 ]
Post subject:  Re: IO Completion Port based LLLFSThread for windows & Linux

Thanks, will give it a try.

I think i might have an idea what happens with the Sound.

The llAudioDecodeMgr also uses the global LLFSThread to write its decoded files.

It seems to manage to write one file (the click sound when opening a floater...), and than hangs and the LFS Thread never finishes the request for some reason (it gets a warning on exit that requests are pending in the LFS QueuedThread during shutdown). Will try to debug into it, to see what happens.

Author:  kathrine [ 2021-09-30 21:35:26 ]
Post subject:  Re: IO Completion Port based LLLFSThread for windows & Linux

Oh,

the mOffset stuff is kind of weird for READ...

The old code does an llassert(mOffsett >= 0) and then goes on to handle the case that mOffset <0 for seeking from the end. So that should basically never happen.

Code:
           llassert(mOffset >= 0);
      LLAPRFile infile;   // auto-closes
      infile.open(mFileName, LL_APR_RB, mThread->getLocalAPRFilePool());
      if (!infile.getFileHandle())
      {
         llwarns << "Unable to read file: " << mFileName << llendl;
         mBytesRead = 0; // fail
         return true;
      }
      S32 off;
      if (mOffset < 0)
      {
         off = infile.seek(APR_END, 0);
      }
      else
      {
         off = infile.seek(APR_SET, mOffset);
      }

Author:  Henri Beauchamp [ 2021-09-30 21:37:33 ]
Post subject:  Re: IO Completion Port based LLLFSThread for windows & Linux

I just noticed a bug in my patch: I inverted the QueuedRequest flags for sUseIoUring. It should read:
Code:
LLLFSThread::Request::Request(LLLFSThread* thread,
                       handle_t handle, U32 priority,
                       operation_t op, const std::string& filename,
                       U8* buffer, S32 offset, S32 numbytes,
                       Responder* responder)
:   QueuedRequest(handle, priority,
              sUseIoUring ? FLAG_AUTO_COMPLETE | FLAG_ASYNC
                       : FLAG_AUTO_COMPLETE),

Page 1 of 5 All times are UTC
Powered by phpBB® Forum Software © phpBB Group
https://www.phpbb.com/