This is the mail archive of the
libc-help@sourceware.org
mailing list for the glibc project.
Serialization of AIO requests to the same fd in posix-aio
- From: Anthony Liguori <aliguori at us dot ibm dot com>
- To: Ulrich Drepper <drepper at redhat dot com>
- Cc: libc-help at sourceware dot org, Ryan Arnold <rsa at us dot ibm dot com>
- Date: Wed, 08 Oct 2008 15:20:00 -0500
- Subject: Serialization of AIO requests to the same fd in posix-aio
Hi Ulrich,
In sysdeps/pthread/aio_misc.c:__aio_enqueue_request(), there is a check
to see if an existing thread is working on the same file descriptor as
the request being enqueued. If it is, it queues it to run after the
existing request instead of possibly spawning a new thread.
The net effect is that submitting multiple requests to the same file
descriptor are always serialized. This significantly affects performance.
The comment beneath this check says:
/* The current file descriptor is worked on. It makes no sense
to start another thread since this new thread would fight
with the running thread for the resources. But we also cannot
say that the thread processing this desriptor shall immediately
after finishing the current job process this request if there
are other threads in the running queue which have a higher
priority. */
I don't understand what "fight with the running thread" is supposed to
suggest. Is this suggesting that there is some fundamental race
condition? Is it assuming that two IO requests cannot happen at the
same time at the kernel level? Since the threads end up using
pread/pwrite, it should be safe to have two request executing
simultaneously.
Currently, we use posix-aio for disk IO in QEMU/KVM. To achieve good
performance, we have to implement a fd pool using dup() to work around
this check. The difference between using the pool and not using the
pool is around 400% in write performance.
Can you further elaborate on what this check is trying to prevent?
Thanks,
Anthony Liguori