1. Forum
  2. Newsgroups
  3. comp.programming

  • Somebody Got It The Wrong Way Round ...

    From Lawrence D'Oliveiro@ldo@nz.invalid to comp.programming on Wed Jul 16 23:21:59 2025
    From Newsgroup: comp.programming

    From <https://www.infoworld.com/article/4018856/4-tips-for-getting-started-with-free-threaded-python.html>:

    As an example, if you have a job that writes a lot of files,
    having each job in its own thread is less effective if each job
    also writes the file. This is because writing files is an
    inherently serial operation. A better approach would be to divide
    jobs across threads and use one thread for writing to disk. As
    each job finishes, it sends work to the disk-writing job. This
    way, jobs don’t block each other and aren’t themselves blocked by
    file writing.

    Actually, blocking system calls (whether for I/O or something else)
    only block the current thread. So having each thread do its own I/O
    should be faster than funnelling it all through one bottleneck thread.

    If you don’t want your worker threads blocked waiting for I/O to
    complete, then each worker context can be a pair of threads: one does
    the CPU-intensive stuff, while the other handles the blocking I/O.
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From c186282@c186282@nnada.net to comp.programming on Fri Jul 18 02:14:05 2025
    From Newsgroup: comp.programming

    On 7/16/25 7:21 PM, Lawrence D'Oliveiro wrote:
    From <https://www.infoworld.com/article/4018856/4-tips-for-getting-started-with-free-threaded-python.html>:

    As an example, if you have a job that writes a lot of files,
    having each job in its own thread is less effective if each job
    also writes the file. This is because writing files is an
    inherently serial operation. A better approach would be to divide
    jobs across threads and use one thread for writing to disk. As
    each job finishes, it sends work to the disk-writing job. This
    way, jobs don’t block each other and aren’t themselves blocked by
    file writing.

    Actually, blocking system calls (whether for I/O or something else)
    only block the current thread. So having each thread do its own I/O
    should be faster than funnelling it all through one bottleneck thread.

    SEEMS true ... but not ALWAYS.

    Software design has to be customized for the
    exact job in mind.

    If you don’t want your worker threads blocked waiting for I/O to
    complete, then each worker context can be a pair of threads: one does
    the CPU-intensive stuff, while the other handles the blocking I/O.


    --- Synchronet 3.21a-Linux NewsLink 1.2