Coverting Ebooks with Multiple Processes

Recently I’ve been converting a batch of ebooks into epub and mobi formats.  I used python tools, parts of my project MyBookShelf, which uses calibre and LibreOffice for all the hard work.   The conversion tool enables to run several conversion in parallel – in separate processes.  I wondered how the conversion will speed up with adding more processes.   I ran it  on my notebook with core i5 processor – two physical cores, each core can run two physical threads, 8GB memory. Graph below shows results for conversion of about 10 books into both formats.

Interesting thing for me is that only notable speed up is between 1 and 2 processes.   Not very much gain with running 3 or 4 –  looks like full utilization of HW threads is held back by I/O or memory speed limits?

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">