xtracopy is a file copy tool designed to backup large files in a fast and efficient way. We use this tool mainly to backup the virtual machine images, but it can be used to backup any kind of file.

Its strength is the high performance even when the system is actively working with other tasks taking a lot of I/O resources on its volume. This is a common scenario in virtualization environments, when the system performs the backup of the virtual machines while these are running.

The performances are achieved by implementing a multi-threaded architecture with lockless ring buffers.

How it works?

• the read thread gets the binary slices from the input file and store that in the read buffer
• the compressor thread gets the binary slices from the read buffer, compress or decompress each slice and put the result in the write buffer
• the write thread gets the binary slices from the write buffer and writes that in the output file

Why this architecture?

• the read thread can benefit from the high speed of the volume while using the band left free by main tasks. It reads all slices that it can and insert them in the read buffer
• the compressor thread can use the available CPU to compress/decompress the slices from the read buffer into the write buffer, without be affected by the I/O waiting on the volumes
• the write thread serializes the slices contained in the write buffer in the output file using the maximum I/O bandwidth of the output volume

So, what are the benefits?

• xtracopy can compress and copy large files without using any temporary file
• it can use all I/O bandwidth available in the input and output volumes
• it can optimize the use one CPU Core for the compression
• the two ring buffers allow to partially compensate the moments when the I/O bandwidth are occupied by other tasks
• xtracopy can run with a low priority keeping good performances without affecting the main processes

For any question, please contact us.

Download xtracopy-0.0.1.tar.gz