I don't get what exactly you are trying to imply here. The CPU needs to "calculate" where it needs to write to the compressed archive?? Where did you read about that?? In any case, I would agree that the two methods would have different results. Writing directly to the final file would be faster than writing to a temp file and then copying it to the final folder. But that doesn't change the fact that the test results should still NOT be manipulated in any way. Even if you want to find out the pure compression speed of the different data compressors, deducting the copy time from the results will still result in skewed data. It will slant the results in favour of data compressors that write directly to the final file. Exactly why manipulating the results would make them totally inaccurate and the comparison unfair. Oh, I see! Now that makes some sense. LOL! Okay, so you just wanted to see how long it takes to JUST copy the UNcompressed files from one test hard drive to the other? Yeah, that would make for an interesting comparison. Err.. Actually, hard drive caching always affects performance. That's why we cannot just manipulate the data. It's not like MS-DOS where you can turn off hard drive caching. Oh yeah, that definitely cleared up the confusion. LOL!! Looks like it was all one big messed-up discussion. I think I get what you mean now. You only want me to add a straight copy result to provide an additional basis for comparison, right? Yes, that would make for an interesting comparison, but readers should be informed NOT to deduct it from the current results to "obtain" pure compression speeds. That would be totally inaccurate for the reasons already explained.
Thanks. Sorry about the long time I'm taking. Been super busy, plus the big CNY holidays are coming soon.
Not "where" but "what" to write. The compression is a manipulation (altering) of the original data, and the CPU is used to figure out exactly what the new file should "look" like. Yes you can (System properties / Advanced => Performance / Settings => Advanced / Virtual memory / Change => "No swap file" / Activate), but shouldn't be done. That's why I make a point of compressing relatively small amounts of data, so that the swap file doesn't come into play. Pure compression speed is pretty irrelevant to me, the total time used/gained/lost vs compression rating is far more important. Cheers Olle
Err.. The compressors certainly use the processor to do that. In fact, as mentioned earlier, I intentionally used a slower processor to reduce the effect of the writing data on the actual compression time. Huh?? Swap file? We are talking about hard drive CACHING, remember? The paging file or swap file is for virtual memory. It has no bearing whatsoever on hard drive caching. The compression speed is a reflection of the actual compression times. I can probably show you the compression times but it will just be a reverse of the compression speed charts. Incidentally, talking about showing the pure copy time of the files from one hard drive to another... While I think it's an interesting comparison, when I had the time to actually think about it, it's a comparison of limited use and one that very few people would be interested in. The only reason why we would want to compare pure copy times with compression times would be if we are used to copying data from one location to another and wanted to see how much longer it would take to compress and move it to that final location. Other than that.. what possible use could pure copy results have? I think it will only confuse the other readers who will be wondering why we are comparing the time it takes to copy the files from one location to another with the time it takes to compress those files. In many cases, this does not serve as a valid baseline as many people just compress the files in the same hard drive, and delete the originals afterwards. The pure copy speed will therefore make no sense at all.
The article has just been updated! Data compression are now part of everyday life. We compress everything from pictures and video clips to documents and more. But just which data compressor is suitable? There are so many of them! This guide will help you choose the best data compressor for the job you have on hand. Read on and find out! The update is:- Link : Compression Comparison Guide Rev. 2.0 Part 1!
Compression Comparison Guide Rev. 2.0 Parts 1 & 2 Posted Data compression are now part of everyday life. We compress everything from pictures and video clips to documents and more. But just which data compressor is suitable? There are so many of them. This guide will help you choose the best data compressor for the job you have on hand. We have just added Normal Compression benchmarks to the Fast Compression results we have posted earlier. Read on and find out. The update is:- Link : Compression Comparison Guide Rev. 2.0 Parts 1 & 2
Another good page is the Practical Compressor Test, which plots the relation between compression ratio and speed for a number of compression algorithms, this gives another interesting look at the same data.
In the table on page 2 of the article, "Best Compression" is listed as TBA at the moment of this writing. Does this mean it isn't tested or does it mean it's just not written down? Because if it isn't tested, then the test is pretty much useless. Not testing e.g. 7-Zip on "Ultra" compression doesn't explore its capabilities at all. The default level is a compromise between speed and compression level and can't be used for any measurement imo. At least not considering the range of different values on the default compression in the different archivers.
Hello asbjornu, We are releasing it in stages because it takes a LONG time to test these compressors using so many filesets. We are releasing the results as we obtain them. Needless to say, the next part to be posted will concentrate on their maximum compression. And then we will compare all data compressors at different settings and add new data compressors. This is an on-going effort that has just started, and will continue over time. As for the default level, we are testing that for a very simple reason. Many users are not bothered or not comfortable changing the compression settings. So, it makes sense to test the default settings for those who like to use the data compressors as they come.
Hardware accelerated compression A few years ago I worked for a company that developed a pci-x based gzip compression acceleration card. Currently the board supports over 3.0 Gbits/sec. If your interested check out www.aha.com.
Why wasn't LZO included? http://www.oberhumer.com/opensource/lzo/ If/When you do a Part II, please include LZO. I'd be interested in how fast "can be used for real time" can actually be. Thanks!
TBH, gzip seems fast enough as it is. We are even using it for on-the-fly compression on our server. But I guess for really, really serious traffic, it wouldn't hurt to offload the processing.
Hi, LZO wasn't included because we thought we would start with the most common formats first and work our way through the other compressors as we go along. We will include LZO once we are done with the current slew of data compressors.
Winzip Upgrades The article was great and a super resource of technical information. However there is one flaw. I was looking at the WINZIP website just yesterday, and upgrades aren't free forever. If you have versions older than 11.0 then there is an upgrade fee to get 11.1.
PKWare? I'm curious why was PkWare excluded? I would be curious to see how it falls in with the rest of the compressors.
Thanks for the info... Very well done article. Thanks for doing research on something I would have never done myself but place the results in an "easy to read" format. Makes my choice easy when selecting a Zip packager...
Hi! It takes time to set up the system properly and then benchmark them. At the time we started the tests, they only had version 11. But if you check the revision details, you will note that they were merely minor updates. Nothing that would affect its compression performance or speed. However, we intend to regularly update when new revisions appear, especially major revisions that affect the performance of these data compressors in any way.
We did not exclude PKZIP on purpose. We merely started with a few data compressors. We chose WinZip because it is arguably more common than PKZIP. Also, it would serve as a general representative of the performance of ZIP archives. As we go along, we will add more and more data compressors, like PKZIP.
Thank you! Do come back for more. As it takes time, we will be posting updates over time. The next one will involve maximum compression, which should be very, very interesting.
