Compression: 32bit or 64bit?

[spwolf]

Hello Everyone,

Over the past few years, popularity of 64bit Windows has gone up through the roof and deservedly so since 64bit can address more than 4 GB of RAM (while 32bit can only address a bit more than 3 GB, which might not be sufficient today).

However, many people think that 64bit Windows is also significantly faster, and we quite often get asked why not build 64bit version of PA? Well answer is quite simple - our IDE (Delphi) does not support 64bit, so we can not make 64bit version of PA. That leads people to think that other utilities have (big) advantage over us in terms of speed/features/etc.

I asked our compression engineer what could the gain be and answer was between 5% and 10%. Which is nothing special really, but hey marketing wise, it is hard to convince people that it is only that much.

Recently, as part of our patchbeam engine, we developed special interface for two apps to talk between each other (which is usually not simple), and it worked fine. So we got this thought into our head - why not do the same for our compression engines? Specifically 7zip and unrar. It is actually not that much of big deal at all, we could do it and have 32bit app that has 64bit compression engine.

So while our engineers are perfecting PA 2011, i got into some testing... I took 100 MB TAR file (some files from Photoshop installation), and compressed/extracted it in 32bit and 64bit versions of Rar and 7zip.

Results were surprising.

Compression Results
7zip 32bit compression = 52.3s
7zip 64bit compression = 49.7s

Rar 32bit compression = 26.5s
Rar 64bit compression = 24.7s

So 7.3% for Rar and 5.2% for 7zip. Not bad? Not great either. Keep in mind that you shouldnt compare the times between the two, as 7zip has much stronger max compression and compressed the files 20% better.

Here comes the surprising part:

Extraction Results:

7zip 32bit extraction = 2.78s
7zip 64bit extraction = 2.87s

Rar 32bit extraction = 2.8s
Rar 64bit extraction = 3.0s

PA 32bit extraction = 2.68s

So 32bit is actually faster than 64bit during extraction... and what do we all do most? Extract files we download from Internet. Our optimized unrar 32bit extraction is 12% faster than 64bit Rar 4.0b3.

Question here is what to do - people will ask us to give them 64bit, even if it is actually slower than our current code. We can not ignore the big marketing that 64bit is and negative feedback we get when people realize there is no 64bit version. Should we ignore the test results and build the 64bit versions anyway?

[drteeth]

Hi,

Too many people think that 64-bit is inherently 'better' or 'faster'. They are wrong.The performance differences above would not be detectable in a double-blind test IMHO.

Basically, do whatever you have to to succeed. If it is not too much trouble, you could produce both types and say that 32 bit is faster. Then use your stats to see which version is more popular - could be very interesting.

DrT

[TBGBe]

I would suggest doing another test run with much larger archives (maybe >2GB) as that is where the advantage (if there is one) would show up.

For general use I don't see a "practical" advantage.

However, I know that my next PC purchase would be at least a 64 bit system - so (even as just a "marketing gimmick") I don't think you can avoid having a 64 bit version.

[spwolf]

there would be no difference when it comes to larger sizes, very little memory is actually used during extraction and in any case, compression engines do not take a lot of memory even at strongest/slowest settings (Nowhere close to 3GB).

[TBGBe]

Originally Posted by spwolf

... compression engines do not take a lot of memory even at strongest/slowest settings (Nowhere close to 3GB).

Is that why the ultra settings say (memory req)

[spwolf]

Originally Posted by TBGBe

Is that why the ultra settings say (memory req)

oh it takes memory, but not 3 GB :-). Easily 1-1.3 GB is taken by Ultra settings in 7zip... But thats far cry from what 32bit can address.

Big difference is number of registers, which is double than 32bit systems, so this is where 5%-10% in compression comes from. However there are drawbacks as shown by extraction times.

So basically perfect archiver would use 64bit for compression and 32bit for extraction :-)

[TBGBe]

Yes, but that is assuming that a user does not have any other applications running when using PA.

Anyway, I think the marketing angle outweighs any technical advantage in this case.

[Luxor]

Originally Posted by TBGBe

Anyway, I think the marketing angle outweighs any technical advantage in this case.

^^ This.
As that's all that some people will care about. They will have 64bit Windows and be convinced in their mind that software that offers 64bit options is the one they need and must have.

[gan]

I'm not saying that 64 bit is necessarily better and faster in all cases, but all the benchmarks/tests i have seen so far for WinRAR shows that the 64 bit edition is basically faster. These benchmarks show that 64 bit OS with 64 bit WinRAR is faster for all operations compared to 64 bit OS with 32 bit WinRAR, but 64 bit OS and 32 bit WinRAR is faster than 32 bit OS and 32 bit WinRAR which seems to be the slowest combination.

Not sure why the results differ for this test performed by PA.

-gan

[spwolf]

Originally Posted by gan

I'm not saying that 64 bit is necessarily better and faster in all cases, but all the benchmarks/tests i have seen so far for WinRAR shows that the 64 bit edition is basically faster. These benchmarks show that 64 bit OS with 64 bit WinRAR is faster for all operations compared to 64 bit OS with 32 bit WinRAR, but 64 bit OS and 32 bit WinRAR is faster than 32 bit OS and 32 bit WinRAR which seems to be the slowest combination.

Not sure why the results differ for this test performed by PA.

-gan

maybe compression benchmarks, but for extractions, it is definitely slower... there is nothing special with these benchmarks, they can be done by anyone.

Basically due to nature of design, 64bit extraction has a lot of redundancy compared to 32bit. There is nothing extra there to gain speed, unlike during the compression.

So you come into situation that 32bit PowerArchiver is significantly faster than 64bit WinRar v4 under same 64bit Windows 7 computer :-)

Of course, PA is always faster anyway due to super duper optimizations our Eugene used for unrar

[gan]

Originally Posted by spwolf

maybe compression benchmarks, but for extractions, it is definitely slower... there is nothing special with these benchmarks, they can be done by anyone.

Basically due to nature of design, 64bit extraction has a lot of redundancy compared to 32bit. There is nothing extra there to gain speed, unlike during the compression.

Well i'm not saying you are wrong and sure you guys know what you are doing I'm just saying all the benchmarks i have seen for WinRAR then WinRAR x64 performed better for both compression and extraction compared to WinRAR x86. If that means 64 bit is faster or if they done better work with the 64 bit version or if it doesn't mean anything at all i don't know.

-gan

[spwolf]

Originally Posted by gan

Well i'm not saying you are wrong and sure you guys know what you are doing I'm just saying all the benchmarks i have seen for WinRAR then WinRAR x64 performed better for both compression and extraction compared to WinRAR x86. If that means 64 bit is faster or if they done better work with the 64 bit version or if it doesn't mean anything at all i don't know.

-gan

well you can see benchmarks above for both 7zip and Rar v4... and with extraction, it is always slower. It has to be slower, it is due to how 64bits work... In the same time, compression is faster.

So ultimate tool would always use 32bit extraction and 64bit compression...

thing is, everyone will still complain, thinking that simply it has to be faster since it is 64bit! :-)

[guido]

Did you do the 32 bit tests on a x64 machine/windows?

[spwolf]

Originally Posted by guido

Did you do the 32 bit tests on a x64 machine/windows?

same computer was used, that was the point :-)

[guido]

So the 32bit is executed using WOW? So x64 generates a lot of overhead.

[spwolf]

Originally Posted by guido

So the 32bit is executed using WOW? So x64 generates a lot of overhead.

yep... for extraction it is useless... for compression, double the number of registers help it.

[guido]

I vote -1 for 64 bit. Give prio to something else.

I never compress P*rn. I just extract it

[TBGBe]

Originally Posted by guido

I never compress P*rn. I just extract it

I just Preview it

[guido]

Originally Posted by TBGBe

I just Preview it

inside or outside archive

[gan]

Originally Posted by spwolf

So ultimate tool would always use 32bit extraction and 64bit compression...

thing is, everyone will still complain, thinking that simply it has to be faster since it is 64bit! :-)

Either way compression is a lot more time consuming so the most important part to improve. If i could pick 64 bit PA to get a bit faster compression, but also a bit slower extraction i would have picked that option.

I believe 64 bit Windows on the desktop really had a breakthrough with Vista and 7. A lot of people want 64 bit for the OS and then usually prefer native 64 bits applications as well even if they don't notice a difference compared to 32 bit. I guess it's not a matter of "if" a software company should have a 64 bit version, but but "when" it should happen since 64 bit obviously is the future. Some competitors are already there and i think PA should consider to do the same thing sooner instead of later.

-gan

[spwolf]

Originally Posted by gan

Either way compression is a lot more time consuming so the most important part to improve. If i could pick 64 bit PA to get a bit faster compression, but also a bit slower extraction i would have picked that option.

I believe 64 bit Windows on the desktop really had a breakthrough with Vista and 7. A lot of people want 64 bit for the OS and then usually prefer native 64 bits applications as well even if they don't notice a difference compared to 32 bit. I guess it's not a matter of "if" a software company should have a 64 bit version, but but "when" it should happen since 64 bit obviously is the future. Some competitors are already there and i think PA should consider to do the same thing sooner instead of later.

-gan

well this is why the thread was started - to see if people want 64bit extraction of RARs even if it is slower than 32bit extraction of RARs and has no benefit except for being slower :-)

[gan]

Originally Posted by spwolf

well this is why the thread was started - to see if people want 64bit extraction of RARs even if it is slower than 32bit extraction of RARs and has no benefit except for being slower :-)

Sure i understand and just giving my vote But faster compression is a benefit i would say and i assume a 64 bit PA would support the creation of zip, zipx and 7z like the 32 bit edition. The ability to create RARs using PA would be great as well, but that's another discussion already to be found in other threads.

-gan

[spwolf]

Originally Posted by gan

Sure i understand and just giving my vote But faster compression is a benefit i would say and i assume a 64 bit PA would support the creation of zip, zipx and 7z like the 32 bit edition. The ability to create RARs using PA would be great as well, but that's another discussion already to be found in other threads.

-gan

it is possible to use 64bit compression and 32bit extraction for instance (in the same app)

whole idea that started the thread was that we thought we might add 64bit extraction of rars to PA 32bit, in the case it was faster... so we tested... and tested... and realizied it is not :-).

PA can not be fully 64bit until there is 64bit version of Delphi (maybe this year), but even then, apparently, it would make sense to actually test if some engine would do better with 32bit version.

Mixing 32/64bit is more work but if it makes things faster, why not.

[Mameluke]

Extraction to me is not as important as compression (Given a reasonable time frame for both). With ISP's limiting traffic and the rise of cheap online storage, I want to be able to effectively backup things. A smaller file size to me is more important than a little extra time waiting to extract it.



Mame

[TBGBe]

Originally Posted by Mameluke

...A smaller file size to me is more important than a little extra time waiting to extract it.

If you are talking seconds (or less) then I would agree.
But if you start needing an extra 10 - 20 mins (for v. large archives) then I would disagree. My point is "real-time" not a percentage change.

[zerone]

I don't think that some (milli)seconds are the problem. Sorry, but i don't sit while compression/extraction with a stopwatch at the pc. The reason why i would use a 64bit Version is the native support. I believe, when more software works native, software could be more optimized on it. At the moment there is no reason for developer to (or microsoft) optimize the code execution, as long the 32bit software works...
Ask yourself... would you run on xp a 16bit application if you have a 32bit that do the same?

[alextorex]

The reason is that I don't want my Windows to be over-bloated with WOW64 emulation to support old 32 bit code. I prefer all code to migrate to 64 bit and MS to drop emulation.

Second reason is that when a developer says he can't support 64 bit it instantly crosses my mind that he uses outdated IDE and compilers. And this makes me think that probably the whole code (GUI and everything) is not optimized for current multi-core processors and 64bit OS.

In Delphi you cannot use for example Intel Parallel Studio to optimize the code for multi core etc. Also Delphi use that over-bloated forms which makes huge executables.

[spwolf]

all of our code is in C libraries, not delphi... which is why it is faster than WinRar and WinZip, and soon 7zip.

IPP is too buggy to be seriously used, but it is good try... we optimize our multicore code manually.

Intel has decent idea with primitives and ipp, it is just buggy.

[Sir Richard]

On an existing business side of things, its far too expensive and complicated to move everyone onto 64Bit system and OS then upgrade every Application. I work for a group that has over 7300 Employees UK they use Windows XP Pro 32bit and office 2003 SP3.

They have a 3 year I.S Plan to move everyone onto Windows 7 32Bit, why not 64bit?? because of known software incompatibilities and Code has to be rewritten from scratch so developers charge more.

Keeping PowerArchiver as a 32bit application although its proven better at extracting than 64bit does not stop skeptics from questioning and in many ways slander the application on forum's, reviews, websites etc all just because it isnt using the more "Modern" 64bit. This in return can cause a domino decline on New Business and increase lapsed sales.

On my side of things, and companies i work closely with both inside my group and outside via charities or contracted assignments they all Compress to Send out large documents, images etc etc and also to backup large folder's on networked drives to then upload to a remote servers via FTP.... All methods the New PowerArchiver now features..

SO.. Having a program that is 64bit with a 64bit OS that can handle large archives faster is a welcome addition to what already is a great application.

My advise to ConeXware....

Finish 2011, work on a new website and then come 2013 Anounce "PowerArchiver X64 Toolbox Edition".

I would not do a 64bit Standard Edition, Why? Because I deem 64bit used more by Profesionals that deal with file archiving every day and that would fall under the new Toolbox edition.

[gan]

Originally Posted by Sir Richard

On an existing business side of things, its far too expensive and complicated to move everyone onto 64Bit system and OS then upgrade every Application. I work for a group that has over 7300 Employees UK they use Windows XP Pro 32bit and office 2003 SP3.

For large companies upgrade of OS is expensive without any doubt, but i work for a Microsoft gold partner and we have a lot of consultants which is basically what this company is all about. Our experience is that more and more customers are moving to 64 bit OS and one common reason is the 4gb RAM limit with a 32 bit OS. Our company use 64 bit only for the OS. So i would have to disagree when you make it sound like companies won't move to 64 because it's too expensive. Yes it's expensive, but that doesn't mean it's too expensive to make this upgrade. I believe it's just a matter of time before most companies switch to 64 bit and we see that the demand for 64 bit increased a lot since Vista and Windows 7. It will take time before everyone switch, but it's just a matter of time and it's not that far into the future i think.

I know that a lot of companies are still stuck with 32 bit XP, but that doesn't mean we should ignore 64 bit Windows since there is a significant marketshare already for Windows 7 x64 and it's growing. We also know that XP will die in time and 64 bit is the future. New server software from Microsoft is almost only 64 bit for the new versions they release and they started to release 64 bit software for the desktop as well. Many other software companies also started to release 64 bit software so it's easy to see the trend.

I don't know if most of ConeXware's customer are companies or not, but i'm under the impression that the advanced home PC users using Windows 7 it's very common with 64 bit Windows.

Originally Posted by Sir Richard

They have a 3 year I.S Plan to move everyone onto Windows 7 32Bit, why not 64bit?? because of known software incompatibilities and Code has to be rewritten from scratch so developers charge more.

I also have to disagree with this one. Actually the compatiblilty are very good. For almost all of the common software used in companies it will work fine with 64 bit OS. Our company even have consultants working with telecom and old pabx's and they use a lot of old software that many people probably didn't think still is being used. That work's fine as well. There is also an option to use XP mode with Windows 7 as a last resort. With enough RAM and a decent computer that works great. If you also have a SSD which is getting common these days you won't notice any delay at all using XP mode. Extremly seldom or maybe even never is there a need to rewrite code from scratch for a 32 bit app to work on 64 bit OS. Most often it will just work even if designed for 32 bit and sometimes minor changes. For software that include drivers then 64 bit drivers are needed of course, but the GUI can still be 32 bit.

I don't know your company and what software you use, but unless there are some very good reasons to choose 32 bit i would say that's a big mistake. My bet is that such a decision will force your company to make the next upgrade sooner than 3 years because the demand for more RAM and other reasons as well. Like i said i don't know your company, but would be very interested to hear the reasons why you would choose 32 bit when doing such a major upgrade. It would be very interesting to hear what apps that a large part of your company use that won't work with 64 bit. Also a upgrade from XP to Windows 7 isn't supported so a new installation have to be done anyway. That would be a good time to make the switch from 32 to 64 bit.

Originally Posted by Sir Richard

Keeping PowerArchiver as a 32bit application although its proven better at extracting than 64bit does not stop skeptics from questioning and in many ways slander the application on forum's, reviews, websites etc all just because it isnt using the more "Modern" 64bit. This in return can cause a domino decline on New Business and increase lapsed sales.

As i said before in this forum, there are benchmark tests using Winrar x86 and Winrar x64 show the opposite. In that case the x64 edition are faster. I don't know all the details and i'm not saying that x64 is the best without exceptions in all cases, but i'm just saying that the benchmark from ConeXware differ from other benchmarks. There is no right or wrong here so when ConeXware say that x64 extraction is slower, that's the fact and everyone saying different are wrong i find that a bit hard to accept. Why Winrar x64 are faster compared to the x86 edition i don't know. Maybe they managed to do something smart that isn't possible with 32 bit or maybe they just don't bother to enhance the x86 version anymore because they believe x64 is the future.....i don't know. I'm just saying that i have never seen a benchmark that says running winrar x86 on a x64 OS will perform faster compared to using winrar x64.

Originally Posted by Sir Richard

My advise to ConeXware....

Finish 2011, work on a new website and then come 2013 Anounce "PowerArchiver X64 Toolbox Edition".

I would not do a 64bit Standard Edition, Why? Because I deem 64bit used more by Profesionals that deal with file archiving every day and that would fall under the new Toolbox edition.

I agree that to finish 2011 x86 should be pri 1, but i also think they should be able to release a x64 version in 2012 at the latest. Work on a new website shouldn't have a higher priority than creating the x64 edition. Hopefully ConeXware can do this in parallel since i'm pretty sure that the people working on the website are not the same as the PA developers. It's still a cost though.

I also have to say that i'm not saying x64 is the best no matter what, but it would be nice if ConeXware where among the first with 64 bit and not the last. Also it seems like a lot of people demand 64 bit for several reasons.....and it's all about selling the products the customers want, right? Customer demand. Even if ConeXware think there is not point it's more important to ask what do the customers want.

I also have to add that PA is probably the fastest or among the fastest when talking about compression/extraction so this part is really good. When talking about the GUI i find PA to be surprisingly slow though. I can launch most other applications like word, excel, Visio, Acrobat and so on faster than PA. So this part is very disappointing. The only app that launch slower on my computer i believe is Photoshop. Even Visual Studio 2010 launch just as fast as PA.

-gan