disk compression for more space

 

Overview

nearly all of us would like out computers to be faster! well, they have been getting faster. windows 7 and a newer box will speed the slowest component of the computer (the hard disk) up by up to 4.5x.

you can apply disk compression to a windows hard disk and you will get a slower machine, but you will get more space. this is useful for SSD's. my worst case test is random data.

uses 1GiB+ RAM.

speed?

disk compression has traditionally slowed things down. even on my i7-3970x with 64GiB RAM and with an SSD. but it isn't too noticeable unless you have larger files.

I think it's because you are adding an additional software filter driver layer on top of the filesystem drivers.

data integrity

as for data integrity, microsoft seems to have taken care of the data integrity issues back in windows 98 days.

they probably carried this fix through to later versions of windows I think, I don't know. but it seems to work well over the years I have used windows since 98.

where should disk compression be especially handy?

  • got 1GiB RAM or more you can spare
  • my documents
  • libraries
  • \program files
  • \program files (x86)
  • \program data
  • \users
  • my videos (if you use AVI's or other raw video formats)
  • adobe premiere [pro] files - I think - try it and see if you get a speed boost!
  • adobe photoshop psd's or other native image file formats?
  • raw tiff scanned images, if uncompressed
  • your thunderbird profile in %APPDATA%
  • your firefox profile in %APPDATA%
  • SSD's - save more space!
  • programming source code!
  • \program files
  • \program files (x86)
  • \program data
  • \users\ or \documents and settings\ (not everything may compress, windows will complain about your .dat etc registry database files, don't mess with those)
  • BMP files
  • web sites you work with or write (HTML files and other web documents especially!) - your www directories
  • downloads? it might compress, but probably not. try it. installers are already compressed. ISOs may or may not be a percentage compressed.

where it won't work well:

  • \windows\ (avoid!!!! thousands of popup dialog boxes you MUST click - wears you out after a very very short time, if you do this, don't reboot in the middle, just continur and leave machgine running until you are done)
  • \system volume information\ (avoid at all costs!!!! NO TOUCH THIS FOLDER! FILESYSTEM INNARDS)
  • my pictures (already compressed)
  • my videos (if you use compressed video formats such as found on the web, like mp4, mpg, webm, vp8, ogg, xvid, divx)
  • already compressed files in general

the raw numbers, please

using 1GB=1,000,000,000=1e9 file.
hard disk, createfile:random on p4 w 3GiB RAM xp pro, compression off: 19MB/s
hard disk, createfile:random on p4 w 3GiB RAM xp pro, compression on: 10MB/s
hard disk, createfile:repeated string on p4 w 3GiB RAM xp pro, compression off: 44MB/s
hard disk, createfile:repeated string on p4 w 3GiB RAM xp pro, compression on: 34MB/s

hard disk, createfile:random on i7-3970x w 64GiB RAM w7, compression off: 86MB/s
hard disk, createfile:random on i7-3970x w 64GiB RAM w7, compression on: 76MB/s
hard disk, createfile:repeated string on i7-3970x w 64GiB RAM w7, compression off: 335MB/s
hard disk, createfile:repeated string on i7-3970x w 64GiB RAM w7, compression on: 299MB/s

ssd, createfile:random on i7-3970x w 64GiB RAM w7 ult, compression off: 85MB/s
ssd, createfile:random on i7-3970x w 64GiB RAM w7 ult, compression on: 72MB/s
ssd, createfile:repeated string on i7-3970x w 64GiB RAM w7 ult, compression off: 339MB/s
ssd, createfile:repeated string on i7-3970x w 64GiB RAM w7 ult, compression on: 318MB/s

hard disk, createfile:repeated 0's on i7-3970x w 64GiB RAM w7, compression on: 385MB/S sustained
(repeated numbers are highly compressable!)

how do I turn disk compression on?

  1. [windows-logo-flag-key]-E or bring up my computer from the start menu
  2. browse to the folder or folder tree you want to compress (watch out for the \windows folder, this one will pop up 1000 dialog boxes or more if you click Apply!)
  3. right click on the folder and pick properties, advanced
  4. check the checkbox which says "compress contents to save disk space" and click OK.
  5. click Apply.
  6. change the radio button to "Apply changes to this folder, subfolder, and files" and click OK.
  7. can take an hour or more to process. depending on size and content.
  8. click OK.
  9. if you cancel anything for any reason during the middle of a compression job, it appears on win7 you won't be able to recompress unless you first uncompress the whole subtree. I ended up with what looked like only a few directories in the tree compressed. recompressing when dir was in compressed mode would do nothing, system just ignored the operation even though it was an incomplete job. tell microsoft about this by googling the windows 7 feedback page.

for example, a typical compiler which has a large amount of text files consumes 1GB but will compress to 300MB - this also has lots of less-compressable binaries so it's probably not the best example of that. alternative instructions:

  1. start, my computer (or just do [windows-logo-flag-key]-E, c:
  2. If the folders I mentioned are not visible, you will probably need to click something in control panel,Folder Options,Advanced Settings,show hidden and system files of the Explorer to allow you to see system folders. Once they are shown to you, they will always be shown. WARNING: if you are not good with windows's innards or you don't know what you are doing and your installation is precious to you, don't mess with this.
  3. make sure the folders button on the toolbar is depressed if you have one (you probably do and it's probably not depressed by default). If it's not visible,
    1. Alt-V to get the View menu, Toolbars, make sure "lock the toolbars" is unchecked
    2. Alt-V to get the View menu, Toolbars, make sure "standard buttons" is checked
    3. Alt-V to get the View menu, Toolbars, make sure "address bar" is checked
    4. Alt-V to get the View menu, Toolbars, make sure "lock the toolbars" is checked (this is purely optional, it prevents you from making mistakes)
  4. expand c: drive if you so choose (a good idea)
  5. Pick which directories you want to compress (folders button is helpful for exploration). you may choose to compress the entire thing. (not all of it will compress). best-plan directories are C:\Temp, c:\program files\, c:\program files (x86)\, c:\programdata\, or its equivalent and anything like it, c:\users\ or c:\documents and settings\ depending on whether you have vista and up or xp. If you have directories under c:\ which you created, do those too. AVOID C:\WINDOWS\ and c:\system volume information\ FOLDERS!that means trying to compress c:\ as well. sorry.
  6. right click on the directory/folder in question, and pick Properties, Advanced button
  7. (you don't have to wait for the churning numbers) click the Compress checkbox, click OK, and go through all the hoops to make sure that this change applies to all subfolders and files.
  8. if you miss it you will have to undo it, redo it, and wait a long time.
  9. when it complains about files it couldn't compress, just click the "ignore all" button and it won't bother you for every little mishap.
  10. after all files processed (could take hours), click any OK buttons until the dialog boxes are all gone.
  11. do the same thing for any other folders you might find useful to compress.
  12. close windows explorer when you are finished with it (my computer)