Fragmentation prevention with IntelliWrite -- say hello to reality

Monday, January 4, 2010 by Joe Abusamra

                                                

You may have read recently about our favorite competitor's (Diskeeper's) latest release and its introduction of a file system filter for defragmentation called IntelliWrite.

As a major developer of enterprise and consumer software for defragmenting computers, many inquiring minds in the industry have begun to ask us what our opinion of IntelliWrite is. It's the kind of inquiry we expect, and eagerly answer. Erwin Solis, PerfectDisk product manager, went about directing our lab tests on the issue. Rather than give you just opinion, we attempt to provide solid numbers and easily reproducible proof.

So what is IntelliWrite and why does it create free space fragmentation?

The good folks at Diskeeper questioned the importance of total free space consolidation -- getting the biggest possible piece of free space -- in a recent blog post. The argument Diskeeper appears to be making is that free space consolidation can be ignored in favor of using a file system filter to eliminate file fragmentation on the fly. There are three key issues with this theory. First, if you intentionally (or unintentionally) cause free space fragmentation to build up, you'll quickly leave a volume in state where files will be forced to fragment. Secondly, in order to prevent such an issue from happening, you'll have to consolidate free space anyway and therefore consume even more resources. Lastly, by using a file system filter, if not done properly, additional overhead is introduced to accomplish something that NTFS already does on its own.

At the heart of this issue is the apparent insistence that the NTFS file system is incapable of preventing file system fragmentation on its own. Actually, NTFS does an excellent job all on its own, and does an even better job when free space is effectively consolidated.

So - how about a reproducable demonstration?

For the following test, feel free to use any sized volume, large or small. The key is to have plenty of consolidated free space. In order to be completely fair, we've used a freshly formatted 100GB volume so that neither NTFS nor Diskeeper's IntelliWrite will have an excuse for their behavior.

This is what we did:

1) Turned IntelliWrite off
2) Completed a Full Install of Office 2007 on a freshly formatted 100GB volume.
3) Analyzed the volume with Diskeeper and noted the statistics provided.

As you can imagine, we then ran the same test over again, this time with IntelliWrite turned on. Here are the results:

Using a 100GB NTFS volume, we can see a dramatic difference in how free space is consolidated:

 

 

NTFS

IntelliWrite

Free Space Fragmentation

 

 

Percent low performing free space

0%

0%

Total free space extents

2

3,995

Largest free space extent

97 GB

94 GB

Average free space extent size

49GB

25 MB

 

 

 

 

 

 

Low-Performing files percentage

 

 

% of entire volume

0%

0%

% of used space

0%

0%

 

 

 

Directory fragmentation

 

 

Total directories

182

182

Fragmented directories

28

21

Excess directory fragments

103

46

 

 

 

File fragmentation

 

 

Total files

4,486

4,494

Average file size

267 KB

267 KB

Total fragmented files

0

0

Total excess fragments

0

0

Average fragments per file

1.00

1.00

Files with performance loss

0

0

 

The data shows that, for the area of the disk where data was written, the average free space gap dropped from 49GB to just 25MB. Instead of free space being broken into just 2 fragments, Intelliwrite splits it into nearly 4,000 pieces. Notice how in this example that NTFS managed to prevent all file fragmentation on its own, without the additional resource impact associated with a 3rd-party file system filter. In this case, IntelliWrite provides ZERO benefit and instead proves to be the exact opposite of what you would consider a "green" technology. Try adding the I/O Other Bytes column to the Processes tab under Task Manager to see just how much I/O activity Diskeeper generates over time; we believe you'll find Diskeeper's resource usage to be a real eye opener. These are the results that IntelliWrite provides during the installation of a common office application; the impact for servers is even greater. Recently, a Diskeeper customer contacted us about a problem that IntelliWrite caused for them on one of their file servers; we saved a snapshot of the summary statistics provided by PerfectDisk Server:

 


As you can see, IntelliWrite was effectively addressing the file fragmentation on the volume - but at the cost of creating other performance issues. By eliminating the resource overhead associated with IntelliWrite and applying SMARTPlacement with PerfectDisk Server, performance was restored just 24 minutes later. Here's an analysis of the results:



As many users know, the PerfectDisk solution to this problem is to combine the intelligent placement of files, with the best possible consolidation of free space - so that NTFS can do the best job possible without additional resource overhead. After all, it's not just whether or not files are fragmented, but how and where they are written that also matters.

Our impression is clear. We suggest that users simply ignore the currently available file system filter - just turn it off. Even if Diskeeper is your preferred disk defrag utility, don't create free space fragmentation to the detriment of your system performance. 

Finally, regarding the issue of IntelliWrite being a "green" solution...because it creates a situation where resources are consumed in order to prevent file fragmentation - but which results in the need to consume yet more resources to clean up the free space fragmentation issues it leaves behind...there might be another color I'd use to describe it.

The bigger point here -- make sure you do your own testing and verification. As with just about all software, results may vary. But do your own checking -- just because someone says something is so, doesn't necessarily mean that something is so, right? I think we learned that lesson time after time in 2009.

                                                      
 
Note: Diskeeper is a registered trademark and IntelliWrite is a trademark of Diskeeper Corporation.

From the Using Windows Home Server blog -- Friday night fun

Monday, December 14, 2009 by Joe Abusamra

                                                        

Defragmenting computer or your Windows Home Server on a Friday night? Seems like a great idea!

Tim Daleo did just that on Friday night. Tim is a Project Resource Analyst and Oracle Applications Trainer for the Jet Propulsion Laboratory in Pasadena, California. Tim currently runs a Dell Power Edge server running Windows Home Server and a HP MSS at an off-site location. On Friday nights, Tim posts in the Using Windows Home Server blog - which classifies itself as "your number one stop for all things Microsoft Windows Home Server, Windows Media Center and anything in the Connected Home."

                               Using Windows Home Server

Tim provides a good overview of PerfectDisk 10 Windows Home Server, although he is still looking for the definitive answer as to whether disk defrag software is needed, especially since there is not a Microsoft defrag for Windows Home Server.

Here is how Tim concludes his review:

Overall I like the functionality and interface of PerfectDisk 10. If you are looking for a Disk Defragmenter for your WHS then this is a great choice. In addition, like I said earlier, their manual has 325 pages so there is a lot more to it then what I covered here.

Only time will tell whether PerfectDisk10, and disk defragmentation in general, is worth the effort and money. For now I will give PerfectDisk 10 the benefit of the doubt.

That said…does WHS really need a Disk Defragmenter? Microsoft would have included it if it needed it, right?"

                                                             

Well, PerfectDisk 10 Windows Home Server lets you not only defrag the server but also defrag PC or multiple PCs in your environment. One of the reasons HP partnered with PerfectDisk for its MediaSmart Servers is that its testing showed PerfectDisk would eliminate hard drive fragmentation, consolidate free space to speed backups, and allow HP users to stream media faster.

HP recognizes the need to defrag Windows Home Servers, and it has partnered with the leading disk defrag utility as part of its validation. This review provides some more insight. And thousands of users around the world provide more evidence.

Your entire connected home defragmented is a better-connected home. You can read the entire review here.

PC World's Missed Myth

Friday, December 4, 2009 by Joe Abusamra
                                                    

PC World is just one of numerous magazine's I read and monitor for various industry news, reviews and opinions. Heck, earlier this week I wrote a post about an article in PC World. But since I work for a software vendor that helps people and businesses achieve performance and storage management improvements (primarily through defragmenting computer), the January 2010 edition's PC Performance Myths article caught my eye immediately.

                                            

So as sure as not to misquote, here is PC World's stated "myth" (I have the print edition, the online version was not online as of the writing of this post):

Defragging your hard drive: Back when drives were small and OSs were simpler, doing this was necessary. But Windows XP, Vista and 7 all have automated disk optimization, and it's rare for a drive to become so fragmented that it hampers performance. While defragmenting isn't harmful, it's usually a waste of time.

Ok, by mentioning that there are built-in XP, Vista and Windows 7 defrag options, presumably at least part of PC World's real message was that you don't need to buy one or use anything other than what the OS comes with. Although this article didn't say that, that is what this portion of the article meant (you're welcome, PC World editors).                                                 See full size image

Now for the rest -- "back when drives were small and OSs were simpler, doing this was necessary," but not now and not since the advent of Windows XP. Really. So a good old- school 80GB drive running Windows 95 might have needed it, for all that Word and Office stuff people were doing. But now, take all that, and add terabyte drives, volumes of pictures and videos, editing and deleting, and even more work and more play being done at home and at work, now fragmentation is not a problem? Humorous at best - detrimental at worst. So more people and more businesses than ever before are buying defraggers, and there are more defrag offerings than ever before, all because Microsoft, Diskeeper, PerfectDisk and a bunch of freeware apps are all crazy? Defrag PC? PC World thinks you're an idiot...

"It's rare for a drive to become so fragmented that it hampers performance." Try doing something real (for fun or work) on your computer today, with larger files and larger drives, without a defragmenter, and see what happens. Anyone saying that performance won't be impacted does not have an even marginal understanding of the NTFS file system. Hello, ivory tower....

                                       
"...it's usually a waste of time." I guess I better get on the phone and call the CIOs and IT directors of Global 1000 companies and others and tell them all that research and testing they did to determine specific performance and resource usage improvements was invalid, and despite what came out of the labs, it ain't true. Sorry World of Warcraft users - you only think you're playing your game faster. Sorry, videographers, the time you thought you were saving was really a dream. Productivity increases and faster access to databases? It's just your imagination...

Are you using a disk defragmenter program, built-in or otherwise? PC World thinks you're crazy. I'm just one of many that knows you're not, and you're smarter than those guys.

This is the type of writing that gives the mainstream media a bad name...