[ale] If you own (almost) ANY HDD, repartition with 4k boundaries

[Ron Frazier (ALE)]

Hi all,

I recently posted a message about misaligned partitions, to which a 
couple of people replied with info about 4k sectors for HDD's.  I 
decided to post this message with a different subject because this can 
have a much broader impact than many people probably know.  Some of you 
may already know this.  Others may not.  I haven't purchased any new 
drives lately, so I'm just learning about it.

To reiterate what happened, I replaced a 500 GB hybrid SSD HDD with a 
new 750 GB one of newer vintage.  I cloned the hard drive using a 3 year 
old version of Acronis True Image, which had always worked before.  When 
I brought up the new drive in Ubuntu 11.10 Disk Utility, for each 
partition I clicked on, it said the partition boundary was misaligned, 
and performance would suffer.  Based on the responses to the other 
threads, I started doing research.

I'm not the best at explaining this, so bear with me.  See also the 
links and quotes below.

Essentially, all HDD's made after January 2011 are now employing 4096 
byte or 4 KB sectors instead of the traditional 512 byte sectors.  The 
firmware in the drive reports sectors as though they were 512 so legacy 
software will work.  Also, traditionally, the first partition of an MBR 
formatted HDD started on sector 63, after the MBR.  So, if the data from 
a 512 byte sector drive is cloned onto a 4 KB sector drive, the 
partition boundaries will be offset and the data will not align properly 
with the new physical media.  You might think, so what, the firmware is 
translating, no problem.  Wrong!  As it turns out, misaligned partitions 
can lead to write performance which is 10 times slower.

There are the equivalent of 8 - 512 byte sectors in 1 - 4 KB sector.

Let's say you want to read 512 bytes, what your software thinks is 1 
sector.  It sends the request to the HDD.  The firmware finds the actual 
4 KB sector with what you want in it.  It then extracts the 512 bytes 
you want and sends that to you.  That works pretty well.  Writing is a 
whole other matter.

Here's a quote from the Seagate article below:

"A write process can be more complicated, particularly when data the 
host computer attempts to write is a subset of a physical 4K sector. In 
these cases, the hard drive must first read the entire 4K sector 
containing the targeted location of the host write request, merge the 
existing data with the new data and then rewrite the entire 4K sector. 
(On the next revolution.)"

So, if you're writing 512 bytes, the firmware finds and reads the 4 KB 
sector that contains the area you want to write, replaces the 512 bytes 
you're writing while retaining the rest, and writes the sector back to 
the disk.  It has to wait for the platter to spin around again before it 
can rewrite the sector.  At 7200 RPM, this takes an additional 8.3 mS.  
Lets say you're writing 1024 bytes and that bridges two 4 KB sectors.  
The system has to read both sectors, merge 512 bytes into one, then 512 
bytes into the other, then write them back on the next revolution.  You 
get the idea.

The IBM article below indicates that this can lead to a 10 TIMES speed 
degradation if doing small writes.

You fix at least part of the problem by realigning both the start and 
end locations of each partition on the disk to 4 KB boundaries.  The 
standard industry practice seems to be to align them on on 1 MiB 
boundaries, which is a multiple of 4 KB.

So, you want your first partition to start at 1 MiB, which is sector 
number 2048 rather than 63.  I would leave 1 MiB between partitions, and 
make the next partition start and end on 1 MiB boundaries.

I should note that you don't want to do this with Windows XP from what 
I've read.  It expects the 1st partition to be at 63, period.

In my case, I cloned a disk with 512 byte sectors to a disk with 4 K 
sectors, but I kept the partition arrangement exactly the same, which 
had always worked before.  So, everything on the clone ended up being 
misaligned.

Here's why I worded the subject the way I did.

If you have an SSD, you MUST align the partitions properly for optimum 
performance.  They have 4K sectors.

If you have a HDD with 4K sectors, you MUST align it properly for 
optimum performance.

If you have a hybrid SSD HDD with 4K sectors, you MUST align it properly 
for optimum performance.

If you have a hybrid SSD HDD with 512 byte sectors on the platters, you 
MOST LIKELY need to align it properly for optimum performance.  Very 
hard to find any data on this.

AND, if you have a lowly old HDD with 512 byte sectors, and you EVER 
MIGHT clone it to any of the others, you probably need to align it 
properly for optimum performance of the drive that you clone it to.

As was pointed out in the other thread, some cloning software 
understands the 4K sectors.  The older stuff, including my 3 year old 
Acronis True Image, does not.  However, I'm pretty sure that if I align 
the partitions of an old HDD to 1 MiB boundaries, then do an identical 
clone, that should be preserved.  If I let it expand the partitions on 
the new HDD, I don't know.  Even with new software that understands, you 
might have to turn on an option such as align to 4K, align to 1 MiB, 
align to target, etc.

I looked at the manual for Terabyte Unlimited's cloning software and I 
may replace Acronis with that.  The product looks pretty impressive, and 
the manual mentions 4K sectors.  I'm not afraid to pay for something 
this complex and critical if I have to.  I've heard good things about 
Paragon software but don't know anything about it.

You can check to see if you have an alignment conflict with Disk Utility 
(in Ubuntu).  I don't know what program that actually runs.  It will 
only show a message if the drive is partitioned the old way and it has 
4K sectors, apparently.  If the drive is partitioned the old way and you 
have 512 byte sectors, it won't complain.  The version of Disk Utility 
in Ubuntu 11.10 does not partition the drive properly by default.

Because of the potential problems of having ANY drives around with 512 
byte sector alignment (also called cylinder alignment, I think), I've 
decided to repartition all my boot drives and am considering doing the 
archival drives.  I'm putting everything on 1 MiB boundaries whether the 
actual drive has 512 byte or 4K sectors.

The only way I could get into trouble with that is if I partition a 
drive with old software or if I clone a drive with old software and 
resize the partitions.

You can use the latest version of Gparted from gparted.org to 
repartition the drive non destructively with data on it.  You should 
backup the data first.  Don't use the one in Ubuntu, for example.  It's 
old and buggy.  Download the ISO for the Gparted live CD and use that so 
you have the newest version.

This lifehacker article tells how to do it.

http://lifehacker.com/5837769/make-sure-your-partitions-are-correctly-aligned-for-optimal-solid-state-drive-performance

It's a little dated.  The UI for Gparted has changed slightly.  There is 
a drop down box for alignment.  Set this to MiB.  There is no longer an 
align to cylinders check box.

So, here's what I did to all my boot drives.

For the first partition, I simultaneously reduce the size and move the 
boundaries to allow 2 MiB before and after.

Then, I move the partition to the left by 1 MiB as the article indicates.

I then move each other partition that has at least a 2 MiB boundary to 
the left to within 1 MiB.  If any partition doesn't have at least 2 MiB 
to it's left, I first move it to the right up to 2 MiB, then back to 1 
MiB.  This is due to an anomaly in the way Gparted calculates space, as 
mentioned in the article.

When, I'm done, there is 1 MiB before the first partition, although you 
cannot see it on the Gparted drive map.  I have 1 MiB unused spaces 
between partitions.  And, every partition begins and ends on a 1 MiB 
boundary.

This will optimize performance if the drive has 4K sectors, or if it is 
ever cloned to a drive that does.

Doing this, particularly moving the boot partition away from sector 63 
will probably break your boot loader.  The article talks about how you 
can fix a Windows installation with a Windows install disk.  If you 
don't have one handy, you may not want to try this.

See the Gparted faq (below) for info on fixing a Linux boot loader.

Here's how to check if you've finished the job properly using Gparted.

Click on the first partition, right click, and select information.

The first sector should be at 2048.
Look at the total sectors.  In one case, for me, it's 307,183,616.
Get out an engineering calculator that can handle big numbers.  Enter 
this in and divide by 2048.
In my case, the answer is 149,992.  If this is a whole number, with no 
decimal, you're good.

The unused space between each partition should have 2048 total sectors, 
or a multiple of that.

For each other partition, check that the <first sector> / 2048 is a 
whole number
and that the <total sectors> / 2048 is a whole number.

If all the numbers check out, you have optimized your partition 
structure for drives with 4K sectors.

Below are a number of links and some quotes that I found helpful.

I hope this info is helpful to you.

Sincerely,

Ron

---------------------------------------

http://en.wikipedia.org/wiki/Advanced_Format
http://www.zdnet.com/blog/storage/are-you-ready-for-4k-sector-drives/731
http://wdc.custhelp.com/app/answers/detail/a_id/5655
https://ata.wiki.kernel.org/index.php/ATA_4_KiB_sector_issues (geeky)
http://gparted.org/faq.php
http://gparted.org/display-doc.php?name=help-manual
http://gparted.org/display-doc.php?name=help-manual#gparted-fix-grub-boot-problem 


---------------------------------

http://www.seagate.com/tech-insights/advanced-format-4k-sector-hard-drives-master-ti/ 
(good)

"As all hard drive manufacturers have agreed to transition to the 
Advanced Format sector design by January 2011, the industry must adapt 
to and embrace this change to minimize potential negative side effects. "

"A write process can be more complicated, particularly when data the 
host computer attempts to write is a subset of a physical 4K sector. In 
these cases, the hard drive must first read the entire 4K sector 
containing the targeted location of the host write request, merge the 
existing data with the new data and then rewrite the entire 4K sector. 
(On the next revolution.) "

"Fdisk: GNU Fdisk is a command line utility that partitions hard drives. 
Versions 1.2.3 and above support Advanced Format drives. "

"Parted: GNU Parted is a graphical utility for partitioning hard drives. 
Versions 2.1 and above support Advanced Format drives. "

------------------------------

http://www.terabyteunlimited.com/index.htm - cloning software looks cool
http://goinglinux.com/ - Just did a podcast, # 195, on SSD's.

from prior thread - http://jrfom.com/2010/05/03/4kb-sectors/
from prior thread - http://lwn.net/Articles/377895/

-----------------------

from prior thread - 
http://www.ibm.com/developerworks/linux/library/l-4kb-sector-disks/index.html?ca=dgr-lnxw074KB-Disksdth-LX 
(good) (dated)

"Most modern file systems use data structures that are 4096 bytes or 
larger in size. "

"Writes of misaligned data structures, on the other hand, require the 
disk's firmware to first read two sectors, modify portions of both 
sectors, and then write two sectors.
The biggest write performance effects occurred with small file creation 
(extracting the kernel tarball). Effects on tarball extraction ranged 
from 1.04 (for ext4fs) to 25.53 (for ReiserFS), with a mean of 10.9. The 
second-best performer in this test was XFS, with a figure of 1.82. 
Because these figures are ratios of unaligned to aligned performance, a 
value of 10.9 means that a tarball extraction that takes 10 seconds on a 
properly aligned partition takes 109 seconds on an improperly aligned 
partition---a huge difference! The XFS figure of 1.82 means that this 
10-second operation takes 18.2 seconds on an improperly aligned partition. "

-- 

(To whom it may concern.  My email address has changed.  Replying to former
messages prior to 03/31/12 with my personal address will go to the wrong
address.  Please send all personal correspondence to the new address.)

(PS - If you email me and don't get a quick response, you might want to
call on the phone.  I get about 300 emails per day from alternate energy
mailing lists and such.  I don't always see new email messages very quickly.)

Ron Frazier
770-205-9422 (O)   Leave a message.
linuxdude AT techstarship.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ale.org/pipermail/ale/attachments/20130103/a98d8cd7/attachment-0001.html>