[mythtv-users] Hitachi 3T drives - reliable?
mythtv at rodsbooks.com
Thu Jan 20 16:09:32 UTC 2011
On 01/20/2011 07:49 AM, Simon Hobson wrote:
> Jean-Yves Avenard wrote:
>> Sorry, but this doesn't make any sense. Why would a 64 bits OS got
>> anything to do with the size of a hard drive.
> Directly, not a lot. But ...
> For whatever reason, 2TB seems to be a common limit (I know it is in
> older versions of Windows). 2TB is 2^31 bytes, so the maximum value for
> a signed 32 bit integer.
I've missed a few posts in this thread; but....
The limit works out like this:
2^32 sectors * 512 bytes/sector = 2^41 bytes = 2 TiB
The elaborate: Various addressing systems for hard disks, including the
Master Boot Record (MBR) partitioning system and internal data
structures on various OSes, rely on 32-bit (unsigned) sector pointers.
Given the near-universal (until recently) practice of using 512-byte
sectors, this becomes 2 TiB (note that's tebibytes, not terabytes, when
being precise with the units).
As others have pointed out, when using disks with larger sectors, the
size limit goes up; however, to the best of my knowledge, no internal
hard disks today use anything but 512-byte logical sectors. ("Advanced
Format" drives use 4096-byte physical sectors but present 512-byte
logical sectors to the OS, so they don't help on this score. There are
also external drives that do the opposite -- they use 512-byte physical
sectors but present larger logical sectors to the OS.) Part of the
problem is that there are a lot of assumptions about sector size written
into BIOSes, OSes, disk partitioners, etc. (Try running GNU Parted on a
disk with something other than 512-byte sectors sometime!) These
assumptions mean that if a disk manufacturer were to release a disk with
4096-byte sectors that presented itself as such, they'd probably get
tons of returns because the disk wouldn't work with a lot of computers.
> I suspect it's more a case of "it's fixed in OS versions that happen to
> be 64bit", while 32bit OSs have been 'left behind'.
Whatever is used internally by the OS, the MBR limit is a hard one; you
just can't get around it and still use MBR. (The MBR data structures are
explicitly 32-bit, no matter what bit size your CPU uses.) Thus, one
step in overcoming the limit is to ditch MBR in favor of something else,
like the GUID Partition Table (GPT), which uses 64-bit pointers. If an
OS supports GPT, it MUST use 64-bit pointers, at least when addressing
the GPT data structures, no matter what its CPU's native bit depth. An
OS could of course convert this to a 32-bit structure at other points in
its internal processing, thus eliminating one of the benefits of GPT --
namely its ability to handle huge disks. My understanding is that some
drivers for Windows XP and earlier did just this, which means that even
add-on GPT drivers for Windows XP only support disks of up to 2^32 sectors.
I did some tests once in a virtual machine, and 32-bit Linux (I think I
used Ubuntu 9.04) was able to handle a 4 TiB virtual disk (with GPT)
just fine. Windows XP is limited by MBR, since it doesn't support GPT.
Windows Vista and above support GPT, but not for boot disks unless the
computer uses EFI rather than BIOS. I'm not sure offhand if 32-bit Vista
or 7 can handle a disk with more than 2^32 sectors that's partitioned
with GPT (I couldn't get these OSes installed under QEMU, which I used
for my tests).
> As for disks (of the computer storage rather than brake variety), I
> suspect there's nothing fundamental about 32bit OSs to stop them
> supporting >2TB drives, more likely that <something> is a 32bit signed
> integer and that sets a limit. For obvious reasons, if the same software
> is compiled for 64bit, then the 32bit value may become a 64bit value and
> the limitation disappears.
That's basically correct, except that you've got a whole range of data
structures -- in disk partitioning software, in the kernel's partition
table handling, in low-level disk drivers, in the computer's firmware, etc.
FWIW, I'm familiar with the partition table aspect of it because I'm the
author of GPT fdisk (aka gdisk; http://www.rodsbooks.com/gdisk/). I'm
not familiar with the details of the Linux kernel internals, much less
those of Windows. As noted earlier, though, I have run tests of several
OSes using virtual machines.
More information about the mythtv-users