[olug] MAX_LUNS: Caveat about selling Linux for SAN to "the management"

[Christopher Cashell]

At Thu, 18 Jan 07, Unidentified Flying Banana swanpoint at cox.net, said:
> With Symmetrix there is a lot of disks too be partitioned into a whole
> lot of LUN. We're using 4 GB and 8 GB sizes for our LUN's; so the
> numbers increment quickly.

Is there a technical requirement for 4GB and 8GB LUNs?  With modern
storage systems, that seems *extremely* small.  With modern disk sizes,
I would go so far as to say it's probably going to have a negative
impact on your storage system because you're essentially fragmenting the
storage space and reducing the SAN's ability to optimize IO.

This is actually similar to the issue that I've run into with Oracle RAC
systems using raw devices.  Each raw device will point to 1 disk or
partition, and Oracle has a maximum usable raw device size of 16GB.
That leaves you with an absolute maximum of roughly 4TB, and unless you
created every raw device at the maximum size from the beginning, it's
easy to get into trouble (if, for example, it was originally created
with 100 2GB raw devices).

Even here, though, we hit the raw device size, but we're nowhere near
the LUN limit, because we've got 15 partitions on each LUN, and each of
the LUNs is over 100GB.

Going back to my earlier question, is there any reason you are required
to use such small LUNs?  Can you create larger LUNs and split them out
logically by partitions on the OS size for your use?

> There can be 256+ LUN's behind a FA port (on the Symmetrix end). Not
> all LUN's are mapped to the same server; but, if the LUN index is
> greater than 255, no good.

Hrm.  I just glanced, and on none of our SANs have we actually created
LUNs numbered higher than 255 (I don't think we have any where we've
created more than about 150 LUNs, as even on the older systems, LUNs are
always at least 8GB, and most of the newer ones are in the hundreds).

I did notice that Linux numbers the LUNs it sees starting from 0
regardless of what is assigned by the storage system (at least for all
of our setups).

> There's a kernel parameter for specifying the max number of lun, but
> that parameter has no effect (recommended solution for RedHat support
> is to specify the max lun parameter at boot, too no avail).

Red Hat specifies somewhere in their knowledge base that the maximum
available is 255 for RHEL4.

-- 
| Christopher
+------------------------------------------------+
| Here I stand.  I can do no other.              |
+------------------------------------------------+