Felipe Alfaro Solana

Once I got the OpenSolaris installation kicked I faced another problem. The computer has 3 x 1.5TB and 1 x 2TB disks installed. This is suboptimal when configuring a RAID volume but I thought that I could cheat here. The cheat consists of slicing the 2TB disk in a 1.5TB slice and using the remaining 0.5TB to store the ZFS rpool.

As nice as it sounds this turned out tobe a little bit tricky. More than I expected. The challenge was convincing the OpenSolaris installer to create one 0.5TB slice and another 1.5TB slice. But I could not: OpenSolaris creates a partition and creates a slice that fills it up entirely. So, I either got a 2.0TB partition and a 2.0TB slice or a 0.5TB partition and a 0.5TB slice. In the end, what I did is to create a 0.5TB partition leaving the rest of the disk unused. The OpenSolaris installer created a SOLARIS2 partition and, inside it, a 0.5TB slice for the ZFS rpool. Once the installation was finished, I booted from a Ubuntu 9.04 LiveCD. As usual, Ubuntu did not require any driver disk or updates: it recognized the HP Smart Array controller automatically. Then I used fdisk to destroy and recreate the SOLARIS2 partition but this time I recreated it to cover the whole disk. I had to make sure the partition was makred active or GRUB would not boot. Since OpenSolaris doesn’t like multiple partitions of SOLARIS2 (hexadecimal bf) type, this is the only way I can think of to do what I want.

Next step was to boot from the OpenSolaris x86 LiveCD, apply the driver update and check that fdisk reported a single Solaris partition that fills the entire 2TB disk. Then I summoned format to create a new s3 slice that covered the remaining 1.5TB. This is how the slices look like now:

Current partition table (original):
Total disk cylinders available: 60796 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders         Size            Blocks
  0       root    wm       1 - 15198      465.69GB    (15198/0/0)  976623480
  1 unassigned    wm       0                0         (0/0/0)              0
  2     backup    wu       0 - 60795        1.82TB    (60796/0/0) 3906750960
  3 unassigned    wm   15199 - 60795        1.36TB    (45597/0/0) 2930063220
  4 unassigned    wm       0                0         (0/0/0)              0
  5 unassigned    wm       0                0         (0/0/0)              0
  6 unassigned    wm       0                0         (0/0/0)              0
  7 unassigned    wm       0                0         (0/0/0)              0
  8       boot    wu       0 -     0       31.38MB    (1/0/0)          64260
  9 unassigned    wm       0                0         (0/0/0)              0

Then, I only had to apply a label to the remaining 3 x 1.5TB disks and create the corresponding s3 slices on each of them. All s3 slices are about the same size:

Current partition table (original):
Total disk cylinders available: 60796 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders         Size            Blocks
  0 unassigned    wm       0                0         (0/0/0)              0
  1 unassigned    wm       0                0         (0/0/0)              0
  2     backup    wu       0 - 60795        1.36TB    (60796/0/0) 2930063220
  3 unassigned    wm       0 - 60795        1.36TB    (60796/0/0) 2930063220
  4 unassigned    wm       0                0         (0/0/0)              0
  5 unassigned    wm       0                0         (0/0/0)              0
  6 unassigned    wm       0                0         (0/0/0)              0
  7 unassigned    wm       0                0         (0/0/0)              0
  8       boot    wu       0 -     0       23.53MB    (1/0/0)          48195
  9 unassigned    wm       0                0         (0/0/0)              0

And finally,

# zpool create -f zfs raidz1 c9t0d0s3 c9t1d0s3 c9t2d0s3 c9t3d0s3
# zpool list
NAME    SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
rpool   464G  5.50G   458G     1%  ONLINE  -
zfs    5.44T   152K  5.44T     0%  ONLINE  -

After all my failures trying to get this HP computer to work, I decided that perhaps OpenSolaris was the solution. And it turned out I was right. OpenSolaris x86 LiveCD uses a 64-bit kernel by default. The tricky part was to get the driver loaded before starting the OpenSolaris installation. And to my fortune, I read Installing OpenSolaris 2008.11 on a server with a HP Smart Array Controller. The post was extremely useful.

To keep it short, download the cpqary3 driver from the HP Web site. Or, if you trust in me (you should always download drivers from the manufacturer’s Web site), I also keep a copy of the 2.2.0 driver that can be downloaded from here.

Next,

$ cd /tmp
$ tar zxvf CPQary3-2.2.0-solaris10-i386.tar
$ cd CPQary3-2.2.0-solaris10-i386
$ pfexec touch /ADD_DRV_IGNORE_ROOT_BASEDIR
$ pfexec pkgadd -d ././CPQary3.pkg
The following packages are available:
1 CPQary3 HP Smart Array Controller Driver
(i386) 2.2.0, ...

Select package(s) you wish to process (or 'all' to process
all packages). (default: all) [?,??,q]:

Processing package instance from

HP Smart Array Controller Driver(i386) 2.2.0,...
Copyright 2009 Hewlett-Packard Development Company, L.P.
## Executing checkinstall script.
Using as the package base directory.
## Processing package information.
## Processing system information.
11 package pathnames are already properly installed.
## Verifying package dependencies.
## Verifying disk space requirements.
WARNING:
The /usr filesystem has 0 free blocks. The current installation
requires 158 blocks, which includes a required 150 block
buffer for open deleted files. 158 more blocks are needed.

WARNING:
The /usr filesystem has 0 free file nodes. The current
installation requires 26 file nodes, which includes a required 25
file node buffer for temporary files. 26 more file nodes are needed.

Do you want to continue with the installation of [y,n,?] y
## Checking for conflicts with packages already installed.
## Checking for setuid/setgid programs.

This package contains scripts which will be executed with
super-user permission during the process of installing this
package.

Do you want to continue with the installation of [y,n,?] y

Installing HP Smart Array Controller Driver as

## Installing part 1 of 1.
/kernel/drv/amd64/cpqary3
/kernel/drv/cpqary3
/kernel/drv/cpqary3.conf
/usr/share/man/man7d/cpqary3.7d
ERROR: attribute verification of failed
pathname does not exist
[ verifying class ]
ERROR: attribute verification of failed
pathname does not exist
[ verifying class ]
[ verifying class ]
## Executing postinstall script.

Installation of partially failed.

Even if the installation seems to fail the key consists of checking if the driver itself has been loaded:

$ modinfo
modinfo | grep cpq
 53 fffffffff79a6000  129f8 265   1  cpqary3 (HP Smart Array Driver Ver 2.2.0)

If modinfo shows a line like that it means the driver has been loaded and the OpenSolaris installer should see all the logical volumes.

And now it is the time to try Solaris 10 on this HP machine. The first thing to take into account is that the HP SmartArray P212 controller requires a closed-source binary from HP that is not distributed in the Solaris media. The cpqary3 driver can be downloaded from the HP Web site, although it is not very easy to find it out.

I downloaded the archived file for the cpqary3 driver, extracted its contents and burnt the ISO file to a CD-RW. Then I booted from the Solaris 10 DVD and applied the driver update. However, this didn’t work very well. The Solaris 10 DVD uses a 32-bit kernel which, apparently, does not like my >1.5TB disks. When the cpqary3 driver gets it prints a message to the console complaining that the detected disks have too many blocks for a 32-bit kernel. Very weird. I’m not entirely sure what a 32-bit kernel has to do with 64-bit LBA addressing but, so far, I haven’t been able to get Solaris 10 to recognize and get to like my disks.

So far, so good. VMware ESXi 4.0 didn’t work. Solaris 10 didn’t work either. Perhaps some of you, that are more experienced with Solaris than me, know how to fix this problem. In the mean time, I decided to take OpenSolaris for a spin. Fortunately, the final outcome with OpenSolaris has been a success. More on that later.

On this second post I want to talk about the interaction problems I experienced with the HP SmartArray P212 controller in this computer. The HP SmartArray P212 controller is certified for VMware ESXi 4.0 and Solaris 10. Initially I thought that using VMware would be useful to me in order to play with Solaris and even Windows 7.

However, I haven’t been able to get VMware ESXi 4.0 to work properly on this controller. If I create 4 logical drives in the HP controller, one for each phyisical disk, VMware finds the drives and figures out their right sizes. However, if configure a 3-drive RAID-5 logical volume in the HP controller, yielding a usable 3.0TB volume size, VMware finds and reports a 0.0B-sized volume. I tried different options from the HP SmartArray BIOS, like limiting the maximum bootable partition size, but the end result is always the same: VMware sees a 0.0B logical volume that can’t be used to install VMware neither to store virtual disks.

In the end, I ditched VMware ESXi 4.0 in favor of OpenSolaris, at least on this machine. I could have created 4 logical volumes, but it doesn’t make much sense for VMware itself. It makes perfect sense when running Solaris and using RAIDZ, though.

I haven’t been able to found any explanation to this problem other than VMware does not support LUNs bigger than 2TB. Is this the case? Do any of you have experience with VMware and LUNs larger than 2TB?

Recently I bought an HP Proliant DL180 G6 from Digitec. The machine looked wonderful on the paper and looks perfect for running VMware ESXi and Solaris. Unfortunately I had some relatively bad experiences with it.

The first minor problem I’ll talk about in this post is related to the drive trays. The machine comes with no pre-installed disks, which is perfect. I already have 4 SATA disks that I want to use on this machine. But as it turned out, the 4 plastic trays that came with the machine can’t be used to install any disk. These plastic trays are just meant to fill the empty space in the cage and to provide for adequate air flow when the trays are empty. So, in the end, I had to buy 4 HP plastic trays for my disks on eBay, which cost me about 150USD extra.

What I find frustrating is that a vendor tries to artificially lock one into buying their hard disks. On a free market, I believe a vendor can encourage one to buy their drives because they are superior, more reliable, or provide something appealing. But trying to force customers to buy HP disks by providing useless plastic trays is not fair, in my humble opinion. Particularly when the trays can be bought on eBay. In the end, the machine was sitting idle for a week while the trays where shipped to me.

Besides that, after placing my 4 drives in their trays and slipping them into the cage, everything was working wonderfully. The next problem comes with the HP SmartArray P212 controller and its interaction problems with VMWare ESXi and Solaris, but that’s something for a later post.

I typically don’t quote opinion-like articles like this, but in this case, I think it’s worth it. To keep it short, Linus thinks that people complaining because Microsoft wrote drivers for their own virtualization platform are hypocritical. And I have to agree with him:

Linus: “Microsoft Hatred Is a Disease”.

I learnt today that HP Proliant DL180 G6 blade servers come by default with no disks installed. So far so good as this is clearly advertised. But what is shocking to me is that the trays in this 4-drive machine are just mere plastic, useless pieces that can’t be used at all. These plastic trays are meant just to fill in the drive cage, but no actual disk can be attached to the trays. In order to install real drives one has to either buy HP disks or buy HP part 373211-001 — for example on eBay.

C’mon HP! What are you playing to? Do you want to force customers to buy your (expensive) disks? What do have HP SATA disks that WD Cavier disks do not have? Is that the reason why no one can buy HP part 373211-001 anymore except outside the HP parts store? I had to spend 140USD on 4 HP trays to install my disks. It’s not about the price, but the waste of time trying to find the right part number, the seller or being forced to use HP’s own dis units.

I honestly can now understand why people decide to build their own cheap machines, based on commodity parts, like for example Google does.

While trying to use Chromium on a Ubuntu 64-bit machine, I discovered I wasn’t able to browse to any web page. I always got the following error message:

This webpage is not available.

The webpage at http://www.google.com/ might be temporarily down or it may have moved permanently to a new web address.

Here are some suggestions:
Reload this web page later.
More information on this error
Below is the original error message

Error 105 (net::ERR_NAME_NOT_RESOLVED): The server could not be found.

DNS name resolution was working properly, so it was something else. I searched for this error and most of the search results were about Chrome on Windows having problems with proxy or firewall configuration. But, who cares about Windows? So, after spending a little bit more, I found the following issue in the official Google Code web site.

In the end, it was just a matter of:

$ sudo apt-get install lib32nss-mdns

Why Chromium has a an explicit dependency on mDNS is something that still puzzles me out.