Importing the old data pool

malachi@serveris:~# zpool import
  pool: data
    id: 16202109509961920001
 state: ONLINE
status: The pool is formatted using an older on-disk version.
action: The pool can be imported using its name or numeric identifier, though
    some features will not be available without an explicit 'zpool upgrade'.
config:

    data        ONLINE
      raidz2    ONLINE
        c6t0d0  ONLINE
        c6t1d0  ONLINE
        c7t0d0  ONLINE
        c7t1d0  ONLINE
        c8t0d0  ONLINE

malachi@serveris:~# zpool import data
cannot import 'data': pool may be in use from other system
use '-f' to import anyway
malachi@serveris:~# zpool import -f data
malachi@serveris:~# zpool status
  pool: data
 state: ONLINE
status: The pool is formatted using an older on-disk format.  The pool can
    still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
    pool will no longer be accessible on older software versions.
 scrub: none requested
config:

    NAME        STATE     READ WRITE CKSUM
    data        ONLINE       0     0     0
      raidz2    ONLINE       0     0     0
        c6t0d0  ONLINE       0     0     0
        c6t1d0  ONLINE       0     0     0
        c7t0d0  ONLINE       0     0     0
        c7t1d0  ONLINE       0     0     0
        c8t0d0  ONLINE       0     0     0

errors: No known data errors

malachi@serveris:~# zpool upgrade data
This system is currently running ZFS pool version 10.

Successfully upgraded 'data' from version 6 to version 10

malachi@serveris:~# zpool status
  pool: data
 state: ONLINE
 scrub: none requested
config:

    NAME        STATE     READ WRITE CKSUM
    data        ONLINE       0     0     0
      raidz2    ONLINE       0     0     0
        c6t0d0  ONLINE       0     0     0
        c6t1d0  ONLINE       0     0     0
        c7t0d0  ONLINE       0     0     0
        c7t1d0  ONLINE       0     0     0
        c8t0d0  ONLINE       0     0     0

errors: No known data errors
malachi@serveris:~# df -k
Filesystem           1K-blocks      Used Available Use% Mounted on
data                 713260393        55 713260339   1% /data

[trimmed...] 

malachi@serveris:~# cd /data

Mirrored ZFS bootable root

This post is no longer valid. Please check out this one instead.
 
So the instructions say that you are supposed to select two disks during installation to have it auto-mirror them. Unfortunately, it didn't allow me to select more than one disk (tried CTRL, Shift, mouse, keyboard, etc etc)... So, we are going to try to do it manually...

First, let's check the initial setup...
malachi@serveris:~$ zpool status
  pool: rpool
 state: ONLINE
 scrub: none requested
config:

    NAME        STATE     READ WRITE CKSUM
    rpool       ONLINE       0     0     0
      c5t0d0s0  ONLINE       0     0     0

errors: No known data errors
malachi@serveris:~$ zpool list
NAME    SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
rpool   230G  2.45G   228G     1%  ONLINE  -
malachi@serveris:~$ zfs list
NAME                                 USED  AVAIL  REFER  MOUNTPOINT
rpool                               2.44G   224G  55.5K  /rpool
rpool@install                         16K      -  55.5K  -
rpool/ROOT                          2.29G   224G    18K  /rpool/ROOT
rpool/ROOT@install                      0      -    18K  -
rpool/ROOT/opensolaris              2.29G   224G  2.23G  legacy
rpool/ROOT/opensolaris@install      58.4M      -  2.22G  -
rpool/ROOT/opensolaris/opt          3.60M   224G  3.60M  /opt
rpool/ROOT/opensolaris/opt@install      0      -  3.60M  -
rpool/export                         157M   224G    19K  /export
rpool/export@install                  15K      -    19K  -
rpool/export/home                    157M   224G   157M  /export/home
rpool/export/home@install             18K      -    21K  -

So how do we know which drives are available for mirroring? We know that c5t0d0s0 is the one we want mirrored, but....

su then format
malachi@serveris:~# format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c5t0d0
          /pci@0,0/pci1043,81fb@5/disk@0,0
       1. c5t1d0
          /pci@0,0/pci1043,81fb@5/disk@1,0
       2. c6t0d0
          /pci@0,0/pci1043,81fb@5,1/disk@0,0
       3. c6t1d0
          /pci@0,0/pci1043,81fb@5,1/disk@1,0
       4. c7t0d0
          /pci@0,0/pci1043,81fb@5,2/disk@0,0
       5. c7t1d0
          /pci@0,0/pci1043,81fb@5,2/disk@1,0
       6. c8t0d0
          /pci@7b,0/pci1043,81fb@5/disk@0,0
Specify disk (enter its number):
Specify disk (enter its number): ^C

It looks like c5t1d0 is the best choice...

malachi@serveris:~# zpool create rpool mirror c5t0d0s0 c5t1d0s0
invalid vdev specification
use '-f' to override the following errors:
/dev/dsk/c5t0d0s0 is part of active ZFS pool rpool. Please see zpool(1M).
malachi@serveris:~# 

hmmm.. yes, we knew it was... we wanted to make a copy... hmmm
malachi@serveris:~# zpool add rpool c5t1d0s0
invalid vdev specification
use '-f' to override the following errors:
/dev/dsk/c5t1d0s0 is part of exported or potentially active ZFS pool rootpool. Please see zpool(1M).


hmmm... is that from the current install or the previous install?
malachi@serveris:~# zpool attach rpool c5t0d0s0 c5t1d0s0
invalid vdev specification
use '-f' to override the following errors:
/dev/dsk/c5t1d0s0 is part of exported or potentially active ZFS pool rootpool. Please see zpool(1M).


hmmm....
malachi@serveris:~# zpool attach -f rpool c5t0d0s0 c5t1d0s0
cannot attach c5t1d0s0 to c5t0d0s0: device is too small


hmmm, I tried formatting c5t1d0s0, but it still won't attach
zpool import shows that the other drives are still set as a raidz, so this is definitely the right drive... how to get it to import?

I think I may have to start over and use the manual installer rather than the GUI installer :(

Setting up a static network configuration with NWAM

Step 1: Figure out which network interface to use...

malachi@serveris:/etc/nwam# ifconfig -a
lo0: flags=2001000849 mtu 8232 index 1
    inet 127.0.0.1 netmask ff000000
nge0: flags=201004843 mtu 1500 index 12
    inet 10.10.1.155 netmask ffffff00 broadcast 10.10.1.255
    ether 0:18:f3:d5:cd:9b
nge1: flags=201000802 mtu 1500 index 13
    inet 0.0.0.0 netmask 0
    ether 0:18:f3:d5:cd:9c
lo0: flags=2002000849 mtu 8252 index 1
    inet6 ::1/128

Step 2: Setup the nameservers....
gedit /etc/resolv.conf
add in the DNS servers...
EX:
nameserver 10.10.1.1
nameserver 66.93.87.2
nameserver 216.231.41.2

Step 3: Switch to using the DNS servers...

malachi@serveris:/etc# cp nsswitch.conf nsswitch.conf.original
malachi@serveris:/etc# cp nsswitch.dns nsswitch.conf
malachi@serveris:/etc# svcadm restart svc:/network/dns/client:default

Step 4: Switch to a static IP
gedit /etc/nwam/llp
change the 'nge0 dhcp' (or whatever device from Step 1) to 'nge0 static 10.10.1.10/24' (or whatever ip)

svcadm restart svc:/network/physical:nwam

Only video mode: 640x480

After a fresh install, the only video mode I had available was 640x480 @ 50 Hz. Well, that's definitely not right. It could be because of the KVM, but I noticed a lot of people using nVidia had the same problem. I used the notes I found here to fix it...

I changed quite a few things, saved, exited, came back in, ran the utility again and saved the config once I had it where I wanted it... That of course overwrote most of the changes (like the monitor name), so... here's the tidbit end result that matters...

Section "Monitor"
    # HorizSync source: xconfig, VertRefresh source: xconfig
    Identifier     "Monitor0"
    VendorName     "Unknown"
    ModelName      "CRT-0"
    HorizSync       30.0 - 81.0
    VertRefresh     56.0 - 75.0
    Option         "DPMS"
EndSection

Fresh Install!

So I decided to start over... I just installed OpenSolaris 2008.05

Will be posting configuration notes as I go so that it is easy to find them again later =)

Server Built

Brett came by to help me get the server built yesterday. I have been waiting for Solaris 10 b60 (which has AMD-V support), but I am starting to seriously consider using b59 to install and then upgrading.

Only real problem with the build was that the Armor Extreme case had drive rails that got in the way of the Addonics drive cages... Brett managed to dremmel them out of the way.

On the left you will see 4 pictures... Yes, I resized it (Brett hates those 13MB photos), but you can still click on it to get a slightly larger photo.
The left images are with the doors open. The right images are with the doors closed.
The top images are with a flash (see detail) and the bottom without (see lights)...

Now to choose an installation medium :)

Belkin F1DE208C

Well, we got the room rearranged, rack moved, and setup the new KVM...

Unfortunately, it doesn't appear to work. Both 2x7-segment LEDs say "88" and the rest of the lights are randomly flashing...

I would look up the error code, but the manual has nothing, the website has nothing, and I am not finding anything on Google. Guess I will have to talk with Belkin support tomorrow.

Good thing I have the laptop.

Monitoring

Consider running Nagios, probably in Xen, unless I'd rather have an appliance.

Solaris*VoIP

Consider whether to use the Solaris version of Asterisk or Linux version (either way, it will be running in Xen)

Solaris 0-day Telnet Exploit

make sure to fix this

Solaris-optimized AMP Stack

ZFS Learning Center

Managing ZFS in Solaris 10 Containers