Server Died

So I got home last night to find the server had power but was not responding (mouse, keyboard, ssh, anything). I tried rebooting, but it kept handing at "Hostname: serveris" and wouldn't go any further (even in single-user mode). I saw some chatlogs online that suggested adding '-k -a -d verbose' and using '/dev/null' to the answer of any questions (like /etc/system replacement)...

I tried that and got this far (see left)...

Looking around some more, I saw that if I changed the '-k' to '-kd' it would drop it into debug mode. At that point, I did the following:



[0] moddebug/W 80000000
[0] :c

This allowed me to see a few more details.... (sorry for the blurriness of the pic - it was about 3am)

After trying to find anything online that would help (and the IRC channel) I finally said screw it and decided I would reinstall opensolaris on the root mirror.

I downloaded the USB version of OpenSolaris 0906 111a, but evidentially my quad core machine does not have the option of booting from USB (WTF?). I reburned the CD version and installed it. One thing that confused me is that although my old system was 10/08 upgraded to 111a and the new version was supposed to be 0906 111a, it now says 101b.

Trying to boot the new one, it again hung. At a different position, but... I was starting to think it was a hardware problem. I let it try to boot overnight and the next morning it was finally at the login prompt... with the old install.

The logs showed that it had tried to load the Belkin UPS a few steps after where it locked up, so I unplugged the UPS. I went ahead and applied all updates and rebooted. It took about 5 hours for it to finally boot again (though it did). It still says I am using 101b and that there are no new updates.

The xVM instance is there and I was able to start it. The whole root zone however is gone. The ZFS partition is there, and empty. zoneadm doesn't show anything but global. So, I am going to try to recreate the global zone, but... I still don't know what happened. I am also concerned that it currently takes about 5 hours to boot.

Playing some music

So I wanted to listen to some music, but neither youtube or pandora were working (not sure why, they were yesterday)...

So I tried Songbird, which I use at work. It wouldn't work either:
Error: GStreamer error: Internal data flow error. Additional information: gstbasesrc.c(2234): gst_base_src_loop (): /GstPlayBin2:player/GstURIDecodeBin:uridecodebin1/GstFileSrc:source:
streaming task paused, reason not-linked (-1)

So I tried Rhythm, but it couldn't play anything either...

WTF? Right?
So I did this instead. Now I have music playing :)

Don't forget to escape the inner class

I think I have used Windows too long...

malachi@serveris[1]:~/work/ambrosia/syrinx % java -cp target/syrinx-2.3-SNAPSHOT.jar org.eoti.sound.BrainwaveOscillator$Test
Error: Main method not found in class org.eoti.sound.BrainwaveOscillator, please define the main method as:
public static void main(String[] args)
Exception in thread "main" java.lang.RuntimeException: Main method not found in org.eoti.sound.BrainwaveOscillator
at sun.launcher.LauncherHelper.signatureDiagnostic(LauncherHelper.java:209)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:197)

malachi@serveris[1]:~/work/ambrosia/syrinx % java -cp target/syrinx-2.3-SNAPSHOT.jar org.eoti.sound.BrainwaveOscillator\$Test

You have to escape the $ for the inner class :(

Hudson Maven Release Plug-in Plug-in + Hudson Google Code plugin

I was excited to be able to use the Hudson Google Code plugin for my project.



Unfortunately, when I tried to also use the Hudson Maven Release Plug-in Plug-in with it, it always failed to authenticate...


[INFO] Checking in modified POMs...
[INFO] Executing: /bin/sh -c cd /home/hudson/jobs/ambrosia-jdk6-standalone/workspace/. && svn --username hudson@eoti.org --password '*****' --non-interactive commit --file /var/tmp/maven-scm-2033681108.commit --targets /var/tmp/maven-scm-1162162468517704084-targets
[INFO] Working directory: /home/hudson/jobs/ambrosia-jdk6-standalone/workspace/.
[HUDSON] Archiving /home/hudson/jobs/ambrosia-jdk6-standalone/workspace/./pom.xml to /home/hudson/jobs/ambrosia-jdk6-standalone/modules/org.eoti$ambrosia/builds/2009-04-24_00-59-40/archive/org.eoti/ambrosia/2.2-SNAPSHOT/pom.xml
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Unable to commit files
Provider message:
The svn command failed.
Command output:
svn: Commit failed (details follow):
svn: MKACTIVITY of '/svn/!svn/act/0776c9b3-28c2-443e-a144-97770aaff2b2': authorization failed: Could not authenticate to server: rejected Basic challenge (http://ambrosia.googlecode.com)

[INFO] ------------------------------------------------------------------------
[INFO] Trace
org.apache.maven.BuildFailureException: Unable to commit files
Provider message:
The svn command failed.
Command output:
svn: Commit failed (details follow):
svn: MKACTIVITY of '/svn/!svn/act/0776c9b3-28c2-443e-a144-97770aaff2b2': authorization failed: Could not authenticate to server: rejected Basic challenge (http://ambrosia.googlecode.com)

at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:699)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeStandaloneGoal(DefaultLifecycleExecutor.java:553)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:523)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:371)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:268)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:181)
at org.apache.maven.lifecycle.LifecycleExecutorInterceptor.execute(LifecycleExecutorInterceptor.java:65)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:356)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:137)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:356)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
at hudson.maven.agent.Main.launch(Main.java:158)
at hudson.maven.MavenBuilder.call(MavenBuilder.java:162)
at hudson.maven.MavenModuleSetBuild$Builder.call(MavenModuleSetBuild.java:578)
at hudson.maven.MavenModuleSetBuild$Builder.call(MavenModuleSetBuild.java:524)
at hudson.remoting.UserRequest.perform(UserRequest.java:97)
at hudson.remoting.UserRequest.perform(UserRequest.java:46)
at hudson.remoting.Request$2.run(Request.java:236)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: org.apache.maven.plugin.MojoFailureException: Unable to commit files
Provider message:
The svn command failed.
Command output:
svn: Commit failed (details follow):
svn: MKACTIVITY of '/svn/!svn/act/0776c9b3-28c2-443e-a144-97770aaff2b2': authorization failed: Could not authenticate to server: rejected Basic challenge (http://ambrosia.googlecode.com)

at org.apache.maven.plugins.release.PrepareReleaseMojo.execute(PrepareReleaseMojo.java:190)
at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:483)
at hudson.maven.agent.PluginManagerInterceptor.executeMojo(PluginManagerInterceptor.java:182)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:678)
... 28 more
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 40 seconds
[INFO] Finished at: Fri Apr 24 01:00:22 PDT 2009
[INFO] Final Memory: 17M/218M
[INFO] ------------------------------------------------------------------------
channel stopped
Finished: FAILURE


This was because it was using http instead of https to do the checkout...
The solution was to only partially use the Google Code plugin...

Compiling in the zone; eg. where's the damned math.h?

So it appears that some of the basic include files were not included by default in the zone either. To get that working (and thus get random stuff compiling correctly)...

pfexec pkg install SUNWlibm

pkgadd, pkginfo, etc missing

Upon installation of the new zone, I was surprised that some of the basic pieces like pkgadd was missing. It appears (at least for zones) that IPS is definitely the default now.

Unfortunately, there are times when you still want those commands...  like when testing build processes written for Solaris 10.

Following the instructions in the previous blog, here's what you need to do to get it working:

If this is the first time...
[kallisti:malachi(127)] ~/work% pfexec pkg set-authority -O http://pkg.opensolaris.org/dev/ os-dev


Then do
[kallisti:malachi(127)] ~/work% pfexec pkg install SUNWpkgcmds

Installing a named IPS package in a zone

Let's say you have a package you'd like to install, but don't have a .pkg for it.... For example, gtar.

Go to http://pkg.opensolaris.org/dev/ and locate the package (search box)
For example, SUNWgtar@1.17,5.11-0.111:20090331T082820Z
Note the name before the @
IE, SUNWgtar

If this is the first time...
root@kallisti:/opt# pkg set-authority -O http://pkg.opensolaris.org/dev/ os-dev

Then to do the install...
root@kallisti:/opt# pkg refresh
root@kallisti:/opt# pkg install SUNWgtar

Hudson

I've been thinking about setting up a continuous integration server for awhile. After doing some configuration of Cruise Control at work, I decided to try Hudson at home.

First things first... Virtualization? I decided I would rather have it virtualized if for no other reason than that dotster already had dns setup for one of my currently unused ips and I didn't want to wait for the ip change to propagate.

I can't run VirtualBox because I am running xVM and VirtualBox refuses to start if you are in xVM mode. I could run it in xVM... but I really don't like dedicating a large chunk of my memory to it... so zone it is.

malachi@serveris[0]:/data % pfexec zonecfg -z eoti.org
eoti.org: No such zone configured
Use 'create' to begin configuring a new zone.
zonecfg:eoti.org> create
zonecfg:eoti.org> set zonepath=/data/eoti.org
zonecfg:eoti.org> set autoboot=true
zonecfg:eoti.org> add net
zonecfg:eoti.org:net> set address=10.10.1.13/16
zonecfg:eoti.org:net> set physical=nge0
zonecfg:eoti.org:net> set defrouter=10.10.1.1
zonecfg:eoti.org:net> end
zonecfg:eoti.org> verify
zonecfg:eoti.org> commit
zonecfg:eoti.org> exit

malachi@serveris[1]:/data % pfexec zoneadm -z eoti.org install
/data/eoti.org must not be group readable.
/data/eoti.org must not be group writable.
/data/eoti.org must not be group executable.
/data/eoti.org must not be world readable.
/data/eoti.org must not be world executable.
could not verify zonepath /data/eoti.org because of the above errors.
zoneadm: zone eoti.org failed to verify

malachi@serveris[1]:/data % pfexec chmod 700 /data/eoti.org

malachi@serveris[1]:/data % pfexec zoneadm -z eoti.org install

malachi@serveris[1]:/data % pfexec zoneadm -z eoti.org boot

malachi@serveris[1]:/data % pfexec zlogin -C eoti.org
(chose option 6 - xterms)
Host name for nge0:1 kallisti
domain: eoti.org
Server's IP address: 10.10.1.1
Server's IP address: 66.93.87.2
Server's IP address: 216.231.41.2
pwd: ***********

malachi@serveris[1]:/data % pfexec zoneadm -z eoti.org reboot
malachi@serveris[1]:/data % zoneadm list -v
malachi@serveris[1]:/data % pfexec zlogin eoti.org

root@kallisti:~# svcadm disable autofs
root@kallisti:~# useradd -g staff -s /usr/bin/bash -c "Malachi de AElfweald" -m -P "Primary Administrator" malachi
root@kallisti:~# passwd malachi

malachi@serveris[1]:/data % pfexec vi /etc/hosts
10.10.1.13 kallisti.eoti.org kallisti

http://wiki.hudson-ci.org/display/HUDSON/OpenSolaris+IPS+package
but had to change this line to include java
<envvar name="'PATH'" value="'/opt/jdk1.7.0/bin:/usr/bin:/usr/sbin:/usr/ccs/bin:/usr/local/bin:/usr/local/sbin:/usr/sfw/bin' />

couldn't have it create the zfs filesystem within the zone... but it was already on a zfs filesystem anyway, so no worries

Every once in awhile, Firefox starts giving me this error when I choose to "Open" instead of "Save As...":

BLAH could not be saved, because you cannot change the contents of that folder. Change the folder properties and try again, or try saving in a different location.

I found this page that gives hints on how to fix it.

It seems just clearing my download history (Clear List) was enough to get it working again...

Failure to boot into graphical mode after upgrade

A coworker upgraded his OpenSolaris from snv_106 to snv_109. Upon rebooting, it failed to launch into X. For awhile, we assumed there was a problem in the grub menu.lst, but it seems the answer was a bit different.

When running OpenSolaris inside VirtualBox, you usually 'Install Guest Additions' so that you can run full-screen. Apparently, the upgrade from snv_106 to snv_109 broke the compatibility with installed guest additions.

The final solution was to uninstall and reinstall the guest additions.

From the menu (or CTRL-Home if you are already full screen) go to Device | Install Guest Additions. That will mount the virtual cdrom.
# cd /cdrom
# pfexec pkgrm SUNWvboxguest
# pfexec pkgadd -d SUNWvboxguest
# pfexec reboot

It should all work now :)

ZSH Tab Completion Fix for Subversion 1.5

So one of the annoying things I noticed was that tab autocompletion for zsh was broken if I was using it with svn; showing:
_arguments:comparguments:303: invalid argument: ARG

Based on notes here I fixed it by doing the following:

$ cd /usr/share/zsh/4.3.4/functions/Completion/Unix/_subversion
$ pfexec mv _subversion ~/original._subversion
$ pfexec wget http://gvn.googlecode.com/svn/trunk/contrib/zsh/_subversion
$ pfexec chmod 444 _subversion

It all works now :)

Python 2.5 breaks OpenSolaris menus

In the last week, some of my menu icons (like 'Main Menu') had stopped working. Since they wouldn't launch, I couldn't determine what the problem was. Since the right-click didn't have a properties, I couldn't even determine what it was trying to do.

From a command line, run 'alacarte'.  That complained about missing libraries for Python. Looking at the system, I could see that there was a recent upgrade from Python 2.4 to Python 2.5.

The correct solution was probably to import all of the libraries... but I didn't care about that. I had only upgrade Python in an effort to get some of the other apps to compile...

So I moved /usr/bin/python to /usr/bin/broken-python and re-symlinked /usr/bin/python2.4 to /usr/bin/python.

Now all the apps work again.

Installing git

wget http://kernel.org/pub/software/scm/git/git-1.6.1.tar.gz
gzip -dc git-1.6.1.tar.gz | tar xvfp -
pfexec mkdir /opt/git-1.6.1
pfexec chmod 777 /opt/git-1.6.1 [probably not really necessary]
cd git-1.6.1
./configure --prefix=/opt/git-1.6.1
gmake
pfexec make install
pfexec chmod 755 /opt/git-1.6.1 [again probably not really necessary]
vi ~/.bashrc
add: export PATH=$PATH:/opt/git/bin
login to new terminal:
git --version