Ha. If at first you don't succeed, cheat. For the last week or so, I've been looking into how one could build a minimal OS that would have just enough stuff in it to support some full-featured Java apps (ie, a Hadoop/Accumulo stack). Ron got me thinking about starting with nothing and working my way up, as opposed to what most people do, which is start with a bloated distro and try to strip it down.
The first part was easy: I pulled the latest Linux Kernel and built a stripped down version with basic drivers to get a VM running. I then installed BusyBox to a blank initramfs to give the node something to do after booting the kernel. Static programs worked great, but when I tried plugging in any executables with dynamic libraries, they of course didn't work because there was no libc. Given that Java needs libc and can't be recompiled (thus no uClibc), I had to figure out how to get glibc in the initramfs.
Every time I've tried to build glibc I've failed- sometimes spectacularly, destroying the machine (funny how much breaks when you overwrite a working glibc with a broken one). Over the last few days, I've tried a number of different things, but none of the installs ever worked. Things landed in the wrong places. Glibc took issue with my kernel. Older versions didn't stick, either.
I finally gave up and cheated. I took a stock fedora glibc rpm, manually unpacked it into my initramfs, and then everything magically worked. I plugged the JRE in as well, and voila I got the below message in the VM. It's annoying to not get it to work from scratch, but whatever, it works. And since KVM boots directly from the kernel/initramfs files, the VM is up and giving a prompt after about one second. The initramfs is 55MB compressed. Not exactly floppy drive material, but also, much better than any other distro installation that includes Java. Guess I've got to figure out how to get Hadoop going on it now.