Developer's Quarterdeck Log

So you want to try out OpenSolaris but you don't have a machine available. Then VMware is what you need. Even if you do, VMware is great tool to get help you get started with OpenSolaris and can be a great tool to help you on your kernal hacking efforts. VMware workstation 5.5 has “experimental” support for Solaris but I have found that OpenSolaris actually runs quite well with it. This article will take you through the steps to get OpenSolaris up and running on VMware and assumes you are new to VMware and OpenSolaris. This article will also be helpful if are just interested in Solaris on VMware.

First, you need to get yourself a copy of VM Workstation. It will run on Windows and some Linux distros. You really don't need super impressive hardware to get started. My first test machine has only 512 mb of memory and is limited with disk space which requires me to run off an external USB drive. In both cases the performance is tolerable. But of course the more memory, disk, and cpu you have the better!

You may be wondering, so what is VMware? VMware is an application that emulates a PC or runs as a virtual machine (VM). So basically what happens is that when you run OpenSolaris on VMware it acts as if it is own dedicated box when actuality it is running as an application on Windows or Linux. Hardware such as NIC and video cards are emulated which can be your savior if you hardware isn't supported yet.

Once VM Workstation is installed start off by creating a new virtual machine (File -> New-> Virtual Machine...). On the first selection just select the typical configuration, after all if you need custom then you don't need this article! Now check “Sun Solaris” and select “Solaris 10 (experimental)” or if you are lucky enough to have a 64-bit machine “Solaris 10 64-bit (experimental)”. The next step will ask where you want to save your VM settings and log files. Think about this for a second before you accept the defaults. On Windows, the default is in the “My Documents” folder which isn't always the best choice. It is a good idea to keep the VM's disk image file with the settings so this folder is going to get big. Another thing to think about is organization. You will probably getting hooked on this virtual machine thing and you'll want to keep it organized. Think also about a good naming scheme. For now a good name would be “OpenSolaris_base” and put it on a partition with lots of space. The next step is to determine what kind of networking you will setup. The best choice is bridged. With this choice Solaris will be just another node on the network. However, if you are unable to assign your instance its own IP address or if you want to move around the VM to different networks then select NAT. Now you need to determine how much disk space you want to allocate. The default is 8.0 gigs but go ahead and double that since you'll need some space to do a build and such. I recommend that you do not allocate all the disk space now unless you have massive free disk space and you want to get every little bit of speed out of the VM. But just remember that having all the disk pre-allocated might get in the way later when you start doing things like cloning images. Also, select “Split into 2 GB files” if you plan on moving the VM around since it will make easier to zip it up or put it onto DVD-R's.

Before you start the VM, you may want to edit the virtual machine settings that were just built. Particularly for the amount of memory allocated to the vm especially if you plan on building OpenSolaris. Give it as much as you can spare.

Now get yourself a copy of Solaris Express. Go ahead and get the DVD version even if you don't have a DVD. Plus, there is no need to burn it to a DVD-R. Once you have the Solaris Express ISO file go back into the VM settings. Select the CD-ROM and choose “Use ISO image” and enter the path to the ISO file. Since VMware is emulating a CD-ROM device it just mounts the ISO and the VM will think it is a physical drive.

Once you have ISO mounted it is time to start up the vm. You'll notice the screen looks a lot like your PC when it starts up.

Eventually you'll get to the Solaris install screens. To start off you'll be setting up networking on the vm's network interface, pcn0. One thing to keep in mind is that if you select DHCP and you are using the NAT networking scheme you'll need to edit the /etc/default/dhcpagent file to keep your hostname from getting screwed up once you have it up and running. After the network and time zone information is complete you'll be asked about the installer options. On the first screen just select “standard”. Next select “Manually eject CD/DVD” since you need a reminder to point to a new ISO if needed (but if you took my advice on the DVD you won't need to worry). You can select yes for automatic reboot if you want. The next screen of interest is the disk partitioning. First let Solaris use the entire disk. Then go ahead an let the installer do an auto-layout. It will ask you to select the file systems for auto-layout. I recommend one for / and one for swap if your new to this stuff. I really don't care for the way auto-layout sets things up but it is easy to customize its defaults. Just hit F4 and adjust accordingly. First of all the swap is probably too small if you want to do a OpenSolaris build. Plus the / slice is too small with /export/home taking up all the space. Since this is a test system here is what I do. I get rid of home slice and then set swap to something like 2048 MB then the rest to /. Here's a trick to do that if you don't want to do math. First set / to 0 then set it to the “Free” number on the last line. That is pretty much it and it is time to let the install run.

Now after the install completes you will run into your first problem. During the install, the screen is at a normal size with no problems but the first time Solaris boots up the screen is all squished up. Some how Solaris forgot what how things were setup during the install. So you'll need to run kdmconfig to straighten things out. What you'll have to do is log into the text mode before the login screen appears. If you miss it I suggest you log in to a CDE session, it will be a little easier to manage with the small screen size. Here you can reboot and try again or run kdmconfig from a maximized terminal window. The only caveat being you won't be able to test your new setup. So once you are at the command line run kdmconfig as root. The first choice you'll have is xorg or xsun. Both will work. If you're an Linux fugitive then xorg setup should be old hat. Otherwise just select xsun just to get started with because it is a little easier. You can play with setting up xorg later. Once you have xsun setup scroll all the way down to the second to the bottom option on the video device list and select “XF86-VMWARE Vmware SVGA virtual video cards”. Next select the first MultiFrequency monitor, the one that is 100khz and can do 1600x1200 @80hz. For the screen size just leave it at 17-inch. Now you can pick a resolution. The minimum you want to mess with is probably 1024x768 but think about the screen resolution you are currently running on the host desktop and adjust accordingly. Check the settings you just entered and then save and test the configuration. You will get a chance to try out the resolution if it looks good then accept the settings otherwise keep trying to adjust things. If the VM screen seems way too big then try to change the monitor type. Don't forget the keyboard and pointing device settings if yours is different than the defaults. Give your new VM a reboot to make sure the changes worked. If all went well you'll see a properly sized login screen.

Before you do anything else make sure you make a snapshot of the VM you just created. This is very important because when you start playing with OpenSolaris the chances of you brickifying a system are quite high. Optionally before you take a snapshot you can create a default user account first. The easiest way for the novice is to login and then run “smc” from the command line. But once you have the new account setup and tested the shut the virtual machine down. Then click on the “Clone this virtual machine” icon. You have the option of a full clone or linked clone. For the first image I suggest is a full clone so you can put the backup in a nice and safe place. That way one day in the future when get a little too cocky and really screw things up you'll have a fall back.

Any time you login to a graphic desktop don't do anything else until you disable the screen saver. Otherwise you'll have a locked VM that won't unlock. There is a bug in the current builds with the screen saver on certain video cards and VMware is one of them. Even when the bug gets fixed you really don't want to be running a screen saver in VMware anyway.

That's it to get the base Solaris Express install up. Next step is to get the OpenSolaris source or BFU and install it. I highly recommend, no actually I INSIST that you clone your working VM before you BFU it. Even if you are an expert at the build process you never know why kind of bug is waiting for you in the latest release. Remember that OpenSolaris is essentially experimental and Solaris on top of VMware is also experimental.

If you need more help with OpenSolaris I suggest you visit the OpenSolaris and the VMware forums. Both are filled with smart people willing to help. Also check on the VMware workstation help. Believe it or not is has information specific to installing Solaris on VMware.