10 free virtualization tools

More and more IT Vendors are validating the benefits of desktop and server virtualization, from energy efficiency to better resource utilization. But if you’re still leery of forking over big dollars for the technology, there are some low-cost and no-cost ways to give virtualization a try.

Virtualization freebies are available both from the open source community and from vendors such as Microsoft and VMware. Here’s a sampling:

Microsoft Virtual Server 2005 R2

A virtualization environment for Windows Server 2003, Windows Server 2003 Service Pack 1, Windows XP Professional Edition and Windows XP Service Pack 2. It works on x86 servers or workstations and is available in 32- or 64-bit versions.

OpenVZ

Linux-based software that lets the IT administrator create secure virtual environments. Each virtual private server can be rebooted independently, co-existing with other virtual private servers.

The open source project is supported by SWsoft, which markets a commercial version called Virtuozzo.

Q

Open source software for running Windows or Linux on a Macintosh. It allows users to switch between guest PCs and restart guest PCs at any point. Q also enables users to exchange files between the host operating system and the guest.

Q is based on the QEMU open source CPU emulator. You need to be careful about using Q – according to the Web site, it is still alpha software.

QEMU

A generic and open source machine emulator and virtualization software developed by Fabrice Bellard, the author of a compact C compiler. QEMU can run operating systems and programs developed for one machine on a different machine.

When used as virtualization software, the host and guest machine must use x86-compatible processors. In emulation mode, it supports x86 PFCs, MIPS R4000, Sun SPARC and PowerPC processors.

Technical Architecture Solutions TCO Calculator

Technical Architecture Solutions, a disaster recovery and business continuity consultancy, is offering a TCO Calculator for server virtualization. It is Web-based and allows IT administrators to estimate how much money they will save by using VMware ESX. The calculator consists of a Microsoft Excel spreadsheet and is based on four virtual machines per virtual server.

VMware Server, Player and Converter

VMware Server is VMware’s starter kit for Windows and Linux server virtualization. It can host Windows, Linux, NetWare and Solaris as guest operating systems.

VMware Player lets IT administrators run virtual machine instances created by other VMware products. With it, you can download prebuilt virtual machines from the VMware Technology Network or share them with other organizations.

Virtual Iron Single Server Virtualization and Management

Virtual Iron’s Single Server Virtualization and Management software creates Linux or Windows virtual servers. It runs in 32- and 64-bit workloads that are stored on iSCSI, storage-area networks or local storage. Virtualized nodes created with Virtual Iron’s Single Server software must have Intel Virtualization Technology or AMD Virtualization hardware assist.

Virtual Iron and Provision Networks Virtual Desktop Infrastructure

An alternative to terminal services environments and fat-client desktop implementations. It lets an IT administrator host five individual Windows desktop PCs inside virtual machines running on servers in the data center.

Xen

This open source package for x86 server virtualization was developed by the University of Cambridge and commercialized by XenSource. Xen, which runs on Linux, supports Windows and Linux as guest operating systems.

XenSource XenExpress

XenExpress, from XenSource, is a free virtualization package for x86 servers that lets IT administrators host as many as four virtual machines on servers equipped with 4GB of physical RAM or two physical sockets.

It supports Windows Server 2003, Windows XP and Windows 2000 as guest operating systems. It also supports Red Hat, SUSE and Debian Linux guest operating systems.

XenExpress is based on the open source Xen hypervisor.

Rebooting the Magic Way

If you have ever had a hard drive fail on a remote server you may remember the feeling you had after trying to issue the following commands:

   
 # reboot
 bash: /sbin/reboot: Input/output error
 # shutdown -r now
 bash: /sbin/shutdown: Input/output error

Obviously, there is a problem with your drive. These commands are failing because the kernel is unable to load the /sbin/reboot and /sbin/shutdown binaries from the disk so that it can execute them.

A fsck on the next boot might be able to correct whatever is wrong with the disk, but first you need to get the system to reboot. If your machine is located at a managed hosting provider then you could submit a reboot ticket, but you'll have to wait for someone to take responsibility.

Wouldn't it be nice if there was a way to ask the kernel to reboot without needing to access the failing drive? Well, there is a way, and it is remarkably simple.

The "magic SysRq key" provides a way to send commands directly to the kernel through the /proc filesystem. It is enabled via a kernel compile time option, CONFIG_MAGIC_SYSRQ, which seems to be standard on most distributions. First you must activate the magic SysRq option:

   
 echo 1 > /proc/sys/kernel/sysrq

When you are ready to reboot the machine simply run the following:

    
echo b > /proc/sysrq-trigger

This does not attempt to unmount or sync filesystems, so it should only be used when absolutely necessary, but if your drive is already failing then that may not be a concern.

In addition to rebooting the system the sysrq trick can be used to dump memory information to the console, sync all filesystems, remount all filesystems in read-only mode, send SIGTERM or SIGKILL to all processes except init, or power off the machine entirely, among other things.

Also, instead of echoing into /proc/sys/kernel/sysrq each time you can activate the magic SysRq key at system boot time using sysctl, where supported:

    
echo "kernel.sysrq = 1" >> /etc/sysctl.conf

If you would like to learn more about magic SysRq you can read the sysrq.txt file in the kernel documentation.

sysrq.txt

 Linux Magic System Request Key Hacks
 Documentation for sysrq.c
 
 *  What is the magic SysRq key?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 It is a 'magical' key combo you can hit which the kernel will respond to
 regardless of whatever else it is doing, unless it is completely locked up.
 
 *  How do I enable the magic SysRq key?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 You need to say "yes" to 'Magic SysRq key (CONFIG_MAGIC_SYSRQ)' when
 configuring the kernel. When running a kernel with SysRq compiled in,
 /proc/sys/kernel/sysrq controls the functions allowed to be invoked via
 the SysRq key. By default the file contains 1 which means that every
 possible SysRq request is allowed (in older versions SysRq was disabled
 by default, and you were required to specifically enable it at run-time
 but this is not the case any more). Here is the list of possible values
 in /proc/sys/kernel/sysrq:
    0 - disable sysrq completely
    1 - enable all functions of sysrq
   >1 - bitmask of allowed sysrq functions (see below for detailed function
        description):
           2 - enable control of console logging level
           4 - enable control of keyboard (SAK, unraw)
           8 - enable debugging dumps of processes etc.
          16 - enable sync command
          32 - enable remount read-only
          64 - enable signalling of processes (term, kill, oom-kill)
         128 - allow reboot/poweroff
         256 - allow nicing of all RT tasks
 
 You can set the value in the file by the following command:
     echo "number" >/proc/sys/kernel/sysrq
 
 Note that the value of /proc/sys/kernel/sysrq influences only the invocation
 via a keyboard. Invocation of any operation via /proc/sysrq-trigger is always
 allowed (by a user with admin privileges).
 
 *  How do I use the magic SysRq key?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 On x86   - You press the key combo 'ALT-SysRq-'. Note - Some
            keyboards may not have a key labeled 'SysRq'. The 'SysRq' key is
            also known as the 'Print Screen' key. Also some keyboards cannot
     handle so many keys being pressed at the same time, so you might
     have better luck with "press Alt", "press SysRq", "release SysRq",
     "press ", release everything.
 
 On SPARC - You press 'ALT-STOP-', I believe.
 
 On the serial console (PC style standard serial ports only) -
            You send a BREAK, then within 5 seconds a command key. Sending
            BREAK twice is interpreted as a normal BREAK.
 
 On PowerPC - Press 'ALT - Print Screen (or F13) - , 
              Print Screen (or F13) -  may suffice.
 
 On other - If you know of the key combos for other architectures, please
            let me know so I can add them to this section.
 
 On all -  write a character to /proc/sysrq-trigger.  e.g.:
 
   echo t > /proc/sysrq-trigger
 
 *  What are the 'command' keys?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 'b'     - Will immediately reboot the system without syncing or unmounting
           your disks.
 
 'c' - Will perform a kexec reboot in order to take a crashdump.
 
 'd' - Shows all locks that are held.
 
 'e'     - Send a SIGTERM to all processes, except for init.
 
 'f' - Will call oom_kill to kill a memory hog process.
 
 'g' - Used by kgdb on ppc and sh platforms.
 
 'h'     - Will display help (actually any other key than those listed
           here will display help. but 'h' is easy to remember :-)
 
 'i'     - Send a SIGKILL to all processes, except for init.
 
 'k'     - Secure Access Key (SAK) Kills all programs on the current virtual
           console. NOTE: See important comments below in SAK section.
 
 'l'     - Shows a stack backtrace for all active CPUs.
 
 'm'     - Will dump current memory info to your console.
 
 'n' - Used to make RT tasks nice-able
 
 'o'     - Will shut your system off (if configured and supported).
 
 'p'     - Will dump the current registers and flags to your console.
 
 'q'     - Will dump per CPU lists of all armed hrtimers (but NOT regular
           timer_list timers) and detailed information about all
           clockevent devices.
 
 'r'     - Turns off keyboard raw mode and sets it to XLATE.
 
 's'     - Will attempt to sync all mounted filesystems.
 
 't'     - Will dump a list of current tasks and their information to your
           console.
 
 'u'     - Will attempt to remount all mounted filesystems read-only.
 
 'v' - Dumps Voyager SMP processor info to your console.
 
 'w' - Dumps tasks that are in uninterruptable (blocked) state.
 
 'x' - Used by xmon interface on ppc/powerpc platforms.
 
 '0'-'9' - Sets the console log level, controlling which kernel messages
           will be printed to your console. ('0', for example would make
           it so that only emergency messages like PANICs or OOPSes would
           make it to your console.)
 
 *  Okay, so what can I use them for?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Well, un'R'aw is very handy when your X server or a svgalib program crashes.
 
 sa'K' (Secure Access Key) is useful when you want to be sure there is no
 trojan program running at console which could grab your password
 when you would try to login. It will kill all programs on given console,
 thus letting you make sure that the login prompt you see is actually
 the one from init, not some trojan program.
 IMPORTANT: In its true form it is not a true SAK like the one in a :IMPORTANT
 IMPORTANT: c2 compliant system, and it should not be mistaken as   :IMPORTANT
 IMPORTANT: such.                                                   :IMPORTANT
        It seems others find it useful as (System Attention Key) which is
 useful when you want to exit a program that will not let you switch consoles.
 (For example, X or a svgalib program.)
 
 re'B'oot is good when you're unable to shut down. But you should also 'S'ync
 and 'U'mount first.
 
 'C'rashdump can be used to manually trigger a crashdump when the system is hung.
 The kernel needs to have been built with CONFIG_KEXEC enabled.
 
 'S'ync is great when your system is locked up, it allows you to sync your
 disks and will certainly lessen the chance of data loss and fscking. Note
 that the sync hasn't taken place until you see the "OK" and "Done" appear
 on the screen. (If the kernel is really in strife, you may not ever get the
 OK or Done message...)
 
 'U'mount is basically useful in the same ways as 'S'ync. I generally 'S'ync,
 'U'mount, then re'B'oot when my system locks. It's saved me many a fsck.
 Again, the unmount (remount read-only) hasn't taken place until you see the
 "OK" and "Done" message appear on the screen.
 
 The loglevels '0'-'9' are useful when your console is being flooded with
 kernel messages you do not want to see. Selecting '0' will prevent all but
 the most urgent kernel messages from reaching your console. (They will
 still be logged if syslogd/klogd are alive, though.)
 
 t'E'rm and k'I'll are useful if you have some sort of runaway process you
 are unable to kill any other way, especially if it's spawning other
 processes.
 
 *  Sometimes SysRq seems to get 'stuck' after using it, what can I do?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 That happens to me, also. I've found that tapping shift, alt, and control
 on both sides of the keyboard, and hitting an invalid sysrq sequence again
 will fix the problem. (i.e., something like alt-sysrq-z). Switching to another
 virtual console (ALT+Fn) and then back again should also help.
 
 *  I hit SysRq, but nothing seems to happen, what's wrong?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 There are some keyboards that send different scancodes for SysRq than the
 pre-defined 0x54. So if SysRq doesn't work out of the box for a certain
 keyboard, run 'showkey -s' to find out the proper scancode sequence. Then
 use 'setkeycodes  84' to define this sequence to the usual SysRq
 code (84 is decimal for 0x54). It's probably best to put this command in a
 boot script. Oh, and by the way, you exit 'showkey' by not typing anything
 for ten seconds.
 
 *  I want to add SysRQ key events to a module, how does it work?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 In order to register a basic function with the table, you must first include
 the header 'include/linux/sysrq.h', this will define everything else you need.
 Next, you must create a sysrq_key_op struct, and populate it with A) the key
 handler function you will use, B) a help_msg string, that will print when SysRQ
 prints help, and C) an action_msg string, that will print right before your
 handler is called. Your handler must conform to the prototype in 'sysrq.h'.
 
 After the sysrq_key_op is created, you can call the kernel function
 register_sysrq_key(int key, struct sysrq_key_op *op_p); this will
 register the operation pointed to by 'op_p' at table key 'key',
 if that slot in the table is blank. At module unload time, you must call
 the function unregister_sysrq_key(int key, struct sysrq_key_op *op_p), which
 will remove the key op pointed to by 'op_p' from the key 'key', if and only if
 it is currently registered in that slot. This is in case the slot has been
 overwritten since you registered it.
 
 The Magic SysRQ system works by registering key operations against a key op
 lookup table, which is defined in 'drivers/char/sysrq.c'. This key table has
 a number of operations registered into it at compile time, but is mutable,
 and 2 functions are exported for interface to it:
  register_sysrq_key and unregister_sysrq_key.
 Of course, never ever leave an invalid pointer in the table. I.e., when
 your module that called register_sysrq_key() exits, it must call
 unregister_sysrq_key() to clean up the sysrq key table entry that it used.
 Null pointers in the table are always safe. :)
 
 If for some reason you feel the need to call the handle_sysrq function from
 within a function called by handle_sysrq, you must be aware that you are in
 a lock (you are also in an interrupt handler, which means don't sleep!), so
 you must call __handle_sysrq_nolock instead.
 
 *  When I hit a SysRq key combination only the header appears on the console?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Sysrq output is subject to the same console loglevel control as all
 other console output.  This means that if the kernel was booted 'quiet'
 as is common on distro kernels the output may not appear on the actual
 console, even though it will appear in the dmesg buffer, and be accessible
 via the dmesg command and to the consumers of /proc/kmsg.  As a specific
 exception the header line from the sysrq command is passed to all console
 consumers as if the current loglevel was maximum.  If only the header
 is emitted it is almost certain that the kernel loglevel is too low.
 Should you require the output on the console channel then you will need
 to temporarily up the console loglevel using alt-sysrq-8 or:
 
     echo 8 > /proc/sysrq-trigger
 
 Remember to return the loglevel to normal after triggering the sysrq
 command you are interested in.
 
 *  I have more questions, who can I ask?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 And I'll answer any questions about the registration system you got, also
 responding as soon as possible.
  -Crutcher
 
 *  Credits
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Written by Mydraal 
 Updated by Adam Sulmicki 
 Updated by Jeremy M. Dolan  2001/01/28 10:15:59
 Added to by Crutcher Dunnavant 

Downloading an Entire Web Site with wget

If you ever need to download an entire Web site, perhaps for off-line viewing, wget can do the
job—for example:

$ wget \
  --recursive \
  --no-clobber \
  --page-requisites \
  --html-extension \
  --convert-links \
  --restrict-file-names=windows \
  --domains website.org \
  --no-parent \
     http://centos.yubis.org/5.3/  

This command downloads the Web site http://centos.yubis.org/5.3/

The options are:

• --recursive: download the entire Web site.

• --domains website.org: don't follow links outside website.org.

• --no-parent: don't follow links outside the directory tutorials/html/.

• --page-requisites: get all the elements that compose the page (images, CSS and so on).

• --html-extension: save files with the .html extension.

• --convert-links: convert links so that they work locally, off-line.

• --restrict-file-names=windows: modify filenames so that they will work in Windows as well.

• --no-clobber: don't overwrite any existing files (used in case the download is interrupted and
  resumed).