Friday, April 3, 2009

10 free virtualization tools

More and more IT Vendors are validating the benefits of desktop and server virtualization, from energy efficiency to better resource utilization. But if you’re still leery of forking over big dollars for the technology, there are some low-cost and no-cost ways to give virtualization a try.

Virtualization freebies are available both from the open source community and from vendors such as Microsoft and VMware. Here’s a sampling:

Microsoft Virtual Server 2005 R2

A virtualization environment for Windows Server 2003, Windows Server 2003 Service Pack 1, Windows XP Professional Edition and Windows XP Service Pack 2. It works on x86 servers or workstations and is available in 32- or 64-bit versions.

OpenVZ

Linux-based software that lets the IT administrator create secure virtual environments. Each virtual private server can be rebooted independently, co-existing with other virtual private servers.

The open source project is supported by SWsoft, which markets a commercial version called Virtuozzo.

Q

Open source software for running Windows or Linux on a Macintosh. It allows users to switch between guest PCs and restart guest PCs at any point. Q also enables users to exchange files between the host operating system and the guest.

Q is based on the QEMU open source CPU emulator. You need to be careful about using Q – according to the Web site, it is still alpha software.

QEMU

A generic and open source machine emulator and virtualization software developed by Fabrice Bellard, the author of a compact C compiler. QEMU can run operating systems and programs developed for one machine on a different machine.

When used as virtualization software, the host and guest machine must use x86-compatible processors. In emulation mode, it supports x86 PFCs, MIPS R4000, Sun SPARC and PowerPC processors.

Technical Architecture Solutions TCO Calculator

Technical Architecture Solutions, a disaster recovery and business continuity consultancy, is offering a TCO Calculator for server virtualization. It is Web-based and allows IT administrators to estimate how much money they will save by using VMware ESX. The calculator consists of a Microsoft Excel spreadsheet and is based on four virtual machines per virtual server.

VMware Server, Player and Converter

VMware Server is VMware’s starter kit for Windows and Linux server virtualization. It can host Windows, Linux, NetWare and Solaris as guest operating systems.

VMware Player lets IT administrators run virtual machine instances created by other VMware products. With it, you can download prebuilt virtual machines from the VMware Technology Network or share them with other organizations.

Virtual Iron Single Server Virtualization and Management

Virtual Iron’s Single Server Virtualization and Management software creates Linux or Windows virtual servers. It runs in 32- and 64-bit workloads that are stored on iSCSI, storage-area networks or local storage. Virtualized nodes created with Virtual Iron’s Single Server software must have Intel Virtualization Technology or AMD Virtualization hardware assist.

Virtual Iron and Provision Networks Virtual Desktop Infrastructure

An alternative to terminal services environments and fat-client desktop implementations. It lets an IT administrator host five individual Windows desktop PCs inside virtual machines running on servers in the data center.

Xen

This open source package for x86 server virtualization was developed by the University of Cambridge and commercialized by XenSource. Xen, which runs on Linux, supports Windows and Linux as guest operating systems.

XenSource XenExpress

XenExpress, from XenSource, is a free virtualization package for x86 servers that lets IT administrators host as many as four virtual machines on servers equipped with 4GB of physical RAM or two physical sockets.

It supports Windows Server 2003, Windows XP and Windows 2000 as guest operating systems. It also supports Red Hat, SUSE and Debian Linux guest operating systems.

XenExpress is based on the open source Xen hypervisor.

Rebooting the Magic Way

If you have ever had a hard drive fail on a remote server you may remember the feeling you had after trying to issue the following commands:

   
# reboot
bash: /sbin/reboot: Input/output error
# shutdown -r now
bash: /sbin/shutdown: Input/output error

Obviously, there is a problem with your drive. These commands are failing because the kernel is unable to load the /sbin/reboot and /sbin/shutdown binaries from the disk so that it can execute them.

A fsck on the next boot might be able to correct whatever is wrong with the disk, but first you need to get the system to reboot. If your machine is located at a managed hosting provider then you could submit a reboot ticket, but you'll have to wait for someone to take responsibility.

Wouldn't it be nice if there was a way to ask the kernel to reboot without needing to access the failing drive? Well, there is a way, and it is remarkably simple.

The "magic SysRq key" provides a way to send commands directly to the kernel through the /proc filesystem. It is enabled via a kernel compile time option, CONFIG_MAGIC_SYSRQ, which seems to be standard on most distributions. First you must activate the magic SysRq option:

   
echo 1 > /proc/sys/kernel/sysrq

When you are ready to reboot the machine simply run the following:

    
echo b > /proc/sysrq-trigger

This does not attempt to unmount or sync filesystems, so it should only be used when absolutely necessary, but if your drive is already failing then that may not be a concern.

In addition to rebooting the system the sysrq trick can be used to dump memory information to the console, sync all filesystems, remount all filesystems in read-only mode, send SIGTERM or SIGKILL to all processes except init, or power off the machine entirely, among other things.

Also, instead of echoing into /proc/sys/kernel/sysrq each time you can activate the magic SysRq key at system boot time using sysctl, where supported:

    
echo "kernel.sysrq = 1" >> /etc/sysctl.conf

If you would like to learn more about magic SysRq you can read the sysrq.txt file in the kernel documentation.

sysrq.txt

1 Linux Magic System Request Key Hacks
2 Documentation for sysrq.c
3
4 * What is the magic SysRq key?
5 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6 It is a 'magical' key combo you can hit which the kernel will respond to
7 regardless of whatever else it is doing, unless it is completely locked up.
8
9 * How do I enable the magic SysRq key?
10 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
11 You need to say "yes" to 'Magic SysRq key (CONFIG_MAGIC_SYSRQ)' when
12 configuring the kernel. When running a kernel with SysRq compiled in,
13 /proc/sys/kernel/sysrq controls the functions allowed to be invoked via
14 the SysRq key. By default the file contains 1 which means that every
15 possible SysRq request is allowed (in older versions SysRq was disabled
16 by default, and you were required to specifically enable it at run-time
17 but this is not the case any more). Here is the list of possible values
18 in /proc/sys/kernel/sysrq:
19 0 - disable sysrq completely
20 1 - enable all functions of sysrq
21 >1 - bitmask of allowed sysrq functions (see below for detailed function
22 description):
23 2 - enable control of console logging level
24 4 - enable control of keyboard (SAK, unraw)
25 8 - enable debugging dumps of processes etc.
26 16 - enable sync command
27 32 - enable remount read-only
28 64 - enable signalling of processes (term, kill, oom-kill)
29 128 - allow reboot/poweroff
30 256 - allow nicing of all RT tasks
31
32 You can set the value in the file by the following command:
33 echo "number" >/proc/sys/kernel/sysrq
34
35 Note that the value of /proc/sys/kernel/sysrq influences only the invocation
36 via a keyboard. Invocation of any operation via /proc/sysrq-trigger is always
37 allowed (by a user with admin privileges).
38
39 * How do I use the magic SysRq key?
40 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
41 On x86 - You press the key combo 'ALT-SysRq-'. Note - Some
42 keyboards may not have a key labeled 'SysRq'. The 'SysRq' key is
43 also known as the 'Print Screen' key. Also some keyboards cannot
44 handle so many keys being pressed at the same time, so you might
45 have better luck with "press Alt", "press SysRq", "release SysRq",
46 "press ", release everything.
47
48 On SPARC - You press 'ALT-STOP-', I believe.
49
50 On the serial console (PC style standard serial ports only) -
51 You send a BREAK, then within 5 seconds a command key. Sending
52 BREAK twice is interpreted as a normal BREAK.
53
54 On PowerPC - Press 'ALT - Print Screen (or F13) - ,
55 Print Screen (or F13) - may suffice.
56
57 On other - If you know of the key combos for other architectures, please
58 let me know so I can add them to this section.
59
60 On all - write a character to /proc/sysrq-trigger. e.g.:
61
62 echo t > /proc/sysrq-trigger
63
64 * What are the 'command' keys?
65 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
66 'b' - Will immediately reboot the system without syncing or unmounting
67 your disks.
68
69 'c' - Will perform a kexec reboot in order to take a crashdump.
70
71 'd' - Shows all locks that are held.
72
73 'e' - Send a SIGTERM to all processes, except for init.
74
75 'f' - Will call oom_kill to kill a memory hog process.
76
77 'g' - Used by kgdb on ppc and sh platforms.
78
79 'h' - Will display help (actually any other key than those listed
80 here will display help. but 'h' is easy to remember :-)
81
82 'i' - Send a SIGKILL to all processes, except for init.
83
84 'k' - Secure Access Key (SAK) Kills all programs on the current virtual
85 console. NOTE: See important comments below in SAK section.
86
87 'l' - Shows a stack backtrace for all active CPUs.
88
89 'm' - Will dump current memory info to your console.
90
91 'n' - Used to make RT tasks nice-able
92
93 'o' - Will shut your system off (if configured and supported).
94
95 'p' - Will dump the current registers and flags to your console.
96
97 'q' - Will dump per CPU lists of all armed hrtimers (but NOT regular
98 timer_list timers) and detailed information about all
99 clockevent devices.
100
101 'r' - Turns off keyboard raw mode and sets it to XLATE.
102
103 's' - Will attempt to sync all mounted filesystems.
104
105 't' - Will dump a list of current tasks and their information to your
106 console.
107
108 'u' - Will attempt to remount all mounted filesystems read-only.
109
110 'v' - Dumps Voyager SMP processor info to your console.
111
112 'w' - Dumps tasks that are in uninterruptable (blocked) state.
113
114 'x' - Used by xmon interface on ppc/powerpc platforms.
115
116 '0'-'9' - Sets the console log level, controlling which kernel messages
117 will be printed to your console. ('0', for example would make
118 it so that only emergency messages like PANICs or OOPSes would
119 make it to your console.)
120
121 * Okay, so what can I use them for?
122 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
123 Well, un'R'aw is very handy when your X server or a svgalib program crashes.
124
125 sa'K' (Secure Access Key) is useful when you want to be sure there is no
126 trojan program running at console which could grab your password
127 when you would try to login. It will kill all programs on given console,
128 thus letting you make sure that the login prompt you see is actually
129 the one from init, not some trojan program.
130 IMPORTANT: In its true form it is not a true SAK like the one in a :IMPORTANT
131 IMPORTANT: c2 compliant system, and it should not be mistaken as :IMPORTANT
132 IMPORTANT: such. :IMPORTANT
133 It seems others find it useful as (System Attention Key) which is
134 useful when you want to exit a program that will not let you switch consoles.
135 (For example, X or a svgalib program.)
136
137 re'B'oot is good when you're unable to shut down. But you should also 'S'ync
138 and 'U'mount first.
139
140 'C'rashdump can be used to manually trigger a crashdump when the system is hung.
141 The kernel needs to have been built with CONFIG_KEXEC enabled.
142
143 'S'ync is great when your system is locked up, it allows you to sync your
144 disks and will certainly lessen the chance of data loss and fscking. Note
145 that the sync hasn't taken place until you see the "OK" and "Done" appear
146 on the screen. (If the kernel is really in strife, you may not ever get the
147 OK or Done message...)
148
149 'U'mount is basically useful in the same ways as 'S'ync. I generally 'S'ync,
150 'U'mount, then re'B'oot when my system locks. It's saved me many a fsck.
151 Again, the unmount (remount read-only) hasn't taken place until you see the
152 "OK" and "Done" message appear on the screen.
153
154 The loglevels '0'-'9' are useful when your console is being flooded with
155 kernel messages you do not want to see. Selecting '0' will prevent all but
156 the most urgent kernel messages from reaching your console. (They will
157 still be logged if syslogd/klogd are alive, though.)
158
159 t'E'rm and k'I'll are useful if you have some sort of runaway process you
160 are unable to kill any other way, especially if it's spawning other
161 processes.
162
163 * Sometimes SysRq seems to get 'stuck' after using it, what can I do?
164 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
165 That happens to me, also. I've found that tapping shift, alt, and control
166 on both sides of the keyboard, and hitting an invalid sysrq sequence again
167 will fix the problem. (i.e., something like alt-sysrq-z). Switching to another
168 virtual console (ALT+Fn) and then back again should also help.
169
170 * I hit SysRq, but nothing seems to happen, what's wrong?
171 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
172 There are some keyboards that send different scancodes for SysRq than the
173 pre-defined 0x54. So if SysRq doesn't work out of the box for a certain
174 keyboard, run 'showkey -s' to find out the proper scancode sequence. Then
175 use 'setkeycodes 84' to define this sequence to the usual SysRq
176 code (84 is decimal for 0x54). It's probably best to put this command in a
177 boot script. Oh, and by the way, you exit 'showkey' by not typing anything
178 for ten seconds.
179
180 * I want to add SysRQ key events to a module, how does it work?
181 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
182 In order to register a basic function with the table, you must first include
183 the header 'include/linux/sysrq.h', this will define everything else you need.
184 Next, you must create a sysrq_key_op struct, and populate it with A) the key
185 handler function you will use, B) a help_msg string, that will print when SysRQ
186 prints help, and C) an action_msg string, that will print right before your
187 handler is called. Your handler must conform to the prototype in 'sysrq.h'.
188
189 After the sysrq_key_op is created, you can call the kernel function
190 register_sysrq_key(int key, struct sysrq_key_op *op_p); this will
191 register the operation pointed to by 'op_p' at table key 'key',
192 if that slot in the table is blank. At module unload time, you must call
193 the function unregister_sysrq_key(int key, struct sysrq_key_op *op_p), which
194 will remove the key op pointed to by 'op_p' from the key 'key', if and only if
195 it is currently registered in that slot. This is in case the slot has been
196 overwritten since you registered it.
197
198 The Magic SysRQ system works by registering key operations against a key op
199 lookup table, which is defined in 'drivers/char/sysrq.c'. This key table has
200 a number of operations registered into it at compile time, but is mutable,
201 and 2 functions are exported for interface to it:
202 register_sysrq_key and unregister_sysrq_key.
203 Of course, never ever leave an invalid pointer in the table. I.e., when
204 your module that called register_sysrq_key() exits, it must call
205 unregister_sysrq_key() to clean up the sysrq key table entry that it used.
206 Null pointers in the table are always safe. :)
207
208 If for some reason you feel the need to call the handle_sysrq function from
209 within a function called by handle_sysrq, you must be aware that you are in
210 a lock (you are also in an interrupt handler, which means don't sleep!), so
211 you must call __handle_sysrq_nolock instead.
212
213 * When I hit a SysRq key combination only the header appears on the console?
214 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
215 Sysrq output is subject to the same console loglevel control as all
216 other console output. This means that if the kernel was booted 'quiet'
217 as is common on distro kernels the output may not appear on the actual
218 console, even though it will appear in the dmesg buffer, and be accessible
219 via the dmesg command and to the consumers of /proc/kmsg. As a specific
220 exception the header line from the sysrq command is passed to all console
221 consumers as if the current loglevel was maximum. If only the header
222 is emitted it is almost certain that the kernel loglevel is too low.
223 Should you require the output on the console channel then you will need
224 to temporarily up the console loglevel using alt-sysrq-8 or:
225
226 echo 8 > /proc/sysrq-trigger
227
228 Remember to return the loglevel to normal after triggering the sysrq
229 command you are interested in.
230
231 * I have more questions, who can I ask?
232 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
233 And I'll answer any questions about the registration system you got, also
234 responding as soon as possible.
235 -Crutcher
236
237 * Credits
238 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
239 Written by Mydraal
240 Updated by Adam Sulmicki
241 Updated by Jeremy M. Dolan 2001/01/28 10:15:59
242 Added to by Crutcher Dunnavant

Downloading an Entire Web Site with wget

If you ever need to download an entire Web site, perhaps for off-line viewing, wget can do the
job—for example:

$ wget \
--recursive \
--no-clobber \
--page-requisites \
--html-extension \
--convert-links \
--restrict-file-names=windows \
--domains website.org \
--no-parent \
http://centos.yubis.org/5.3/

This command downloads the Web site http://centos.yubis.org/5.3/

The options are:

  • --recursive: download the entire Web site.

  • --domains website.org: don't follow links outside website.org.

  • --no-parent: don't follow links outside the directory tutorials/html/.

  • --page-requisites: get all the elements that compose the page (images, CSS and so on).

  • --html-extension: save files with the .html extension.

  • --convert-links: convert links so that they work locally, off-line.

  • --restrict-file-names=windows: modify filenames so that they will work in Windows as well.

  • --no-clobber: don't overwrite any existing files (used in case the download is interrupted and
    resumed).