Creating a Swap File

To add a swap file:

1. Determine the size of the new swap file in megabytes and multiply by 1024 to determine the number of blocks. For example, the block size of a 64 MB swap file is 65536.

2. At a shell prompt as root, type the following command with count being equal to the desired block size:

   dd if=/dev/zero of=/swapfile bs=1024 count=65536
   

3. Setup the swap file with the command:

   mkswap /swapfile
   

4. To enable the swap file immediately but not automatically at boot time:

   swapon /swapfile
   

5. To enable it at boot time, edit /etc/fstab to include the following entry:

   /swapfile          swap            swap    defaults        0 0
   

   The next time the system boots, it enables the new swap file.

6. After adding the new swap file and enabling it, verify it is enabled by viewing the output of the command cat /proc/swaps or free.

Extending Swap on an LVM2 Logical Volume

To extend an LVM2 swap logical volume (assuming /dev/VolGroup00/LogVol01 is the volume you want to extend):

1. Disable swapping for the associated logical volume:

# swapoff -v /dev/VolGroup00/LogVol01

2. Resize the LVM2 logical volume by 256 MB:

# lvm lvresize /dev/VolGroup00/LogVol01 -L +256M

3. Format the new swap space:

# mkswap /dev/VolGroup00/LogVol01

4. Enable the extended logical volume:

# swapon -va

5. Test that the logical volume has been extended properly:

# cat /proc/swaps # free

How to Create a swap file in Linux partition

Linux divides its physical RAM (random access memory) into chucks of memory called pages. Swapping is the process whereby a page of memory is copied to the preconfigured space on the hard disk, called swap space, to free up that page of memory. The combined sizes of the physical memory and the swap space is the amount of virtual memory available.

Swapping is necessary for two important reasons. First, when the system requires more memory than is physically available, the kernel swaps out less used pages and gives memory to the current application (process) that needs the memory immediately. Second, a significant number of the pages used by an application during its startup phase may only be used for initialization and then never used again. The system can swap out those pages and free the memory for other applications or even for the disk cache.

Steps to create swap file

1. Use the "dd" command to create a file.

dd if=/dev/zero of=/aSwapFile bs=1024 count=65536

where dd: - is used to copy a specified number of bytes from an Input file (if) to an Output file (of).
Here the dd command copy null characters from the special file "/dev/zero" and copy it to the output file "aSwapFile" in the "/" directory. The "bs" specifies that the characters are read as BYTES. The "count" specifies the size of the bytes block that is to be created in the output file.

2. Use the "mkswap" command to set up the Linux swap area on the file created.

mkswap /aSwapFile

3. Use the "swapon" command to activate the swap file created.

swapon /aSwapFile

4. Edit the fstab "/etc/fstab" to enable the swap after a reboot.

vi /etc/fstab
(Add the following line to the fstab file.)
/aSwapFile swap swap defaults 0 0

Batch file renaming

Batch file renaming can be pretty handy as it saves loads of time when you have to rename thousands of files. There are many simple ways to do it though.

If you have files with same ending pattern it can be done as:

e.g. if you want all .htm to be renamed to .html

rename .htm .html *.htm

and On other Unixes, we'd have to do something like this to batch rename files:

for i in *.html
do
j=`echo $i | sed 's/.htm$/.html/'`
# or, in this simple case even just: j=$"i"l
mv $i $j
done

Though if we had bash or ksh, we could make that a little less cumbersome:

for i in *.htm
do
j=${i%.htm}.html
# or: j=${i}l
mv $i $j
done

The Linux "rename" isn't going to handle to more complex cases though. For example, I had to transfer mail files from one system to another recently. On the old system, each message would be named something like "1124993500.7359464636.e-smith". On the new system, they'd be "00000001.eml", with hexadecimal numbering going on up, so you'd get "00000009.eml" and the next message would be "0000000a.eml", and so on. There's no way for "rename" to do that, but a loop can:

for i in *
do
j=`printf "%0.9x.eml\n" $x`
mv $i $j
x=$((x+1))
done

And if you dont want to follow any pattern and simply add new extension to all the files in a directory:

for i in $(ls -lArt /path/to/directory |awk '{print $9}');do mv $i $i.txt; done

adding to it if files are placed recursively in directories:

for i in $(find /path/to/root -type f -print);do mv $i $i.txt; done

Using Foremost

Foremost Syntax

foremost [-h][-V][-d][-vqwQT][-b][-o

] [-t][-s][-i]

Available Options

-h Show a help screen and exit.
-V Show copyright information and exit.
-d Turn on indirect block detection, this works well for Unix file systems.
-T Time stamp the output directory so you don’t have to delete the output dir when running multiple times.
-v Enables verbose mode. This causes more information regarding the current state of the program to be dis-played on the screen, and is highly recommended.
-q Enables quick mode. In quick mode, only the start of each sector is searched for matching headers. That is,the header is searched only up to the length of the longest header. The rest of the sector, usually about 500 bytes, is ignored. This mode makes foremost run con- siderably faster, but it may cause you to miss files that are embedded in other files. For example, using quick mode you will not be able to find JPEG images embedded in Microsoft Word documents.

Quick mode should not be used when examining NTFS file systems. Because NTFS will store small files inside the Master File Table, these files will be missed during quick mode.

-Q Enables Quiet mode. Most error messages will be sup-pressed.
-w Enables write audit only mode. No files will be extracted.
-a Enables write all headers, perform no error detection in terms of corrupted files.
-b number Allows you to specify the block size used in foremost. This is relevant for file naming and quick searches. The default is 512. ie. foremost -b 1024 image.dd

-k number Allows you to specify the chunk size used in foremost.This can improve speed if you have enough RAM to fit the image in. It reduces the checking that occurs between chunks of the buffer. For example if you had > 500MB of RAM. ie. foremost -k 500 image.dd

-i file The file is used as the input file. If no input file is specified or the input file cannot be read then stdin is used.

-o directory Recovered files are written to the directory directory.

-c file Sets the configuration file to use. If none is speci-fied, the file “foremost.conf” from the current direc-tory is used, if that doesn’t exist then “/etc/fore-most.conf” is used. The format for the configuration file is described in the default configuration file included with this program. See the CONFIGURATION FILE section below for more information.

-s number Skips number blocks in the input file before beginning the search for headers. ie. foremost -s 512 -t jpeg -i /dev/hda1

Foremost examples

Search for jpeg format skipping the first 100 blocks

sudo foremost -s 100 -t jpg -i image.dd

Only generate an audit file, and print to the screen (verbose mode)

sudo foremost -av image.dd

Search all defined types

sudo foremost -t all -i image.dd

Search for gif and pdf

sudo foremost -t gif,pdf -i image.dd

Search for office documents and jpeg files in a Unix file sys-tem in verbose mode.

sudo foremost -v -t ole,jpeg -i image.dd

Run the default case

sudo foremost image.dd

image.dd means you need to enter your hardisk mount point i.e /dev/sda1 or /dev/sda2

Recover deleted files in Linux

Although there's no common "undelete" command for a Linux EXT3 file system, you can recover many types of accidentally erased files, including documents, graphics, and system files using the "Foremost" console application.

When you delete a file, the data is not really overwritten. The pointer in the filesystem to the file is simply removed so the disk area can be overwritten when necessary. The more the disk is written to after the file is deleted, the larger the chance it will be overwritten and become unrecoverable.

Foremost is a command line utility for finding and recovering deleted files based on their type. It was origionally developed for the US Air Force Office of Special Investigations. It can recover files from a number of filesystems, including fat, ext3 and NTFS. It can be installed and run from the live cd.

Foremost can recover files with the following extensions:
jpg, gif, png, bmp, avi ,exe, mpg, wav, riff, wmv, mov, pdf, ole, Excel, Access, doc, zip, XML, SXW, SXC, SXI, SX, rar, htm, cpp

For other file extensions we may need to edit /etc/foremost.conf which can be found in man page of Foremost (man foremost)

How to Install:

Enable the universe repository and install foremost:

sudo apt-get install foremost

Assuming the lost files are on a USB drive (sda), you need to create a writeable directory on another drive where you can put the recovered files

sudo mount /dev/sdb1 /recovery
sudo mkdir /recovery/foremost

And then run foremost:

sudo foremost -i /dev/sda -o /recovery/foremost

or for specific file format e.g. video (avi):

sudo foremost -t avi -i /dev/sda -o /recovery/foremost

The recovered files will then be owned by root. Change their ownership so that you can use them:

sudo chown -R youruser:yourgroup /recovery/foremost



Please note that there's no guarantee that foremost will succeed in recovering your files, but at least there's a chance.

Tools to access Linux Partitions from Windows

If you dual boot with Windows and Linux , and have data spread across different partitions on Linux and Windows, you should be really in for some issues.

It happens sometimes you need to access your files on Linux partitions from Windows, and you realize it isn’t possible easily. Not really, with these tools in hand - it’s very easy for you to access files on your Linux partitions from Windows.

• Explore2fs

Explore2fs is a GUI explorer tool for accessing ext2 and ext3 filesystems. It runs under all versions of Windows and can read almost any ext2 and ext3 filesystem.

Project Home Page :- http://www.chrysocome.net/explore2fs

Latest Version :- 1.07

Sample Screenshot

• DiskInternals Linux Reader

DiskInternals Linux Reader is a new easy way to do this. This program plays the role of a bridge between your Windows and Ext2/Ext3 Linux file systems. This easy-to-use tool runs under Windows and allows you to browse Ext2/Ext3 Linux file systems and extract files from there.

Project Home Page :- http://www.diskinternals.com/linux-reader/

Latest Version :- 1.0

Sample Screenshot

• Ext2 Installable File System for Windows

It provides Windows NT4.0/2000/XP/2003 with full access to Linux Ext2 volumes (read access and write access). This may be useful if you have installed both Windows and Linux as dual boot environment on your computer.

Project Home Page :- http://www.fs-driver.org/

Latest Version :- 1.10c

Sample Screenshot

• rfsd: ReiserDriver

ReiserDriver is an Installable File System Driver (IFSD), used to easily (and natively!) read ReiserFS disk partitions under Microsoft Windows(2K/XP) by allowing ReiserFS partitions to appear as additional disks to the Windows operating system.

Project Home Page :- http://sourceforge.net/projects/rfsd/

Latest Version :- 1.

Restricting Remote Logins to listed users

To restrict remote logins to specific users, do the following:

1. Create a file called /etc/remusers with the names of the users, that are allowed to perform remote logins. It can look like:

root
nixuser

2. Modify the /etc/profile and /etc/csh.login files by adding the code listed below.

Putting the following code in /etc/profile and /etc/csh.login will keep users not listed in the file /etc/remusers from being able to login from remote location or telnet session. Be sure carriage returns are not included in the script files when you add the below code to them or the scripts will not run correctly, giving strange errors. Carriage returns are many times accidently embedded when code is copied from Windows or DOS based machines to Linux based machines.

1. trap "" 2 3
2. if { $LOGNAME != "root" ]
3. then
4. if [ $TERM != "linux" ]
5. then
6. if [ -z `cat /etc/remusers |grep $LOGNAME` ]
7. then
8. echo " *************************************************** "
9. echo " * * "
10. echo " * Remote logins are not allowed on this system * "
11. echo " * Please use a terminal or see the administrator. * "
12. echo " * Press RETURN to exit. * "
13. echo " * * "
14. echo " *************************************************** "
15. echo
16. read
17. exit
18. fi
19. fi
20. fi
21. trap 2 3

Line 1 traps SIGINT and SIGQUIT, so users cannot abort the script. Line 2 is a safety, in case you change the /etc/profile before you create the /etc/remusers file. Line 4 only runs the script if the terminal is not local. The "linux" terminal type is used locally. You may need to change this to:

if [ $TERM == "vt100" ]

if you are using serial terminals also. As an alternate, add another if statement that excludes the serial terminal type inside the first if statement to exclude both serial terminals and local terminals. You can determine what terminal type is being used by looking at the value of the TERM variable with the env command after logging in from the terminal in question. Also there are various types of terminals that telnet clients may emulate, so, you will want to be sure not to allow any terminals that a telnet client can emulate.

Line 6 determines if the user who just logged in, $LOGNAME, is listed in the /etc/remusers file. Line 16 reads a line from the user, requiring them to press an end of line key such as RETURN. Line 17 causes the shell to exit.

Linux Configuration and Diagnostic Tools

System and Network Configuration linuxconf - A GUI interactive interface available on Redhat 6.0 or later which includes netconf configuration. netconf - A GUI interactive interface available on Redhat 6.0 and later. kbdconf - A Redhat Linux tool which configures the /etc/sysconfig/keyboard file which specifies the location of the keyboard map file. This is a GUI based tool. mouseconfig - A Redhat Linux tool used to configure the /etc/sysconfig.mouse file. This is a GUI tool. timeconfig - A Redhat Linux tool used to configure the /etc/sysconfig/clock file. This is a GUI tool used to set timezone and whether or not the clock is set to GMT time. kernelcfg - A Redhat kernel configuration utility to be started from X. stty - Used to configure and print the console devices. setterm - Set terminal attributes. vmstat - Report statistics on virtual memory. X Configuration XF86Setup - A newer X configuration program with a GUI interface which modifies the "/etc/X11/XF86Config" configuration file. xf86config - An older X configuration program with a text based interface. It also modifies the "/etc/X11/XF86Config" configuration file. Xconfigurator - The Redhat tool used during system setup to configure X. SuperProbe - A program that probes the video card to determine its type for use with setting up X. xvidtune - This program will test video modes on the fly without modification to your X configuration. Read the usr/X11R6/lib/X11/doc/VideoModes.doc file before running this program.

Library and kernel Dependency Management

Library management:

• ldd - Used to determine shared libraries used by binary files. Type "ldd /bin/ls" to see the shared libraries used by the "ls" command.
• ldconfig - Used to update links and cache for system use of the most recent runtime shared libraries.

Kernel Management:

• lsmod - List currently installed kernel modules.
• depmod - Creates a dependency file, "modules.dep" in the directory "/lib/modules/x.x.x", later used by modprobe to automatically load the relevant modules.
• insmod - Installs a loadable kernel module into the running kernel.
• rmmod - Unloads modules, Ex: rmmod ftape
• modprobe - Used to load a module or set of modules. Loads all modules specified in the file "modules.dep".

General Diagnostic

System resources

• free - Show system memory availability and usage
• df - Show the amount of disk free space on each mounted filesystem.
• du - Show disk usage
• lspci - List PCI devices
• pnpdump - Lists ISA PNP device resource information.
• vmstat - Reports virtual memory statistics.

Other:

• env - List the current environment variables.
• printenv - Print a copy of the environment.
• set - Shows how the environment is set up. This command can be very useful when debugging the environment.
• runlevel - List the current and previous runlevel.
• uname - Print system information. In my case, it prints "Linux".
• dmesg - Show the last kernel messages printed during the last boot.

Tools for working with processes

• accton - Turns process accounting on and off. Uses the file /var/log/pacct. To turn it on type "accton /var/log/pacct". Use the command with no arguments to turn it off.
• kill - Kill a process by number
• killall - Send a signal to a process by name
• lastcomm (1) - Display information about previous commands in reverse order. Works only if process accounting is on.
• nice - Set process priority of new processes.
• ps(1) - Used to report the status of one or more processes.
• pstree(1) - Display the tree of running processes.
• renice(8) - Can be used to change the process priority of a currently running process.
• sa(8) - Generates a summary of information about users' processes that are stored in the /var/log/pacct file.
• skill - Report process status.
• snice - Report process status.
• top - Displays the processes that are using the most CPU resources.

Unix / Linux shortcut keys

Shortcuts are designed to help shorten the time required to perform frequently used commands or actions. In the below sections I have listed keyboard shortcut keys that can be performed by pressing two or more keys at once. In addition to keyboard shortcut keys, I have also listed command line shortcut keys that can be typed in at the shell.

Please note that the below shortcut keys and command line shortcuts will not work on all variants of Unix and/or Linux.

Keyboard shortcut keys

• CTRL + B Moves the cursor backward one character.
• CTRL + C Cancels the currently running command.
• CTRL + D Logs out of the current session.
• CTRL + F Moves the cursor forward one character.
• CTRL + H Erase one character. Similar to pressing backspace.
• CTRL + P Paste previous line and/or lines.
• CTRL + S Stops all output on screen (XOFF).
• CTRL + Q Turns all output stopped on screen back on (XON).
• CTRL + U Erases the complete line.
• CTRL + W Deletes the last word typed in. For example, if you typed 'mv file1 file2' this shortcut would delete file2.
• CTRL + Z Cancels current operation, moves back a directory and/or takes the current operation and moves it to the background. See bg command for additional information about background.

Command line shortcuts

In addition to the below command line shortcuts, it is also helpful to use the alias command that allows you to specify a keyword for frequently used commands or mistakes.

• ~ Moves to the user's home directory.
• !! Repeats the line last entered at the shell. See history command for previous commands.
• !$ Repeats the last argument for the command last used. See history command for previous commands.
• reset Resets the terminal if terminal screen is not displaying correctly.
• shutdown -h now Remotely or locally shuts the system down.

Upgrading to Ubuntu 9.04 From 8.10 (Desktop)

Here is the tutorial for upgrading Ubuntu 8.10 (Intrepid Ibex) desktop to latest Ubuntu Linux 9.04 (Jaunty Jackalope) over the Internet.

Ubuntu Linux support direct upgrade Ubuntu 9.04 from Ubuntu 8.10.

Make sure you have all updates applied to Ubuntu 8.10 before you start upgrade. To do this visit:
System -> Administration -> Update Manager

Ubuntu Update Manager

Click on "Install Updates"

Network Upgrade for Ubuntu Desktops over the Network

You can easily upgrade over the network as follows:

Visit System > Administration > Update Manager

Click the Check button to check for new updates.

A message will appear informing you of the availability of the new release.

Click Upgrade.

Follow the on-screen instructions.

Upgrade Ubuntu Server 8.10 to 9.04

You can upgrade the server either from console or from network remotely.

Be sure that you have all updates applied to Ubuntu 8.10 server before you upgrade. Type the following command to apply updates:

$ sudo apt-get update
$ sudo apt-get upgrade

WARNING! Backup important data, sql and configuration file before server running the following upgrade commands. The following discussion only applies to the SERVER edition. For Ubuntu 9.04 desktop upgrade click here.

Next, install update-manager-core if it is not already installed:

$ sudo apt-get install update-manager-core

Finally, start the upgrade tool, enter:

$ sudo do-release-upgrade

Now, just follow the on-screen instructions to upgrade your server over ssh session.

Upgrade SuSE11.0 to SuSE11.1

Here are the three steps to do this with zypper (the command line package manager):

1. zypper update

Make sure your system is up to date

2. change repos

change all your repositories over to 11.1 recomend you just stick to oss non-oss and packman to avoid problems

3. upgrade

zypper dist-upgrade

Done, you should have now successfully upgraded your system to 11.1. You can check that with
cat /etc/SuSE-release

Analyze apache log with awk

Lets say we want to find the amount of times a specific ip address has hit your webserver,
on this example we are assuming your apache access_log is located in /usr/local/apache/logs

The full command would be:

awk '{print $1}' /usr/local/apache/logs/access.log | sort | uniq -c | sort -fr

and the output would be like this:

155 90.193.xx.xx
154 86.143.xx.xx
109 82.17.xx.xx
85 90.213.xx.xx
74 193.28.xx.xx

How to change the shell

The chsh command can be used to change your shell.

There are other options that may be used as well. To check the available shells on your system, use the following command:

chsh --list

which will produce a list similar to:

/bin/sh
/bin/bash
/sbin/nologin
/bin/ash
/bin/bsh
/bin/ksh
/usr/bin/ksh
/usr/bin/pdksh
/bin/tcsh
/bin/csh

To change the shell to csh, use the command:

chsh -s /bin/csh

To verify the current shell, use the command:

echo $SHELL

Note:

The above example is for the root user. Non-root users are prompted for their password before the shell is changed. These changes will take effect on the next login.

Tips for Cron

In addition to Crontab tutorial here is the post for tips on cron.

Use of operators

An operator allows you to specifying multiple values in a field. There are three operators:

1. The asterisk (*) : This operator specifies all possible values for a field. For example, an asterisk in the hour time field would be equivalent to every hour or an asterisk in the month field would be equivalent to every month.

2. The comma (,) : This operator specifies a list of values, for example: "1,5,10,15,20, 25".

3. The dash (-) : This operator specifies a range of values, for example: "5-15" days , which is equivalent to typing "5,6,7,8,9,....,13,14,15" using the comma operator.

Disabling Email output:

By default the output of a command or a script (if any produced), will be email to your local email account. To stop receiving email output from crontab you need to append >/dev/null 2>&1.

For example:

0 3 * * * /root/backup.sh >/dev/null 2>&1

To mail output to particluer email account let us say rdx@nixtrix.co.cc you need to define MAILTO variable to your cron job:

MAILTO="rdx@nixtrix.co.cc"
0 3 * * * /root/backup.sh >/dev/null 2>&1

Task:

To list your crontab jobs use the command type the following command:
# crontab -l

To remove or erase all crontab jobs use the command:
# crontab -r

Use special string to save time

Instead of the first five fields, you can use any one of eight special strings. It will not just save your time but it will improve readability.

Special string	Meaning
@reboot	Run once, at startup.
@yearly	Run once a year, "0 0 1 1 *".
@annually	(same as @yearly)
@monthly	Run once a month, "0 0 1 * *".
@weekly	Run once a week, "0 0 * * 0".
@daily	Run once a day, "0 0 * * *".
@midnight	(same as @daily)
@hourly	Run once an hour, "0 * * * *".

Run ntpdate every hour:

@hourly /path/to/ntpdate

Make a backup everyday:

@daily /path/to/backup/script.sh

Understanding /etc/crontab file and /etc/cron.d/* directories

/etc/crontab is system crontabs file. Usually only used by root user or daemons to configure system wide jobs. All individual user must must use crontab command to install and edit their jobs as described above. /var/spool/cron/ or /var/cron/tabs/ is directory for personal user crontab files. It must be backup with users home directory.

Typical /etc/crontab file entries:

SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
HOME=/

# run-parts
01 * * * * root run-parts /etc/cron.hourly
02 4 * * * root run-parts /etc/cron.daily
22 4 * * 0 root run-parts /etc/cron.weekly
42 4 1 * * root run-parts /etc/cron.monthly

Additionally, cron reads the files in /etc/cron.d/ directory. Usually system daemon such as sa-update or sysstat places their cronjob here. As a root user or superuser you can use following directories to configure cronjobs. You can directly drop your scripts here. run-parts command run scripts or programs in a directory via /etc/crontab

Directory	Description
/etc/cron.d/	Put all scripts here and call them from /etc/crontab file.
/etc/cron.daily/	Run all scripts once a day
/etc/cron.hourly/	Run all scripts once an hour
/etc/cron.monthly/	Run all scripts once a month
/etc/cron.weekly/	Run all scripts once a week

Here is a sample shell script (clean.cache) to clean up cached files every 10 days. This script is directly created at /etc/cron.daliy/ directory i.e. create a file called /etc/cron.daily/clean.cache:

#!/bin/bash
CROOT="/tmp/cachelighttpd/"
DAYS=10
LUSER="lighttpd"
LGROUP="lighttpd"

# start cleaning
/usr/bin/find ${CROOT} -type f -mtime +${DAYS} | xargs -r /bin/rm

# if directory deleted by some other script just get it back
if [ ! -d $CROOT ]
then
/bin/mkdir -p $CROOT
/bin/chown ${LUSER}:${LGROUP} ${CROOT}
fi