Schedule Tasks on Linux Using Crontab

If you've got a website that's heavy on your web server, you might want to run some processes like generating thumbnails or enriching data in the background. This way it can not interfere with the user interface. Linux has a great program for this called cron. It allows tasks to be automatically run in the background at regular intervals. You could also use it to automatically create backups, synchronize files, schedule updates, and much more. Welcome to the wonderful world of crontab.

Crontab

The crontab (cron derives from chronos, Greek for time; tab stands for table) command, found in Unix and Unix-like operating systems, is used to schedule commands to be executed periodically. To see what crontabs are currently running on your system, you can open a terminal and run:

$ sudo crontab -l

To edit the list of cronjobs you can run:

$ sudo crontab -e

This wil open a the default editor (could be vi or pico, if you want you can change the default editor) to let us manipulate the crontab. If you save and exit the editor, all your cronjobs are saved into crontab. Cronjobs are written in the following format:

* * * * * /bin/execute/this/script.sh

Scheduling explained

As you can see there are 5 stars. The stars represent different date parts in the following order:

• minute (from 0 to 59)
• hour (from 0 to 23)
• day of month (from 1 to 31)
• month (from 1 to 12)
• day of week (from 0 to 6) (0=Sunday)

Execute every minute

If you leave the star, or asterisk, it means every. Maybe that's a bit unclear. Let's use the the previous example again:

* * * * * /bin/execute/this/script.sh

They are all still asterisks! So this means execute /bin/execute/this/script.sh:

• every minute
• of every hour
• of every day of the month
• of every month
• and every day in the week.

In short: This script is being executed every minute. Without exception.

Execute every Friday 1AM

So if we want to schedule the script to run at 1AM every Friday, we would need the following cronjob:

0 1 * * 5 /bin/execute/this/script.sh

Get it? The script is now being executed when the system clock hits:

• minute: 0
• of hour: 1
• of day of month: * (every day of month)
• of month: * (every month)
• and weekday: 5 (=Friday)

Execute on workdays 1AM

So if we want to schedule the script to Monday till Friday at 1 AM, we would need the following cronjob:

0 1 * * 1-5 /bin/execute/this/script.sh

Get it? The script is now being executed when the system clock hits:

• minute: 0
• of hour: 1
• of day of month: * (every day of month)
• of month: * (every month)
• and weekday: 1-5 (=Monday til Friday)

Execute 10 past after every hour on the 1st of every month

Here's another one, just for practicing

10 * 1 * * /bin/execute/this/script.sh

Fair enough, it takes some getting used to, but it offers great flexibility.

Neat scheduling tricks

What if you'd want to run something every 10 minutes? Well you could do this:

0,10,20,30,40,50 * * * * /bin/execute/this/script.sh

But crontab allows you to do this as well:

*/10 * * * * /bin/execute/this/script.sh

Which will do exactly the same. Can you do the the math? ; )

Special words

For the first (minute) field, you can also put in a keyword instead of a number:

@reboot     Run once, at startup
@yearly     Run once  a year     "0 0 1 1 *"
@annually   (same as  @yearly)
@monthly    Run once  a month    "0 0 1 * *"
@weekly     Run once  a week     "0 0 * * 0"
@daily      Run once  a day      "0 0 * * *"
@midnight   (same as  @daily)
@hourly     Run once  an hour    "0 * * * *"

Leaving the rest of the fields empty, this would be valid:

@daily /bin/execute/this/script.sh

Storing the crontab output

By default cron saves the output of /bin/execute/this/script.sh in the user's mailbox (root in this case). But it's prettier if the output is saved in a separate logfile. Here's how:

*/10 * * * * /bin/execute/this/script.sh >> /var/log/script_output.log 2>&1

Explained

Linux can report on different levels. There's standard output (STDOUT) and standard errors (STDERR). STDOUT is marked 1, STDERR is marked 2. So the following statement tells Linux to store STDERR in STDOUT as well, creating one datastream for messages & errors:

2>&1

Now that we have 1 output stream, we can pour it into a file. Where > will overwrite the file, >> will append to the file. In this case we'd like to to append:

>> /var/log/script_output.log

Mailing the crontab output

By default cron saves the output in the user's mailbox (root in this case) on the local system. But you can also configure crontab to forward all output to a real email address by starting your crontab with the following line:

MAILTO="yourname@yourdomain.com"

Mailing the crontab output of just one cronjob

If you'd rather receive only one cronjob's output in your mail, make sure this package is installed:

$ aptitude install mailx

And change the cronjob like this:

*/10 * * * * /bin/execute/this/script.sh 2>&1 | mail -s "Cronjob ouput" yourname@yourdomain.com

Trashing the crontab output

Now that's easy:

*/10 * * * * /bin/execute/this/script.sh > /dev/null 2>&1

Just pipe all the output to the null device, also known as the black hole. On Unix-like operating systems, /dev/null is a special file that discards all data written to it.

Caveats

Many scripts are tested in a Bash environment with the PATH variable set. This way it's possible your scripts work in your shell, but when run from cron (where the PATH variable is different), the script cannot find referenced executables, and fails.
It's not the job of the script to set PATH, it's the responsibility of the caller, so it can help to echo $PATH, and put PATH=<the result> at the top of your cron files (right below MAILTO).

Quorum Explanation For AIX Servers

Abstract:-
Hence, we don’t have 100% mirror copy or no mirror copy available in our virtual servers. The same scenario in normal AIX hosts too. Due to this, one disk failure also leading the volume group locks. The impact is huge incase of virtual servers because one disk has been sliced, and it has been shared to more than one clinet, in order to optimize the RAID controller.
So that, we are going to disable the Quorum, after disabling the Quorum even one physical volume failure will not lock the volume group, and it ensures still the volume groups will be stands on.
Quorum Explanation:-
Quorum is a check point in LVM which ensures VGDA voting and resolving data conflict issue to prevent volume group data corruption. LVM always refers most recent data to run LVM commands.
The QUORUM concept that applies only to the varied on Volume Groups in order to force varyoff of the Volume Group should it lose more than 51% of its disks.
If QUORUM is disabled in a volume group even losing a one disk will not cause a volume group to be unavailable or varyoff. Otherwise, QUROUM enabled volume group always checks VGDA voting in-case any deviation on it or less than 51% of VGDA which forces to varyoff a volume group and data became inaccessible.
Problem Statement:-
In order to vary on the volume group successfully we need to have all the physical volumes in available state, otherwise we have the use the force option to vary on the volume group. It may lead the data lose or inconsistency of the volume group.
Without proper Quorum, if try to vary on the volume group the following error message will be thrown,

To make the volume group available we have to do number of work around in case of where Quorum has been enabled and any one of the disk fails.,
Proactive Action:-
We have to disable the quorum manually to avoid the Quorum lose,
1) Categories the disks, disks are belongs to storage or internal.
#lsdev –Cc disk | grep –i “SAS” ( To grep the SAS disks)
i.e

#lsdev –Cc disk | grep –i “SCSI” ( To grep the SCSI disks)
i.e

2) Check this pv’s belongs to which volume
i.e
for i in ` lsdev -Cc disk | grep -i “SAS” | awk ‘{print $1}’`
do
lspv | grep -w $i
done

The same need to repeat for SCSI disks or other available disks (If the server belongs to virtual clinet you have use the variable as virtual i.e lsdev –Cc disk | grep –i virtual.
Here, we have to check the Quorum for active volume groups only

The quorum is enabled on above snap shot ; here we are going disable the same
i.e

Note: - The same procedure needs to follow for the AIX hosts also, we don’t have SAN Boot, so the clinet disks belong to any one of these SAS or SCSI or Virtual.
The MPIO disks also fall in to the Virtual disk category.

IBM AIX Interview Questions

What is IBM AIX ?

AIX(Advanced Interactive eXecutive) is a series of proprietary Unix operating systems developed and sold by IBM for several of its computer platforms. Originally released for the IBM 6150 RISC workstation, AIX now supports or has supported a wide variety of hardware platforms, including the IBM RS/6000 series and later IBM POWER and PowerPC-based systems, IBM System i, System/370 mainframes, PS/2 personal computers, and the Apple Network Server. AIX is based on UNIX System V with 4.3BSD-compatible extensions. It is one of four commercial operating systems that are presently certified to The Open Group’s UNIX 03 standard (The others are Mac OS X, Solaris and HP-UX.)

What is System Management Interface Tool (SMIT) ?

SMIT is the System Management Interface Tool for AIX. It allows a user to navigate a menu hierarchy of commands, rather than using the command line. Invocation is typically achieved with the command smit. Experienced system administrators make use of the F6 function key which generates the command line that SMIT will invoke to complete the proposed task. SMIT also generates a log of commands that are performed in the smit.script file. The smit.script file automatically records the commands with the command flags and parameters used. The smit.script file can be used as an executable shell script to rerun system configuration tasks. SMIT also creates the smit.log file, which contains additional detailed information that can be used by programmers in extending the SMIT system.

What are the unique features of AIX ?

The following are some unique system management features of AIX.

• Logical Volume Manager :- The Logical Volume Manager (LVM) maintains the hierarchy of logical structures that manage disk storage.
• System Resource Controller:– The System Resource Controller (SRC) provides a set of commands and subroutines for creating and controlling subsystems and is designed to minimize the need for human intervention in system processing.
• Object Data Manager :- The Object Data Manager (ODM) is a data manager intended for the storage of system data.
• Software Vital Product Data:- Certain information about software products and their installable options is maintained in the Software Vital Product Data (SWVPD) database.

Workload management :- Workload Manager (WLM) is designed to provide the system administrator with increased control over how the scheduler virtual memory manager (VMM) and the disk I/O subsystem allocate resources to processes. You can use WLM to prevent different classes of jobs from interfering with each other and to allocate resources based on the requirements of different groups of users.

Operating system updates:- The operating system package is divided into filesets, where each fileset contains a group of logically related customer deliverable files. Each fileset can be individually installed and updated.

What is VGDA?

Volume Group Descriptor Area. It contains the information about volume group to which the physical volume belongs to. It also contains the information about physical partitions, logical partitions and logical volumes.

How do I know if my volume group is normal, big, or scalable?

Run the lsvg command on the volume group and look at the value for MAX PVs. The value is 32 for normal, 128 for big and 1,024 for scalable volume group.

What does Logical Volume Manager(LVM) mean?

The set of operating system commands, library subroutines and other tools that allow you to establish and control logical volume storage is called the Logical Volume Manager (LVM).

How do I create a volume group?

Use the following command, where s partition_size sets the number of megabytes (MB) in each physical partition where the partition_size is expressed in units of MB from one through 1,024. (It’s one through 131,072 for AIX 5.3.)

mkvg -y name_of_volume_group -s partition_size list_of_hard_disks

The partition_size variable must be equal to a power of two (for example: one, two, four or eight). The default value for standard and big volume groups is the lowest value to remain within the limitation of 1,016 physical partitions per physical volume. The default value for scalable volume groups is the lowest value to accommodate 2,040 physical partitions per physical volume.

How can I change the characteristics of a volume group?

You use the chvg command to change the characteristics of a volume group.

What is the login process to the AIX system?

AIX is a multi-user system

When the user login he needs to provide the user name and password for authentication purposes.

After the successful login, AIX starts the SHELL program

How do I create a logical volume?

#mklv -y name_of_logical_volume name_of_volume_group number_of_partition

How do I increase the size of a logical volume?

To increase the size of the logical volume represented by the lv05 directory by three logical partitions, for example, type extendlv lv05 3.

How do I display all logical volumes that are part of a volume group (e,g, rootvg)?

You can display all logical volumes that are part of rootvg by typing the following command: #lsvg -l rootvg/

How do I list information about  logical volumes?

Run the following command to display information about the logical volume #lv1: lslv lv1.

How can I clone the rootvg?

You can run the alt_disk_copy command to copy the current rootvg to an alternate disk. The following example shows how to clone the rootvg to hdisk1: alt_disk_copy -d hdisk1.]

How do I replace a disk?

extendvg VolumeGroupName hdisk_new

migratepv hdisk_bad hdisk_new

reducevg -d VolumeGroupName hdisk_bad

How do I mirror a logical volume? 

mklvcopy LogicalVolumeName Numberofcopies
syncvg VolumeGroupName

How can I display or set values for network parameters?

The no command sets or displays current or next boot values for network tuning parameters

How do I get the IP address of my machine?

Type one of the following: ifconfig -a or host Fully_Qualified_Host_Name. For example, host cyclop.austin.ibm.com.

How do I identify the network interfaces on my server?

Either of the following two commands will display the network interfaces: lsdev -Cc if or ifconfig -a. To get information about one specific network interface, for eaxample tr0, run the command ifconfig tr0.

How do I activate a network interface?

To activate the network interface tr0, run the command ifconfig tr0 up.

How do I display routing table, interface and protocol information?

To display routing table information for an Internet interface, type netstat -r -f inet. To display interface information for an Internet interface, type netstat -i -f inet. To display statistics for each protocol, type netstat -s -f inet.

How will you create a file if a disk is given to you?

#mkvg –y datavg –s 128 hdisk1 (pv name)

#mklv -y datalv –t jfs2 datavg 1

#crfs –v jfs2 –d datalv –g datavg –a size=10M –m /fs1

Difference between JFS & JFS2?

JFS:

JFS is normal filesystem. We cannot create large files in JFS.

Files can be access dynamically.

Max file system size=1TB

Max file size=64GB

Inode size=128b

JFS2:-

Can create a large size filesystems.

Files can be access accordingly as required. (not randomly or sequentially)

Max files system size=4PB

Max file size=4PB

Inode size=512

How will you find the inode number?

# ls –li

# istat /etc/passwd

How will you find LTG size?

#lsvg vgname

#lquerypv –M hdiskname

How will you change LTG size?

#rmdev -l hdiskx

#chdev -l hdiskx -a max_transfer=0*80000

#mkdev -l hdiskx

How will you find whether a quorum is up or not?

# lsvg vg name

Get through the interview bar with our selected interview questions for IBM AIX Training enthusiasts

How to copy a LV from one VG to another?

# cplv -v <new vg name> <sourcelvname>

Mirroring concept?

Check disk size belong to vg , take two same size disk to mirror
# bootinfo –s hdisk2
#lsvg –p rootvg
Add one more disk of same size of already have disk having rootvg.
# extendvg rootvg hdisk2
# lsvg –p rootvg
Now mirror vg with background sync LV. It sync LVs in background with new LVs
# mirrorvg –s rootvg
Create a boot image
# bosboot –ad /dev/hdisk2
#bosboot –ad /dev/hdisk1
Add a blv
record of newly added disk, first check and then add.
# bootlist –m normal -o

#bootlist –m normal hdisk2 hdisk1

How will you unmirror a VG if a PV gets failed?

First check the blv record using,
# bootlist –m normal –o
Remove the old blv record, using
# chpv –c hdisk2
# bootlist –m normal hdisk1
# bootlist –m normal hdisk2
Now unmirrorvg
# unmirrorvg rootvg hdisk2
# reducevg rootvg hdisk2
#lsvg –p rootvg
# lspv
# bootlist –m normal –o

How will you change a PP size?

PP size can be changed using a T-factor. If not then add a partitions in VG by adding one disk.

# chvg –t3 rootvg

T – Factor?

T – factor means that it will change the pp size to increase the efficiency of vg

#chvg –t16 datavg

Cal-if suppose that datavg pp size is 1016 we can take t factor is 16 then it comes to 2048 and pv contains in vg is 2.

What is the default PP Size in AIX?

128MB

How will you rename a VG?

Unmount all filesystems in VG.

# umount /test (mount point name)

#varyoffvg datavg

#exportvg datavg0

Importvg with new name

# importvg –y newvg hdisk4

 What is major number?

A major number refers to a type of device. A device number is a major number, and a minor number specifies a particular device of that type or sometimes the operation mode of that device type.

 What will you do if a disk gets failed?

diag command

How to enable & disable quorum?

# chvg –vy pv_name

# chvg –vn pv_name

#chvg –Qn vgname

#chvg –Qs vgname

What are the types of error?

-H/W error

-S/W error

-Operator error

-Unknown error.

What are the error levels severity?

-permanent

-Temporary

-Informational

-Unknown.

What is the command to see the error log?

# errpt –a
# errpt –d H [H/W]
# errpt –d S [S/W]

What is the command to clear the errorlog?

# errclear

What is the command to increase the size of error log when it is ful?

#usr/lib/errdemon –s ____ [ size ]

What is the command to start , stop, restart and list a demon?

# startsrc –g [ Demon name ]

# stopsrc [Demon name ]

# refrest [Demon name ]

# lssrc –al

When does error demon start in boot process?

rc.boot phase 3

Which file that error daemon stay?

/usr/lib/errdemon

Two states of device availability and explain them?

Available state and defined state. When device is in defined state it means it cannot use. Not in ready state. Run cfgmgr try to make available known in the system that device. Available state means that device is ready to use

# lsdev –Cc disk

It displays which disks are available

How will you check whether a device is installed or not?

# lsdev –Cc disk

How to change a 64 bit kernel to 32 bit kernel?

-Modify the /usr/lib/boot/unix directory and the /unix directory to be   a symbolic link to the binary for the desired kernel.

-Run the bosboot command to write a new system boot image.

-Reboot the system.

The path name of the 64-bit kernel is /usr/lib/boot/unix_64, and the
path name of the multiprocessor versions of the 32-bit kernel is
/usr/lib/boot/unix_mp.
# ln -sf /usr/lib/boot/unix_mp /unix
# ln -sf /usr/lib/boot/unix_mp /usr/lib/boot/unix
# lslv -m hd5
# bosboot -ad /dev/ipldevice
# shutdown -Fr
# bootinfo -K (should now be 32)

How to change a 32 bit kernel to 64 bit kernel?

To truly change the kernel to 64-bit from 32-bit, the system must be at the AIX® 5.1 or AIX 5.2levels.

# bootinfo –y

It shows a kernel is either 32-bit or 64-bit.

To truly change the kernel to 64-bit from 32-bit, the system must be at the AIX® 5.1 or AIX 5.2levels.

# bootinfo –y

It shows a kernel is either 32-bit or 64-bit.

To change to a 64-bit kernel, enter the following commands:

# ln -sf /usr/lib/boot/unix_64 /unix

# ln -sf /usr/lib/boot/unix_64 /usr/lib/boot/unix

# lslv -m hd5

# bosboot -ad /dev/ipldevice

# shutdown -Fr

How many VGs can be created in a LVM?

255 any no. of vg can create.

Types of VG & explain about them?

Normal vg -32 pv, 256 lv

# mkvg –y normvg –s 128 hdisk4

Big vg :- 128pv,512lv

# mkvg -B –y bigvg –s 128 hdisk9

Scalable vg: -1024pv, 4096 lv

# mkvg –S –y scalvg –s  512 hdisk1

How to convert a normal VG to big & scalable VG and also Big to Scalable VG?

#chvg –B vgname      /big vg

# chvg –S/-G vgname  /scale vg

What is VGDA, VGSA?

VGDA: Volume Group Descriptor Area. It contains the information about volume group to which the physical volume belongs to. It also contains the information about physical partitions, logical partitions and logical volumes.

VGSA: Volume Group Status Area it contains the information about physical partition from all physical volumes of the same volume group. It contains the status of VG

Default dump device?

hd6

Default secondary dump device?

/dev/sysdumpdevnull

How find a system dump?

#sysdumpdev -L

Where the file does the paging space stay?

/etc/swapspaces

How will check whether a fix is installed or not?

# instfix –ik  ptfname

What are the two states of installation?

Applied and commit state

In applied state if want to do any changes on installed software then we can do

But in commit the changes are not allowed to do. Once did installation.

A committed fileset update cannot be rejected.

Output from the installp -s command, which is used to get a list of applied software fileset     updates and updates that are available to be either committed or rejected.

How will you reject a committed fileset?

# installp –r –g fileset name

How will you check whether service pack is installed or not?

# oslevel –s

What is the latest OS level?

#oslevel -rq

How will you check whether software is installed or not?

# lslpp -L |grep -i   (software_name)

How do you find history of a file set?

#lslpp –h

How to remove a failed package installation?

#installp -C

How do remove the entries of the device in the odm?

#rmdev -dl

What are the types of ODM Database ?

Pre-defined database PdDv, PdAt
Customised database CuDv, CuAt

ODM Commands.

odmcreate: To customize Odm

To remove ODM: odmdrop

odmdelete: To remove object from ODM

To see info in ODM  odmget

To change the fields in object class  odmchange

To add object to ODM  odmadd

Types of Installation?

New and complete overwrite

Migration

Preservation

Where does log files resides?

AIX logs messages as specified in /etc/syslog.conf file.

How to unlock user account?

chuser –a account=true username

How to lock a user’s account?

#chuser account_lock=true username

How do u take backup of file systems?

#backup -f /dev/rmt0

How will u restore a file from already take backup using backup?

#restore -xvqf /dev/rmt0

How will u append a file to a previous archive take by Tar?

#tar -rvf /dev/rmt0

What command to take backup other vg’s?

# savevg –if /dev/rmt0 /vgname

What command to restore backup of non – rootvg?

# restvg  –x –d –f /dev/rmt0