BASH is a free software UNIX shell written for the GNU Project. Its name is an acronym which stands for Bourne-again shell. The name is a pun on the name of the Bourne shell (sh), an early and important UNIX shell written by Stephen Bourne and distributed with Version 7 UNIX circa 1978, and the concept of being born again. BASH was created in 1987 by Brian Fox. In 1990 Chet Ramey became the primary maintainer. BASH is the default shell on most GNU/Linux systems as well as on Mac OS X and it can be run on most UNIX-like operating systems. It has also been ported to Microsoft Windows using the POSIX emulation provided by Cygwin, to MS-DOS by the DJGPP project and to Novell NetWare.
AWK is a general purpose programming language that is designed for processing text-based data, either in files or data streams, and was created at Bell Labs in the 1970s. The name AWK is derived from the family names of its authors — Alfred Aho, Peter Weinberger, and Brian Kernighan; however, it is not commonly pronounced as a string of separate letters but rather to sound the same as the name of the bird, auk. awk, when written in all lowercase letters, refers to the UNIX or Plan 9 program that runs other programs written in the AWK programming language. AWK is an example of a programming language that extensively uses the string data type, associative arrays (that is, arrays indexed by key strings), and regular expressions. The power, terseness, and limitations of AWK programs and sed scripts inspired Larry Wall to write PERL. Because of their dense notation, all these languages are often used for writing one-liner programs. AWK is one of the early tools to appear in Version 7 UNIX and gained popularity as a way to add computational features to a UNIX pipeline. A version of the AWK language is a standard feature of nearly every modern UNIX-like operating system.
What is ROCKS?
Rocks Cluster Distribution (originally called NPACI Rocks) is a Linux distribution intended for high-performance computing clusters. It was started by NPACI and the SDSC in 2000, and was initially funded in part by an NSF grant (2000-2007) but is currently funded by the followup NSF grant. Rocks was initially based on the Red Hat Linux distribution, however modern versions of Rocks are now based on CentOS, with a modified Anaconda installer that simplifies mass installation onto many computers. Rocks includes many tools (such as MPI) which are not part of CentOS but are integral components that make a group of computers into a cluster. Installations can be customized with additional software packages at install-time by using special user-supplied CDs (called Roll CDs). The Rolls extend the system by integrating seamlessly and automatically into the management and packaging mechanisms used by base software, greatly simplifying installation and configuration of large numbers of computers. Over a dozen Rolls have been created, including the SGE roll, the Condor roll, the Lustre roll, the Java roll, and the ganglia roll.
General Wrapper for qstat
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | #! /bin/bash # Save this as /home/local/bin/qstat # A wrapper for 'qstat' command to show all currently submitted jobs along # with the total number processors used/in use # # Check the path to 'qstat' # # Thu, 02 Oct 2008 21:06:05 -0400 # Modify this variable to meet the cluster's requirements export PROC_TOTAL="128" export PROC_INUSE=`/opt/gridengine/bin/lx26-amd64/qstat | \ awk '{print} (NR>2) {used_procs+=\$9} END {print used_procs}' | \ tail -1 | \ awk '{print $NF}'` export PROC_AVAIL=`expr $PROC_TOTAL - $PROC_INUSE` export JOBS_LIST=`/opt/gridengine/bin/lx26-amd64/qstat` echo '-----------------------------------------------------------------------------------------------------------------' echo "$JOBS_LIST" echo '-----------------------------------------------------------------------------------------------------------------' echo " $PROC_TOTAL total processors :: $PROC_INUSE are in use :: $PROC_AVAIL are available" echo '-----------------------------------------------------------------------------------------------------------------' echo |
User Specific Wrapper for qstat
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | #! /bin/bash # Save this as /home/local/bin/qu # A wrapper for 'qstat' command to show all currently submitted jobs by a given # user ($USER) along with the total number processors used/in use # If the $USER has no jobs, displays all currently submitted jobs. # # Check the path to 'qstat' # # Thu, 02 Oct 2008 21:35:04 -0400 # Modify this variable to meet the cluster's requirements export PROC_TOTAL="128" export PROC_INUSE=`/opt/gridengine/bin/lx26-amd64/qstat | \ awk '{print} (NR>2) {used_procs+=\$9} END {print used_procs}' | \ tail -1 | \ awk '{print $NF}'` export PROC_AVAIL=`expr $PROC_TOTAL - $PROC_INUSE` export JOBS_LIST=`/opt/gridengine/bin/lx26-amd64/qstat -u $USER` if [ "$JOBS_LIST" == "" ]; then export JOBS_LIST=`/opt/gridengine/bin/lx26-amd64/qstat` fi echo '-----------------------------------------------------------------------------------------------------------------' echo "$JOBS_LIST" echo '-----------------------------------------------------------------------------------------------------------------' echo " $PROC_TOTAL total processors :: $PROC_INUSE are in use :: $PROC_AVAIL are available" echo '-----------------------------------------------------------------------------------------------------------------' echo |
