What is ROCKS?
Rocks Cluster Distribution (originally called NPACI Rocks) is a Linux distribution intended for high-performance computing clusters. It was started by NPACI and the SDSC in 2000, and was initially funded in part by an NSF grant (2000-2007) but is currently funded by the followup NSF grant. Rocks was initially based on the Red Hat Linux distribution, however modern versions of Rocks are now based on CentOS, with a modified Anaconda installer that simplifies mass installation onto many computers. Rocks includes many tools (such as MPI) which are not part of CentOS but are integral components that make a group of computers into a cluster. Installations can be customized with additional software packages at install-time by using special user-supplied CDs (called Roll CDs). The Rolls extend the system by integrating seamlessly and automatically into the management and packaging mechanisms used by base software, greatly simplifying installation and configuration of large numbers of computers. Over a dozen Rolls have been created, including the SGE roll, the Condor roll, the Lustre roll, the Java roll, and the ganglia roll.
Why wrappers?
Functionality of qstat [part of the SGE Roll] changed in ROCKS 5.2.2 [it might have changed earlier than that, but that’s when I first noticed]. To keep cluster users, supervisors and systems-administrators happy, I ended up writing these two wrappers:
General Wrapper for qstat
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 | #! /bin/bash # A wrapper for 'qstat' command to show # the total number processors used/in use. # ROCKS 5.2.2 # Thu, 10 Dec 2009 12:27:59 -0500 export PROC_TOTAL="142" export USERS_LIST=" user0 user1 user2 user3 user4" export PROC_INUSE=`/opt/gridengine/bin/lx26-amd64/qstat -u $USERS_LIST | \ awk '{print} (NR>2) {used_procs+=\$9} END {print used_procs}' | \ tail -1 | awk '{print $NF}'` export PROC_AVAIL=`expr $PROC_TOTAL - $PROC_INUSE` echo '-----------------------------------------------------------------------------------------------------------------' /opt/gridengine/bin/lx26-amd64/qstat -u $USERS_LIST echo '-----------------------------------------------------------------------------------------------------------------' echo " $PROC_TOTAL total processors :: $PROC_INUSE is/are in use :: $PROC_AVAIL is/are available" echo '-----------------------------------------------------------------------------------------------------------------' echo |
User Specific Wrapper for qstat
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 | #! /bin/bash # A wrapper for 'qstat' command to show only the jobs # submitted by a given user but show the total number # processors used/in use. # ROCKS 5.2.2 # # Usage: # qu [USERNAME] # In the absence of any jobs submitted by USERNAME, or if # USERNAME is not specified at all, it just displays the jobs, # if any, by the currently logged in user. # # Thu, 10 Dec 2009 12:27:59 -0500 export EXPECTED_ARGS="1" export PROC_TOTAL="142" export USERS_LIST=" user0 user1 user2 user3 user4" if [ $# -ne $EXPECTED_ARGS ]; then export MYUSER="$USER" else export MYUSER="$1" fi export PROC_USER=`/opt/gridengine/bin/lx26-amd64/qstat -u $MYUSER | \ awk '{print} (NR>2) {used_procs+=\$9} END {print used_procs}' | \ tail -1 | awk '{print $NF}'` export PROC_INUSE=`/opt/gridengine/bin/lx26-amd64/qstat -u $USERS_LIST | \ awk '{print} (NR>2) {used_procs+=\$9} END {print used_procs}' | \ tail -1 | awk '{print $NF}'` export PROC_AVAIL=`expr $PROC_TOTAL - $PROC_INUSE` if [[ $PROC_USER -gt 0 ]] ; then echo '-----------------------------------------------------------------------------------------------------------------' /opt/gridengine/bin/lx26-amd64/qstat -u $MYUSER echo '-----------------------------------------------------------------------------------------------------------------' echo " $PROC_TOTAL total processors :: $PROC_INUSE is/are in use :: $PROC_USER is/are use by $MYUSER :: $PROC_AVAIL is/are available " echo '-----------------------------------------------------------------------------------------------------------------' echo else echo echo " `hostname` @ `date -R`" echo echo " $PROC_TOTAL total processors" echo " $PROC_INUSE is/are in use " echo " 0 is/are use by $MYUSER" echo " $PROC_AVAIL is/are available " echo fi |