How to sumbit jobs on HiperGator

• preparation
  • FileZilla
  • terminal (Mac, Linux)
  • putty (Windows)
• scripts: https://caleb-huo.github.io/teaching/2017FALL/lectures/week4_Matrix/figure/HiperGatorUseR.zip
• account
  • apply https://www.rc.ufl.edu/access/account-request/

How to login HiperGator (Windows)

• putty
  • Host Name: hpg2.rc.ufl.edu
  • Port: 22
  • Protocal: SSH
  • Open
• username and passwd

How to login HiperGator (Mac, Windows)

• terminal
  • ssh user@hpg2.rc.ufl.edu

Common linux commands:

• pwd: currect working directory
• ls: list all files in current working directory
• mkdir afolder: create a folder named afolder
• cd afolder: get into afolder
• cd ..: go back one folder
• cd ~: go to your home directory
• top: monitor CPU/memory usages

Some exercises:

• cd /ufrc/phc6068/share/zhuo/testR ## go the the folder with all test files

• cp saveCars.R /ufrc/phc6068/your directory

• cat saveCars.R: print everying
• more saveCars.R: interactive print
• head saveCars.R: head of the contents

• cat saveCars.sh

FileZilla

• Host: sftp://sftp.rc.ufl.edu
• username:
• password:
• Port: 22
• connect

You can transfer files between your local computer and hiperGator

Submit a job (I)

• R script (saveCars.R): contains your R code
• SLURM job script (saveCars.sh): coordinate your job with the server
• submit: sbatch yourJob

R script (a simple one) (I)

setwd("/ufrc/phc6068/share/zhuo/testR") ## change to your own directory!
mycars <- mtcars
write.csv(mycars,"mycars.csv")

SLURM job script (I)

#!/bin/sh
#SBATCH --job-name=serial_job_test    # Job name
#SBATCH --account=phc6068             # your own sponser or account from this class
#SBATCH --qos=phc6068                 # your own sponser or account from this class
#SBATCH --mail-type=ALL               # Mail events
#SBATCH --mail-user=xx@xx.xx          # Where to send mail  
#SBATCH --ntasks=1                    # Run on a single CPU
#SBATCH --mem=1gb                     # Memory limit
#SBATCH --time=00:05:00               # Time: hrs:min:sec
#SBATCH --output=serial_test_%j.out   # Output and error log 

pwd; hostname; date 

module load R 

echo "Running save cars script on a single CPU core" 

R CMD BATCH saveCars.R
## R --no-save --quiet --slave < saveCars.R 

date

submit the job (I)

cd /ufrc/phc6068/share/zhuo/testR
sbatch saveCars.sh ## submit job

Interactive session

srun --account=phc6068 --qos=phc6068 --ntasks=1  --mem=1gb  --time=00:05:00 --pty bash -i ## open interactive R session

module load R ## load R

R

Submit a job with external argument (II)

• R script (saveCarsArgs.R): contains your R code
• SLURM job script (saveCarsArgs.sh): coordinate your job with the server
• submit: sbatch yourJob

R script (with external arguments) (II)

args = commandArgs(trailingOnly = TRUE) ## order aalpha, S, sigmaNoise, sigma_0

rowID <- args[1]
aarg <- as.numeric(rowID)

setwd("/ufrc/phc6068/share/zhuo/testR")
mycars <- mtcars[aarg,]
filename <- paste0("arg",aarg,".csv")
write.csv(mycars,filename)

SLURM job script (II)

#!/bin/sh
#SBATCH --job-name=serial_job_test    # Job name
#SBATCH --account=phc6068             # your own sponser or account from this class
#SBATCH --qos=phc6068                 # your own sponser or account from this class
#SBATCH --mail-type=ALL               # Mail events
#SBATCH --mail-user=xx@xx.xx          # Where to send mail  
#SBATCH --ntasks=1                    # Run on a single CPU
#SBATCH --mem=1gb                     # Memory limit
#SBATCH --time=00:05:00               # Time: hrs:min:sec
#SBATCH --output=serial_test_%j.out   # Output and error log 

pwd; hostname; date 

module load R 

echo "Running save cars script on a single CPU core" 

R --no-save --quiet --slave --args 1 < saveCarsArgs.R 

date

submit the job (II)

cd /ufrc/phc6068/share/zhuo/testR
sbatch saveCarsArgs.sh ## submit job

Submit a job with loops (III)

• R script (saveCars.R): contains your R code
• SLURM job script (saveCarsArgsLoops.sh): coordinate your job with the server
• submit: sbatch yourJob

SLURM job script (III)

#!/bin/sh
#SBATCH --job-name=serial_job_test    # Job name
#SBATCH --account=phc6068             # your own sponser or account from this class
#SBATCH --qos=phc6068                 # your own sponser or account from this class
#SBATCH --mail-type=ALL               # Mail events
#SBATCH --mail-user=xx@xx.xx          # Where to send mail  
#SBATCH --ntasks=1                    # Run on a single CPU
#SBATCH --mem=1gb                     # Memory limit
#SBATCH --time=00:05:00               # Time: hrs:min:sec
#SBATCH --output=serial_test_%j.out   # Output and error log 

pwd; hostname; date 

module load R 

for i in {2..10}
do
echo "Running save cars" $i 
R --no-save --quiet --slave --args $i < saveCarsArgs.R 
done

date

submit the job (III)

cd /ufrc/phc6068/share/zhuo/testR
sbatch saveCarsLoops.sh ## submit a loop job