The National Institute for Computational Sciences

The HPSS queue

Using the HPSS Queue

The hpss queue can be used to transfer files or directories to hpss using a batch script. You may only submit jobs to the hpss queue if you logged into a node with your RSA SecurID OTP token. The maximum wall clock limit for the hpss queue is 24 hours. To submit jobs to the hpss queue, use the qsub -q option (#PBS -q hpss). Using job dependencies, you can schedule an HPSS job to stage data before and/or after a normal production job.

Following is an example of a simple HPSS batch file with a dependency.

#!/bin/bash
#PBS -A TG-EXAMPLE

#PBS -l walltime=10:00:00
#PBS -q hpss
#PBS -W depend=afterok:123456

cd $PBS_O_WORKDIR
hsi put file
htar cvf this_run.tar dir/

This will simply be submitted using

> qsub filename
  • #PBS -q hpss - submit jobs to the hpss queue
  • #PBS -W depend=afterok:123456- stage the transfer after the completion of job 123456
  • #PBS -l walltime=10:00:00 - set the wall clock limit to 10 hours.

Note: the hpss queue is not currently working on Darter.