Nautilus was decommissioned on May 1, 2015. For more information, see Nautilus Decommission FAQs.
Nautilus consists of one UV1000, which has 1024 cores, 4 terabytes of global shared memory in a single system image, 960 TB Lustre file system, access to the NICS HPSS storage system. Nautilus also has four UV10 Harpoon Nodes, containing 32 nodes, 128 GB memory, 4 NVIDIA Tesla (codename Fermi) GPUs with 3GB of GDDR5 memory per GPU, supporting CUDA C/C++/Fortran, OpenCL, and DirectCompute Toolkits.
Nautilus provides C, C++ and Fortran compilers from
with Intel being the default. Special programming environment or PE modules are used to easily switch compilers.
MPI is provided on Nautilus by
SGI's Message Passing Toolkit (MPT)
which supports version 2.2 of the MPI standard as well as providing the SHMEM
MPT is loaded by default with any of the PE modules. When compiling your MPI
code, use the
-lmpi linker flag.
For both batch and interactive jobs that utilize GPUs one can request this resource
-l gpus=N. Currently, there is a limitation in that Moab interprets this value
When you log into Nautilus at login.nautilus.nics.tennessee.edu or login.nautilus.nics.xsede.org, you are placed on one of two login nodes, arronax or conseil. An additional login node, nedland, is also available for gsissh connections. Login nodes should be used for basic tasks such as file editing, file transfers, code compilation, data backup, and job submission. The login nodes should not be used to run production jobs. Production work should be performed on the systems compute resources.
To submit a job to PBS, use the
qsub command. The most common way to use
qsub is to simply pass it the name of a 'batch script' that contains all of the information about running your job:
> qsub myscript.pbs
For information on how to write a batch script and the various PBS options available, see the Batch Scripts page.
You can also use
qsub to start an interactive session on Nautilus by including the
> qsub -I -A XXXYYY -l ncpus=16,walltime=1:00:00,mem=64GB
As you can see in this case, you must specify the PBS options along with the
qsub command (again, see the Batch Scripts page for an explanation of PBS options.)
The Nautilus system consists of one UV1000 (Nautilus) with 1016 available cores and four UV10 (Harpoon) nodes with 32 cores each. By default, jobs with 32 cores or less will be placed on a Harpoon node if available. You may use the
-l feature PBS option to explicitly request Nautilus or a Harpoon node. To guarantee job placement on Nautilus use:
#PBS -l feature=nautilus
To guarantee placement on a Harpoon node (with 32 cores or less) use:
#PBS -l feature=uv10
Upon submission, your job will be placed in a particular queue. The job scheduler, Moab, uses queues to aid in the organization and scheduling of jobs. See the Queues page for the available queues on Nautilus and their benefits.
Nautilus supports MPI and OpenMP for running parallel jobs. See the Parallel Execution page for details on their use along with various examples.
If you need to run many concurrent instances of serial code, we provide the Eden tool for managing such jobs. Please see the Eden documentation page for instructions on use.
File I/O and Ramdisk
$SCRATCH_RAMDISK environment variable contains the
pathname to a job-specific directory residing on a RAM-based file system.
$SCRATCH_RAMDISK, applications can perform I/O to blade
memory rather than to persistent storage. Memory I/O is, in general, several
orders of magnitude faster than mechanical disks and certain applications may
see improved I/O speeds by using
$SCRATCH_RAMDISK to create files
in memory. While the actual performance will vary between applications,
some I/O patterns are more likely to benefit than others.
- Frequent small I/O requests
- Shared read-only files
- File-per-process with large numbers of small files
Please note that:
$SCRATCH_RAMDISKis not persistent and any data that needs to be saved must be copied to
/lustre/medusabefore the job terminates. Once a job terminates, all data on
$SCRATCH_RAMDISKwill be lost. The amount of time needed to copy data should be taken into account when weighing the benefits of using
$SCRATCH_RAMDISKtakes advantage of free memory (within a job allocation) and, as a result, is relatively limited in size compared to
/lustre/medusa. Therefore, users should not save files whose total size are larger than the memory available.
For information on how projects are charged for time on Nautilus, see the Job Accounting page.
Scheduling Policy - Nautilus
Moab on Nautilus is configured to do “first fit” backfill. Backfilling allows smaller, shorter jobs to use otherwise idle resources.
Computation Queue Policies
- The scheduler places a limit of one running computation job per user.
- Computation jobs can be preempted by analysis jobs at any time. Preempted jobs will be re-queued.
- The maximum wall time for computation jobs is 48 hours.
Analysis Queue Policies
- There is no limit on the number of running analysis jobs per user.
- The maximum wall time for analysis jobs is 48 hours.
VNC Session on Nautilus
For instructions on how to start a VNC session on Nautilus, see the VNC Session page.
NX Session on Nautilus
For instructions on how to start a NX session on Nautilus, see the NX Session page.