The National Institute for Computational Sciences

Open OnDemand

  Open OnDemand

Introduction


OpenOnDemand is a web-based portal that provides access to the ACF’s HPC resources. You can manage files, jobs, and access graphical interfaces all through OpenOnDemand. This document describes how to access and use OpenOnDemand and its related services.

Access and Login


To access Open OnDemand, open a web browser and type acf-login1.nics.utk.edu in the address bar. This will bring up the login page for Open OnDemand. Use your UT NetID and password to login. Figure 1.1 shows the login page for OpenOnDemand.

OpenOnDemand login
Figure 1.1 - OpenOnDemand login screen

After you successfully login, Duo TFA will come up. Select Duo Push or Passcode to log in. The authentication method will be sent to your mobile device. Figure 1.2 shows the Duo TFA options that will appear.

Duo authentication options
Figure 1.2 - Duo authentication options

Upon successful authentication, you will see the Open OnDemand dashboard. The top-left of the dashboard contains several menus that grant access to files, jobs, and your current interactive sessions.

File Explorer


The “Files” menu in the top-left of the OpenOnDemand dashboard gives you access to your files on the ACF from within a browser. Initially, the File Explorer shows your home directory. Figure 2.1 shows the File Explorer interface.

File Explorer
Figure 2.1 - File Explorer Interface

File Explorer Navigation

To navigate through directories, double-click on the directory you wish to enter. To return to your former directory, double-click on the two dots. If you are unsure where you are located within the directory tree, look above the directory listing pane. You will see several names and letters separated by forward slashes (/). This indicates the absolute pathname of your current location. In Figure 2.1, the current directory of the user is his home directory located at /nics/b/home/.

In the left pane of the File Explorer, you see the structure of your home directory. If at any time you wish to return to the top-level of your home directory, click on “Home Directory.” You may also navigate to directories within your home directory using this pane.

To quickly navigate to a specific directory, select the option labeled “Go To…” It has a dossier icon to the left of its label. When it opens, type the absolute pathname of the directory to which you wish to go. For example, to navigate to your Lustre directory, type /lustre/haven/user/<username> where <username> is your NICS username. The file explorer will then place you in the specified directory and display its contents. Figure 2.2 highlights the Go To option and the current directory location.

File Explorer Navigation
Figure 2.2 - Current Location and Go To locations in File Explorer

File Explorer Views

OpenOnDemand’s file explorer can show permissions, ownership, and hidden files. In the top-right of the file explorer interface, check the boxes for “Show Dotfiles” and “Show Owner/Mode” to enable these views. Figure 2.2 shows the directory listing pane with the Show Owner/Mode option enabled. Permissions and ownership cannot be modified from within this interface. Hidden files are prefaced with a dot (.) and are usually user configuration files.

To view a specific file, select the file in the directory listing pane. Click on the option labelled “View.” It has the icon of an eye to the left of the label. The File Explorer will show the plaintext contents of the file.

Creating, Modifying, and Deleting Files

To create a new file in the File Explorer, select the option labelled “New File.” Its icon is a sheet of paper with a plus (+) symbol in the bottom-right. When you click on the option, the File Explorer will prompt you to name the file. After you name the file and confirm its creation, it will be placed within your current directory.

To modify the file, select the file in the directory listing pane, then select the option labelled “Edit.” Its icon is a canvas with a paintbrush on top of it. The OpenOnDemand editor will open. You can then type the information you wish to put in the file. The editor features several customization options, such as the editor’s theme and text mode. Figure 2.3 shows the OpenOnDemand file editor. When you finish editing the file, click “Save” in the top-left of the editor. Figure 2.3 highlights the Save option. When you return to the File Explorer and view the file, you can view the changes you made.

File Editor
Figure 2.3 - OpenOnDemand File Editor

The File Explorer also allows you to rename existing files and directories. To rename a file or directory, select the item you wish to rename in the directory listing pane. Click the option labelled “Rename.” The File Explorer will prompt you for the new name, and once you confirm it, your changes will take immediate effect.

Files and directories can be copied and pasted within the File Explorer. To copy a file or directory, select the item in the directory listing pane, then select the option labelled “Copy.” Once the item has been copied to your clipboard, navigate to the location to which you intend to copy the file, then select the option labelled “Paste.” The copied item will then appear in your current directory.

If you wish to delete a file or directory, select the item in the directory listing pane, then select the “Delete” option on the right-side of the File Explorer. You will be prompted to confirm the deletion. Once confirmed, the item will be permanently deleted.

Downloading and Uploading Files

Files can be both downloaded from and uploaded to the ACF through the File Explorer. Only small files should be transferred between the ACF and your local system through the File Explorer. For larger file transfers, please use Globus. More information on how to configure and use Globus can be found in the Data Transfer document.

To download a file from the ACF to your local system, select the file in the directory listing pane. If you wish to download multiple files, hold down the Ctrl key on Windows and Linux systems or the Command key on MacOS, then select each file in the directory listing pane. The background of the files will become purple to indicate they are selected. Figure 2.4 shows how the files will appear when they are selected.

Multiple File Selection
Figure 2.4 - Multiple files selected in File Explorer

Once all the files you wish to download are selected, click the option labelled “Download.” It is indicated with a down arrow. After the download completes, the file(s) will appear in your “Downloads” folder unless you have specified another location for downloads. Be aware that most browsers block attempts to download multiple files unless you explicitly allow the transfer.

To upload a file to the ACF from your local system, select the “Upload” option in the top-right of the File Explorer interface. The Explorer will prompt you to choose a file. Select “Choose Files” when the dialog box appears. You may then select the file(s) you want to upload to the ACF from your local system. In addition to this method, you may also drag and drop files from your local system into the File Explorer. These items will then appear in the directory listing pane.

Job Management


In the “Jobs” menu, you have two options: Active Jobs and Job Composer. Both options provide advanced job management from within a web browser.

Job Monitoring

In the Active Jobs section, you will see an organized list of your queued and running batch jobs on the ACF. Figure 3.1 shows the Active Jobs interface. You can filter through jobs to see particular entries, in addition to sorting by the attributes of your choice. To do this, click on the arrows next to the attributes until the list is sorted as you desire.

Active Jobs Interface
Figure 3.1 - Active Jobs interface

When a job is queued or running on the ACF, you can view detailed information on the job from within the Active Jobs interface. Figure 3.2 shows the information you will see on queued and running jobs.

To access this view on a queued or running job, click the right-facing arrow on the far-left of the job’s row. You will then see statistics and information about the selected job. Of note is the “Output Location” of the job, which Figure 3.2 highlights. Each job you create and start through OpenOnDemand is placed within your /home/ondemand directory, so when you wish to retrieve the job’s data, remember to navigate to your ondemand directory. You can specify another output location when you create the job through OpenOnDemand or edit the job script to supply another location.

Detailed Job View
Figure 3.2 - Detailed view of a running job

Job Creation

In the Job Composer section, you can create new batch jobs to run on the ACF based upon several criteria. Figure 3.3 shows the Job Composer interface. When you initially open the Job Composer, the system will briefly explain each option available to you. Once you click through these prompts, you may then click “New Job,” then select the template, path, or job you wish to add. Note that if you close the initial information menu, you will not need to click through each prompt.

Job Composer
Figure 3.3 - Job Composer Interface

When you add a new job from the default template, the Job Composer will create a Simple Sequential Job. In addition to the job, it will create a job script and a job directory. Figure 3.4 shows the Job Composer interface when a job is created with the default template. To modify the job’s options, select Job Options. You can specify the job’s name, cluster, script, and account. To edit the job script, highlight the job and select the Edit Files option in the Job Composer interface. Select the job script, then select Edit. You may also rename the job script if necessary. Additional files can also be uploaded or created in the job’s directory from the File Explorer.

Default Job Template
Figure 3.4 - Selecting the Default Template for a New Job

Jobs may also be created with other templates. These can be the default templates or templates that you build within the Job Composer. Figure 3.5 shows the Templates menu.

Job Template Creation
Figure 3.5 - Templates Menu within the Job Composer

In addition to template-based job creation, you may also create jobs based upon a specified path or an existing job. With a specified path, you create a job based upon a directory that already exists. The path will either be to your home or project directory. Once you specify the path, you then modify the typical attributes of the job. With a job created based upon an existing one, select an existing job in the queue, navigate to New Job, then select “From Selected Job.”

Beyond job creation, the Job Composer allows you to submit, stop, and delete your existing jobs. Like the Active Jobs menu, you can filter and sort the available jobs.

Interactive Sessions


OpenOnDemand enables you to run interactive sessions on the ACF, which is largely the same as an interactive batch job with a graphical interface. At the time of this writing, there are several options for interactive sessions, each with different purposes and capabilities.

noVNC

noVNC allows users to interact with the ACF in a desktop environment from a web browser. In situations where you require a graphical user interface (GUI), you should choose this option for your interactive session. Be aware that while most browsers support noVNC, outdated or uncommon browsers may not function. Please try to use Chrome, Firefox, or Safari when running noVNC interactive sessions on the ACF.

To launch a noVNC interactive session, login to OpenOnDemand. In the dashboard, select “Interactive Apps” from the top-left of the window. In the dropdown menu, select “ACF Desktop.” That will open the session submission form depicted in Figure 4.1.

noVNC Session Creation
Figure 4.1 - ACF Desktop Session Submission Form

Like an interactive batch job, you specify an account to which the session should be charged, the amount of time you require the session, and the number of nodes the session should use. The Feature field is equivalent to the -l feature option in PBS, which allows you to use a special feature of a given node. The Queue field, equivalent to the -q option in PBS, specifies a particular QoS (Quality of Service) attribute. Please refer to the Running Jobs document for specific features and QoS attributes you can apply to your interactive sessions.

At the bottom of the form, you can check the box to receive an email when your session starts. It should not take the system long to provide an interactive session, though this depends on the options you specified in the submission form.

Be aware that only the number of hours and number of nodes must be specified for the job to run. The Account, Feature, and Queue fields are optional. When they are left blank, the system will use its default settings.

Once you specify the relevant information for your interactive session, select “Launch.” The session will be submitted to the system, and when it is ready, you can connect and receive a noVNC desktop environment by selecting the “Launch noVNC in New Tab” option. You can also share a View-Only link of the session with others if they need to watch your activity. Figure 4.2 shows what will appear when the noVNC interactive session is ready and highlights the “Launch” option.

Connect to noVNC
Figure 4.2 - Active noVNC Interactive Session

When you launch the session, a desktop environment will open in a new browser tab. Remember that you only have the session for the duration you specified in the “Number of hours” field when you created it. After your time expires, the session will be terminated. You may also delete the session manually if you no longer need it. If you manually terminate the session, you will receive email confirmation of the job’s deletion.

Jupyter Notebook

The Jupyter Notebook is a web-based application that enables you to intuitively create and manipulate data on the ACF. It features a fully-functional Python 2 IDE (integrated development environment), a text editor, and a file manager. The sections below outline how to access and set up Jupyter Notebooks on the ACF.

To access the Jupyter Notebook, navigate to “Interactive Apps” on the OpenOnDemand dashboard. Select “Jupyter Notebook” from the dropdown menu. The Jupyter Notebook session submission form will appear as it does in Figure 4.3.

Jupyter Session Creation
Figure 4.3 - Jupyter Notebook Session Submission Form

In this form, you must specify at a minimum the number of hours and the number of nodes you require for the session. The Account and Queue fields are optional. If you leave them blank, the system will use its default settings.

When you specify the necessary options in the form, select “Launch.” It will not take the system long to deploy the Jupyter Notebook session, but this is dependent on the resources you requested. When the session is ready, you will see the same interface that appears in Figure 4.4. The “Connect to” option will open a new Jupyter Notebook tab in your browser. Figure 4.4 highlights this option.

Connect to Jupyter
Figure 4.4 - Jupyter Notebook Ready to Launch

The initial Jupyter interface appears as it does in Figure 4.5. Like the OpenOnDemand File Explorer, you can manipulate your files and directories from within the Jupyter Notebook session. Files and directories can be created, edited, deleted, or moved. In addition, you can upload files to the ACF from your local system through the Jupyter Notebook session.

Jupyter File Explorer
Figure 4.5 - Jupyter File Explorer

Under the “New” menu on the right-side of the interface, the dropdown menu depicted in Figure 4.6 appears. From here, you can create a Python 2 Notebook session, a text file, a folder (directory), or a terminal.

Jupyter New Menu
Figure 4.6 - New Item Menu

The first option, the Python 2 Notebook session, creates a Python 2 IDE in which you can write and execute Python 2 code interactively. Figure 4.7 shows what this session looks like. There are multiple options available to use, though they are beyond the scope of this document. Please consult the official Jupyter Notebook documentation for detailed information on all the options available to you within this IDE. You can rename the session in the upper-left of the interface by selecting the default name, Untitled, and then entering the name you wish to assign. In Figure 4.7, the user assigned the name “Py2-Test.”

Jupyter Python IDE
Figure 4.7 - Jupyter Notebook Python 2 IDE

As you work within the IDE, Jupyter saves your work in a .ipynb file within your home directory. Its name will be the same as the name you assign within the IDE. You can view the file from the “Files” section within Jupyter. Additionally, if the session is running, the icon for the file will be green and will state that it is running next to the file’s timestamp.

The second option, text file, opens the Jupyter text editor. Figure 4.8 shows the initial text editor interface. Within this editor you can write code for multiple languages or simply use plaintext. Several different programming languages are supported by the editor. To see them all, click on the “Language” menu as shown in Figure 4.9. When you select a language, the editor will perform syntax highlighting and other functions to assist you as you code.

Jupyter Editor
Figure 4.8 - Jupyter Text Editor

Jupyter Editor Languages
Figure 4.9 - Jupyter Text Editor Languages

Once you finish working in the text editor, select File, then Save. Be sure to rename the file by clicking on the default title, untitled.txt, and changing it to a more descriptive name. The saved file will appear in your home directory by whatever name you selected.

The third option, folder, creates a new directory in your current one. The fourth option, terminal, opens a ssh session in another browser tab. Select this option to perform certain tasks that the graphical interface does not allow, such as long listings of a directory or file searches.

To see the running Jupyter processes, return to the Jupyter Files menu by clicking on the Jupyter icon in the top-left of the interface. From there, select the “Running” menu. If you have any terminals or notebooks active, they will appear here. You can shut these processes down if you no longer need them or have completed your work. Figure 4.10 shows an example of the Running menu.

Jupyter's Running Menu
Figure 4.10 - Jupyter's Running Menu

At the time of this writing, the “Clusters” menu seen in Figure 4.10 next to the Running menu has no functionality.

When you finish your work within Jupyter, select the “Logout” option in the top-right of the window. Close the Jupyter Notebook tab, then return to your interactive sessions in OpenOnDemand. If you have time remaining on your job, please delete it. Do note that if you exceed your allotted time in the session, it will be terminated. Also be aware that when you manually delete the job for a Jupyter Notebook session, you will receive an email that confirms its deletion.

RStudio

RStudio is a web-based IDE (integrated development environment) for the R statistical language on the ACF. Like Jupyter, it provides additional functionality beyond a standard interactive console. The specifics of how to access and manipulate RStudio are described below.

To access RStudio, open the OpenOnDemand dashboard. In the top-left, select “Interactive Apps.” In the dropdown menu, select “RStudio Server.” After you click that option, the submission form in Figure 4.11 will appear.

RStudio Session Creation
Figure 4.11 - RStudio Session Submission Form

You can specify any of the options in the form to whatever you require; however, only the number of hours and the number of nodes are necessary. The other options will use the default values for those fields if you leave them blank. For more information on the Account, Queue, QoS, and Feature fields, please refer to the Running Jobs document.

After you specify the options for your session, select Launch. The session will be placed into the queue. The start time for the RStudio session depends on the options you specified in the form. When the session is ready, the interface will appear as it does in Figure 4.12. Select the “Connect to RStudio Server” option to begin the session. Figure 4.12 highlights this option.

Connect to RStudio
Figure 4.12 - RStudio Ready to Launch

A new browser tab will open with the RStudio Server session. When it initially starts, you will see an interface with three panes: a console pane, an environment pane, and a file explorer pane.

The console pane, depicted in Figure 4.13, is where you run R code. At the top of the window, there are several menus with different options. It is not the purpose of this document to describe these menus in detail. Instead, please consult the official RStudio documentation for more information.

RStudio Console Pane
Figure 4.13 - RStudio Console Pane

In addition to the menus, there are several buttons that allow you to create, open, and save files and projects, in addition to options for menu layouts and addins. Below these buttons is the console itself. You can input R code into the console, press Enter (Return), and see the output of that code.

The environment pane contains information that pertains to the IDE itself, such as what packages are enabled, what dataset is in use, and your command history. Figure 4.14 shows the environment pane with the Environment tab selected.

RStudio Environment Pane
Figure 4.14 - RStudio Environment Pane

In the file explorer pane, you can view and modify your files on the ACF, view additional packages, read documentation, and more. Figure 4.15 shows the file explorer pane. In the Files tab of the file explorer pane, you can create new folders, upload, delete, and rename files, in addition to several other options. To navigate through directories in the file explorer pane, click on the directory you wish to enter. To return to the previous directory, click the double dots (..). To return to your home directory, select “Home,” which has the icon of a house next to it.

RStudio File Explorer Pane
Figure 4.15 - RStudio File Explorer Pane

Whenever you type the “help()” command, the RStudio Server documentation will appear in the “Help” tab of the file explorer pane. You can view all relevant documentation for RStudio from this tab, if necessary. To expand the view so it is easier to read, minimize the environment pane by clicking the horizontal bar in the top-right of the pane. This will expand the file explorer pane, making it easier to read documentation. After you finish your RStudio Server session, select the red IO button in the top-right of the interface and confirm the shutdown. You will then be disconnected from the RStudio Server session. Be sure to also delete the job if any time remains on it. You will receive an email that confirms this deletion.

Known Issues

If you encounter a problem with OpenOnDemand and its services, please review the official Known Issues document before submitting a ticket. If the issue is not present in the official documentation or the proposed workarounds fail, then please submit a ticket to help@nics.utk.edu.

Last Updated: 11 / 13 / 2019