Skip to content Skip to navigation Skip to collection information

OpenStax_CNX

You are here: Home » Content » An Introduction to High-Performance Computing (HPC) » Editing, Compiling and Submitting Jobs on Ela

Navigation

Lenses

What is a lens?

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

This content is ...

In these lenses

  • eScience, eResearch and Computational Problem Solving

    This module and collection are included inLens: eScience, eResearch and Computational Problem Solving
    By: Jan E. Odegard

    Click the "eScience, eResearch and Computational Problem Solving" link to see all content selected in this lens.

Recently Viewed

This feature requires Javascript to be enabled.
 

Editing, Compiling and Submitting Jobs on Ela

Module by: Tim Stitt Ph.D.. E-mail the author

Summary: In this module you will learn the fundamentals of editing, compiling and submitting codes on the CSCS Gele system.

Introduction

For this tutorial you will be using the Ela front-end system for editing, compiling and running your codes. Your executables will be submitted on Gele which is the test system for CSCS's flagship machine Monte Rosa.

Please complete this section before attempting the remaining exercises. If you require any assistance, please do not hesitate to contact the available support staff.

Logging into Ela

From your local system, you can log into Ela (using your supplied username and password) with the following commands:

Logging on to the Front-End System


ssh -X username@ela.cscs.ch

When you log into the front-end node Ela, you need to set up your programming environment as follows:

Setting Your Programming Environment


module load gele
module load PrgEnv-cray
module load pbs

Modules Tip:

Loading modules allows the user to flexibly customize their programming environment and system software within their session. In the commands above, we have selected the mandatory system software for the Gele system and the Cray Programming Environment. For managing the submission and scheduling of jobs on the compute nodes of Gele, we have selected the PBS batch queuing system (more about this later).

Module Information:



To view the available modules on the system use the command:

module avail

To view the currently loaded modules, use the command:

module list

Editing Files on Ela

On Ela you can use either the emacs or vi editors to modify source code or job submission scripts. To invoke the editors, use the following commands:

Invoking Emacs (with GUI)


emacs foo.f90

Invoking Emacs (without GUI)


emacs -nw foo.f90

Invoking Vi


vi foo.f90

Compiling Source Codes on Ela

To compile source codes on Ela, please use one of the available compiler wrappers.

The Fortran Compiler Wrapper


ftn -o foo foo.f90 

The C and C++ Compiler Wrappers


cc -o foo foo.c
CC -o foo foo.cpp

Compilers Tip:

For more information on using the compiler wrappers and adding compiler flags for debugging and optimization, please consult the respective man pages on Ela e.g.

man ftn
man cc
man CC

Submitting Jobs to Gele

Codes are executed by submitting a job script to a batch submission queue which in turn schedules the jobs to run on the compute nodes of Gele. To submit a job to Gele, please use the following submission script and submissions commands.

Sample Batch Submission Script


#! /bin/bash
#PBS -l mppwidth=8
#PBS -l walltime=00:10:00
#PBS -V

set -ex

cd $PBS_O_WORKDIR

aprun -n 8 ./foo.exe

exit

In this script we request 8 parallel tasks (mppwidth=8) for a maximum duration of 10 minutes (walltime=00:10:00). We ensure that our environment settings are exported to the compute nodes for execution (-V) and that our execution command is invoked within the same directory as that which the job script was submitted in.

Batch Script Tip:

The $PBS_O_WORKDIR environment variable always contains the path to the submission directory for the batch script.

To invoke an instance of the executable on the compute nodes, you are required to use the aprun command. You can run multiple instances of the same executable across multiple cores by including the -n parameter. In the sample batch script above, we have requested that our foo executable be duplicated on all 8 cores requested within the batch script resources.

Important:

Please ensure that your aprun command uses all the resources requested by the mppwidth parameter, otherwise valuable resources will be wasted.

You can modify most of the above settings if required. Please refer to the man pages (man qsub and man aprun) for further information.

To submit your job script to the batch queuing system use the following command:

Submit Job Script to Queue


qsub script

Note:

In this example the job script is named script

When you submit your script successfully, the batch queuing system will issue you with a unique jobid for your submitted job. You can view the queuing and running status of your job by using the following commands:

Viewing Jobs by Job ID


qstat jobid

Viewing Multiple Jobs by Username


qstat -u username

After termination of your job, you will receive stdout (standard output) and stderr (standard error) logs to your submission directory. These logs are suffixed with the id of your submitted job. Please review these logs carefully to determine whether you had successful or unsuccessful termination of your executable.

Practical Exercises

You are now in a position to attempt the "hands-on" practicals that complement the presentations. A number of practical exercises are included in this tutorial covering:

  1. Compilation and Submission of a Simple Code
  2. Experimentation with Compiler Optimization Flags and Timing Routines
  3. Basic MPI
  4. Basic OpenMP

Obtaining the Exercise Sources

Before attempting the practicals you should copy across the exercise source templates to your home directory. To do this complete the following instructions:

  1. Step 1. cp ~tstitt/LinkSceem/Practicals.tgz ./
  2. Step 2. tar -zxvf Practicals.tgz

Tip:

Please feel free to attempt the practicals in any order depending on your preferences and level of experience.

Note:

If you need any assistance with the exercises, please don't hesitate to contact the available support staff who will be glad to help with your problem.

Glossary

Batch Queue:

A job queue maintained by the scheduler software for determining the order in which jobs are executed based on user priority, estimated execution time and available resources.

Compiler Wrapper:

A compiler wrapper is a script that invokes the low-level compiler command with the required link libraries for a specific computing system and message-passing framework. Typically standard scientific and vendor-specific libraries are also included in the wrapper script.

Debugging:

The art of identifying and removing compile-time and runtime errors from a code.

Emacs:

Emacs is a feature-rich editor that has been in development since the mid-70s.

Please take a look at the emacs tutorial for more information on using the editor.

Ela:

Ela is a 16 core front-end node which is used to edit and compile source code and submit jobs for execution on the compute nodes of Monte Rosa.

Gele:

Gele is a 76 node dual-core system. It is the test system for the CSCS flagship machine Monte Rosa.

Job Script:

A script which describes the command line execution of an executable along with the core, memory and time resources required by the calculation.

Module:

A script the configures the search paths and link commands for a given software package or library.

mppwidth:

The total number of cores requested by the user for the completion of the job.

Optimization:

The art of improving the memory performance and runtime performance of an executing code.

Programming Environment:

The Programming Environment is the collection of compiler and pre-built libraries that are specific to an individual compiler suite.

Monte Rosa:

Monte Rosa is a Cray XT5 supercomputer with a peak performance of 140 TeraFlop/s. Its current specification is:

  1. 3688 AMD quad core Opteron @ 2.4 GHz
  2. 29.5 TB DDR2 RAM
  3. 290 TB Disk
  4. 9.6 GB/s interconnect bandwidth
Figure 1: Monte Rosa
Image of Monte Rosa

Vi:

Vi is a family of screen-oriented text editors which has been developed since 1976.

Please take a look at the Vi tutorial for more information on using the editor.

walltime:

The total real time for completing a task. In a batch script, it refers to the maximum wall-clock time required by the job.

Collection Navigation

Content actions

Download:

Collection as:

PDF | EPUB (?)

What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

Downloading to a reading device

For detailed instructions on how to download this content's EPUB to your specific device, click the "(?)" link.

| More downloads ...

Module as:

PDF | EPUB (?)

What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

Downloading to a reading device

For detailed instructions on how to download this content's EPUB to your specific device, click the "(?)" link.

| More downloads ...

Add:

Collection to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks

Module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks