Skip to content Skip to navigation

Connexions

You are here: Home » Content » Practical 3 - Basic MPI

Navigation

Lenses

What is a lens?

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

This content is ...

In these lenses

  • eScience, eResearch and Computational Problem Solving

    This module is included inLens: eScience, eResearch and Computational Problem Solving
    By: Jan E. OdegardAs a part of collection: "An Introduction to High-Performance Computing (HPC)"

    Click the "eScience, eResearch and Computational Problem Solving" link to see all content selected in this lens.

Recently Viewed

This feature requires Javascript to be enabled.
 

Practical 3 - Basic MPI

Module by: Tim Stitt Ph.D.. E-mail the author

Summary: In this module you will gain some experience with the fundamentals of the MPI programming paradigm by executing some simple MPI codes through a series of simple exercises.

Introduction

In this practical you will develop and execute some simple codes, which incorporate the most fundamental MPI routines. Exercises include the querying of MPI ranks and size for a trivial MPI code, to the calculation of speedup for computing pi in parallel.

If you require any assistance, please do not hesitate to contact the available support staff.

Disclaimer:

This course (and accompanying practical exercises) are not designed to teach you MPI programming but are provided to give you a feel for the MPI programming paradigm. A detailed introduction to MPI will be given in further courses that are under development.

Objectives

The objectives of this practical are to gain experience in:

  1. executing a MPI code with different process counts
  2. using basic MPI routines to query the size and rank of a MPI program
  3. calculating speedup for a simple MPI program

MPI: A Crash Course

MPI generally follows a Single Program Multiple Data (SPMD) programming model, whereby a single executable (called a process) is duplicated and executed on multiple processors. Each process (within the MPI Communication World) is uniquely identified by its rank and individual processes can communicate between each other using MPI message-passing routines.

Note:

MPI can also accommodate a Multiple Programming Multiple Data (MPMD) programming paradigm (where each process is a different executable) but this approach is less used (and supported) compared to the SPMD approach.

Figure 1 shows a typical MPI communication world (initially referred to as MPI_COMM_WORLD) with 4 processes (labeled P) each identified with a unique rank in the range 0-3. Each process executes the same task T. Processes communicate among other processes in the communication world by sending and receiving messages.

Figure 1: The Standard MPI Communication World
MPI Communication World

MPI Info:

Even though each process contains an instance of the same executable, different instruction streams can be executed by embedding rank dependent code into the executable e.g.
if (myrank .eq. 0) then
  call do_this_work_on_process_0()
else if (myrank .eq. 1) then
  call do_this_work_on_process_1()
else
  call do_this_work_on_remaining_processes()
end if

The most fundamental MPI operations can be classified into 3 groups:

  1. MPI Initialization and Finalization
    • Initialization: MPI_Init()
    • Finalization: MPI_Finalize()
  2. Querying Process Rank and Communication World Size
    • Query Process Rank: MPI_Comm_rank()
    • Query World Size: MPI_Comm_size()
  3. Sending and Receiving of Messages
    • Send Message: MPI_Send()
    • Receive Message: MPI_Receive()

Note:

For more detailed information on MPI, download and review the latest MPI standard specification.

Basic MPI: "Hello World"

Exercise 1: Executing A Simple MPI Code

Develop, compile and execute a code (which uses the MPI framework) to display a "Hello World" message on multiple processes. Test your MPI code works correctly for 1, 2 4 and 8 cores.

Code Tip:

If you are not brave enough (yet) to write the code, you can use the MPI template provided in the ../Practicals/Practical_3/Exercise_1 folder.

Batch Script Tip:

Make sure you modify your submission script to request multiple processes and cores in your job e.g. to test your code for 8 processes modify your script to contain the following: 
...
#PBS -l mppwidth=8        # Request 8 cores
...
...
aprun -n 8 ./hello_world  # Request 8 processes

Exercise 2: Querying MPI Rank and Size

Modify your solution in Exercise 1 to print the "Hello World" message, along with the rank of the current process and the total number of processes in the MPI communication world.

Hint A

Use the MPI_COMM_RANK() and MPI_COMM_SIZE() routines to find the rank of the current process and the size of the communication world, respectively.

Fortran Solution B


...
! Initialize MPI
call MPI_INIT(error) 
 
call MPI_COMM_RANK(MPI_COMM_WORLD,myrank,error) 
call MPI_COMM_SIZE(MPI_COMM_WORLD,np,error)
print *, 'Hello World! I am process', myrank, ' of ', np 
  
! Shut down MPI 
call MPI_FINALIZE(error)
...

C Solution C

...
MPI_Init(&argc,&argv);

MPI_Comm_size(MPI_COMM_WORLD,&np);
MPI_Comm_rank(MPI_COMM_WORLD,&myrank);

printf("Hello World! I am process %d of %d\n", myrank, np);

MPI_Finalize();
...

Measuring Speedup

In parallel computing, speedup refers to how much a parallel algorithm is faster than a corresponding sequential algorithm.

Speedup is defined as:
S p = T 1 T p S p T 1 T p

where

  • p is the number of processers
  • T 1 T 1 is the time of the sequential algorithm
  • T p T p is the time of the parallel algorithm with p processors

Exercise 3: Calculating PI

The value of pi can be computed approximately using an integration technique (see the course slides for more information). Using the serial and parallel code implementations found in ../Practicals/Practical_3/Exercise_2, calculate the speedup obtained for 1, 2, 4 and 8 processors.

Tip:

Briefly review the serial and parallel code implementations so you are familiar with the computational method and its MPI parallelisaton.

Record your results in a table similar to Table 1.

Table 1: Speedup Results for PI Calculation
Processors Time Taken(sec) Speedup
1		    
2		    
4		    
8		    

Glossary

Message Passing:

A communication protocol whereby processes communicate by sending and receiving data through messages.

MPI Communication World:

A collection of MPI processes which can communicate through passing messages. The default communication world for a MPI process is MPI_COMM_WORLD

MPI Rank:

Each process in a MPI communication world of P processes is assigned an unique id in the range 0..(P-1). This id is called the rank of the process.

Speedup:

In parallel computing, speedup refers to how much a parallel algorithm is faster than a corresponding sequential algorithm.

Figure 2: Speedup Diagram
Speedup Diagram
SPMD:

Single Program Multiple Data: multiple autonomous processors simultaneously execute the same program at independent points.

Further information: http://en.wikipedia.org/wiki/SPMD

Content actions

Download module as:

PDF | EPUB (?)

What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

Downloading to a reading device

For detailed instructions on how to download this content's EPUB to your specific device, click the "(?)" link.

| More downloads ...

Add module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks