Skip to content

Commit 2d5505c

Browse files
jsquyresabouteiller
authored andcommitted
docs: numerous updates to MPI_Init*/MPI_Finalize*/MPI_Session_* man pages
Including, but not limited to: * Added much more description of and distinction between the MPI world model and the MPI session model. Updated a lot of old, pre-MPI-world-model/pre-MPI-session-model text that was now stale / outdated, especially in the following pages: * MPI_Init(3), MPI_Init_thread(3) * MPI_Initialized(3) * MPI_Finalize(3) * MPI_Finalized(3) * MPI_Session_init(3) * MPI_Session_finalize(3) * Numerous formatting updates * Slightly improve the C code examples * Describe the mathematical relationship between the various MPI_THREAD_* constants in MPI_Init_thread(3) * Note that the mathematical relationships render nicely in HTML, but don't render entirely properly in nroff. This commit author is of the opinion that the nroff rendering is currently "good enough", and some Sphinx maintainer will fix it someday. * Add descriptions about the $OMPI_MPI_THREAD_LEVEL env variable and how it is used in MPI_Init_thread(3) * Added more seealso links Signed-off-by: Jeff Squyres <jeff@squyres.com> (cherry picked from commit aff3afd)
1 parent 6ad6cc8 commit 2d5505c

File tree

7 files changed

+309
-125
lines changed

7 files changed

+309
-125
lines changed

docs/man-openmpi/man3/MPI_Finalize.3.rst

Lines changed: 70 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ MPI_Finalize
55

66
.. include_body
77
8-
:ref:`MPI_Finalize` |mdash| Terminates MPI execution environment.
8+
:ref:`MPI_Finalize` |mdash| Terminates MPI world model.
99

1010
SYNTAX
1111
------
@@ -48,56 +48,82 @@ OUTPUT PARAMETER
4848
DESCRIPTION
4949
-----------
5050

51-
This routine cleans up all MPI states. Once this routine is called, no
52-
MPI routine (not even MPI_Init) may be called, except for
53-
:ref:`MPI_Get_version`, :ref:`MPI_Initialized`, and :ref:`MPI_Finalized`. Unless there has
54-
been a call to :ref:`MPI_Abort`, you must ensure that all pending
55-
communications involving a process are complete before the process calls
56-
:ref:`MPI_Finalize`. If the call returns, each process may either continue
57-
local computations or exit without participating in further
58-
communication with other processes. At the moment when the last process
59-
calls :ref:`MPI_Finalize`, all pending sends must be matched by a receive, and
60-
all pending receives must be matched by a send.
61-
62-
:ref:`MPI_Finalize` is collective over all connected processes. If no processes
63-
were spawned, accepted, or connected, then this means it is collective
64-
over MPI_COMM_WORLD. Otherwise, it is collective over the union of all
65-
processes that have been and continue to be connected.
51+
This routine finalizes the MPI world model. If the MPI world model
52+
has been initialized in an MPI process, it *must* be finalized exactly
53+
once by invoking this routine during the lifetime of that MPI process.
54+
This is different than the MPI session model, which can be initialized
55+
and finalized multiple times in an MPI process. See
56+
:ref:`MPI_Session_init` and :ref:`MPI_Session_finalize`.
57+
58+
Unless there has been a call to :ref:`MPI_Abort`, you must
59+
ensure that all pending communications in the MPI world model
60+
involving a process are complete before the process calls
61+
:ref:`MPI_Finalize`. If the call returns, each process may either
62+
continue local computations or exit without participating in further
63+
communication with other processes in the MPI world model. At the
64+
moment when the last process calls :ref:`MPI_Finalize`, all pending
65+
sends in the MPI world model must be matched by a receive, and all
66+
pending receives in the MPI world model must be matched by a send.
67+
68+
See `MPI-5.0:11.4.1 <https://www.mpi-forum.org/>`_ for a list of MPI
69+
functionality that is available (e.g., even when the MPI
70+
world model has not yet initialized or has already been finalized).
71+
72+
:ref:`MPI_Finalize` is collective over all connected processes. If no
73+
processes were spawned, accepted, or connected, then this means it is
74+
collective over ``MPI_COMM_WORLD``. Otherwise, it is collective over
75+
the union of all processes that have been and continue to be
76+
connected.
6677

6778
NOTES
6879
-----
6980

70-
All processes must call this routine before exiting. All processes will
71-
still exist but may not make any further MPI calls. :ref:`MPI_Finalize`
72-
guarantees that all local actions required by communications the user
73-
has completed will, in fact, occur before it returns. However,
74-
:ref:`MPI_Finalize` guarantees nothing about pending communications that have
75-
not been completed; completion is ensured only by :ref:`MPI_Wait`, :ref:`MPI_Test`, or
76-
:ref:`MPI_Request_free` combined with some other verification of completion.
77-
78-
For example, a successful return from a blocking communication operation
79-
or from :ref:`MPI_Wait` or :ref:`MPI_Test` means that the communication is completed
80-
by the user and the buffer can be reused, but does not guarantee that
81-
the local process has no more work to do. Similarly, a successful return
82-
from :ref:`MPI_Request_free` with a request handle generated by an :ref:`MPI_Isend`
83-
nullifies the handle but does not guarantee that the operation has
84-
completed. The :ref:`MPI_Isend` is complete only when a matching receive has
85-
completed.
86-
87-
If you would like to cause actions to happen when a process finishes,
88-
attach an attribute to MPI_COMM_SELF with a callback function. Then,
89-
when :ref:`MPI_Finalize` is called, it will first execute the equivalent of an
90-
:ref:`MPI_Comm_free` on MPI_COMM_SELF. This will cause the delete callback
91-
function to be executed on all keys associated with MPI_COMM_SELF in an
92-
arbitrary order. If no key has been attached to MPI_COMM_SELF, then no
93-
callback is invoked. This freeing of MPI_COMM_SELF happens before any
94-
other parts of MPI are affected. Calling :ref:`MPI_Finalized` will thus return
95-
"false" in any of these callback functions. Once you have done this with
96-
MPI_COMM_SELF, the results of :ref:`MPI_Finalize` are not specified.
81+
The MPI session model is different than the MPI world model, and has
82+
different scopes of availability for MPI functionality. See
83+
:ref:`MPI_Session_init` and :ref:`MPI_Session_finalize`.
84+
85+
All processes that initialized the MPI world model must call this
86+
routine before exiting. All processes will still exist but may not
87+
make any further MPI calls in the MPI world model. :ref:`MPI_Finalize`
88+
guarantees that all local actions required by communications in the
89+
MPI world model that the user has completed will, in fact, occur
90+
before it returns. However, :ref:`MPI_Finalize` guarantees nothing
91+
about pending communications in the MPI world model that have not been
92+
completed; completion is ensured only by the :ref:`MPI_Wait` and
93+
:ref:`MPI_Test` variants, or :ref:`MPI_Request_free` combined with
94+
some other verification of completion.
95+
96+
For example, a successful return from a blocking communication
97+
operation or from one of the :ref:`MPI_Wait` or :ref:`MPI_Test`
98+
varients means that the communication is completed by the user and the
99+
buffer can be reused, but does not guarantee that the local process
100+
has no more work to do. Similarly, a successful return from
101+
:ref:`MPI_Request_free` with a request handle generated by an
102+
:ref:`MPI_Isend` nullifies the handle but does not guarantee that the
103+
operation has completed. The :ref:`MPI_Isend` is complete only when a
104+
matching receive has completed.
105+
106+
If you would like to cause actions to happen when a process finalizes the MPI
107+
world model, attach an attribute to ``MPI_COMM_SELF`` with a callback
108+
function. Then, when :ref:`MPI_Finalize` is called, it will first
109+
execute the equivalent of an :ref:`MPI_Comm_free` on
110+
``MPI_COMM_SELF``. This will cause the delete callback function to be
111+
executed on all keys associated with ``MPI_COMM_SELF`` in an arbitrary
112+
order. If no key has been attached to ``MPI_COMM_SELF``, then no
113+
callback is invoked. This freeing of ``MPI_COMM_SELF`` happens before
114+
any other parts of the MPI world model are affected. Calling
115+
:ref:`MPI_Finalized` will thus return ``false`` in any of these
116+
callback functions. Once you have done this with ``MPI_COMM_SELF``,
117+
the results of :ref:`MPI_Finalize` are not specified.
97118

98119
ERRORS
99120
------
100121

101122
.. include:: ./ERRORS.rst
102123

103-
.. seealso:: :ref:`MPI_Init`
124+
.. seealso::
125+
* :ref:`MPI_Finalized`
126+
* :ref:`MPI_Init`
127+
* :ref:`MPI_Initialized`
128+
* :ref:`MPI_Session_init`
129+
* :ref:`MPI_Session_finalize`

docs/man-openmpi/man3/MPI_Finalized.3.rst

Lines changed: 18 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ MPI_Finalized
55

66
.. include_body
77
8-
:ref:`MPI_Finalized` |mdash| Checks whether MPI has been finalized
8+
:ref:`MPI_Finalized` |mdash| Checks whether the MPI world model has been finalized
99

1010
SYNTAX
1111
------
@@ -45,20 +45,31 @@ Fortran 2008 Syntax
4545
OUTPUT PARAMETER
4646
----------------
4747

48-
* ``flag`` : True if MPI was finalized, and false otherwise (logical).
48+
* ``flag`` : True if the MPI world model was finalized, and false
49+
otherwise (logical).
4950
* ``ierror`` : Fortran only: Error status (integer).
5051

5152
DESCRIPTION
5253
-----------
5354

54-
This routine may be used to determine whether MPI has been finalized. It
55-
is one of a small number of routines that may be called before MPI is
56-
initialized and after MPI has been finalized (:ref:`MPI_Initialized` is
57-
another).
55+
This routine may be used to determine whether the MPI world model has
56+
been finalized. A different routine |mdash| :ref:`MPI_Initialized`
57+
|mdash| is used to indicate whether the MPI world model has been
58+
initialized.
59+
60+
See `MPI-5.0:11.4.1 <https://www.mpi-forum.org/>`_ for a list of MPI
61+
functionality that is available (e.g., even when the MPI
62+
world model has not yet initialized or has already been finalized).
5863

5964
ERRORS
6065
------
6166

6267
.. include:: ./ERRORS.rst
6368

64-
.. seealso:: :ref:`MPI_Init`
69+
.. seealso::
70+
* :ref:`MPI_Init`
71+
* :ref:`MPI_Init_thread`
72+
* :ref:`MPI_Finalize`
73+
* :ref:`MPI_Finalized`
74+
* :ref:`MPI_Session_init`
75+
* :ref:`MPI_Session_finalize`

docs/man-openmpi/man3/MPI_Init.3.rst

Lines changed: 38 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ MPI_Init
66

77
.. include_body
88
9-
:ref:`MPI_Init` |mdash| Initializes the MPI execution environment
9+
:ref:`MPI_Init` |mdash| Initializes the MPI world model
1010

1111

1212
SYNTAX
@@ -56,35 +56,53 @@ OUTPUT PARAMETER
5656
DESCRIPTION
5757
-----------
5858

59-
This routine, or :ref:`MPI_Init_thread`, must be called before most other MPI
60-
routines are called. There are a small number of errors, such as
61-
:ref:`MPI_Initialized` and :ref:`MPI_Finalized`. MPI can be initialized at most once;
62-
subsequent calls to :ref:`MPI_Init` or :ref:`MPI_Init_thread` are erroneous.
59+
This routine, or :ref:`MPI_Init_thread`, initializes the MPI world
60+
model. Either of these routines must be called before MPI
61+
communication routines are called within the MPI world model. The MPI
62+
world model can be initialized at most exactly once in the lifetime of
63+
an MPI process. This is different than the MPI session model, which
64+
can be initialized and finalized multiple times in an MPI process.
65+
See :ref:`MPI_Session_init` and :ref:`MPI_Session_finalize`.
6366

64-
All MPI programs must contain a call to :ref:`MPI_Init` or :ref:`MPI_Init_thread`.
65-
Open MPI accepts the C *argc* and *argv* arguments to main, but neither
66-
modifies, interprets, nor distributes them:
67+
See `MPI-5.0:11.4.1 <https://www.mpi-forum.org/>`_ for a list of MPI
68+
functionality that is available (e.g., even when the MPI
69+
world model has not yet initialized or has already been finalized).
70+
71+
Open MPI's :ref:`MPI_Init` and :ref:`MPI_Init_thread` both accept the
72+
C *argc* and *argv* arguments to main, but neither modifies,
73+
interprets, nor distributes them:
6774

6875
.. code-block:: c
6976
70-
/* declare variables */
71-
MPI_Init(&argc, &argv);
72-
/* parse arguments */
73-
/* main program */
74-
MPI_Finalize();
77+
#include <mpi.h>
7578
79+
int main(int argv, char *argv[]) {
80+
MPI_Init(&argc, &argv);
81+
/* ...body of main MPI pogram... */
82+
MPI_Finalize();
83+
return 0;
84+
}
85+
86+
By default, :ref:`MPI_Init` is effectively equivalent to invoking
87+
:ref:`MPI_Init_thread` with a *required* value of
88+
``MPI_THREAD_SINGLE``. However, if the ``OMPI_MPI_THREAD_LEVEL``
89+
environment variable is set to a valid value when :ref:`MPI_Init` is
90+
invoked, it is equivalent to invoking :ref:`MPI_Init_thread` with
91+
*required* set to the corresponding value of the ``OMPI_MPI_THREAD_LEVEL``
92+
environment variable. See :ref:`MPI_Init_thread` for more details.
7693

7794
NOTES
7895
-----
7996

8097
The Fortran version does not have provisions for *argc* and *argv* and
8198
takes only IERROR.
8299

83-
The MPI Standard does not say what a program can do before an :ref:`MPI_Init`
84-
or after an :ref:`MPI_Finalize`. In the Open MPI implementation, it should do
85-
as little as possible. In particular, avoid anything that changes the
86-
external state of the program, such as opening files, reading standard
87-
input, or writing to standard output.
100+
The MPI Standard does not specify what a program using the MPI world
101+
model can do before invoking :ref:`MPI_Init` or :ref:`MPI_Init_thread`
102+
or after invoking :ref:`MPI_Finalize`. In the Open MPI implementation,
103+
it should do as little as possible. In particular, avoid anything that
104+
changes the external state of the program, such as opening files,
105+
reading standard input, or writing to standard output.
88106

89107

90108
ERRORS
@@ -97,3 +115,5 @@ ERRORS
97115
* :ref:`MPI_Initialized`
98116
* :ref:`MPI_Finalize`
99117
* :ref:`MPI_Finalized`
118+
* :ref:`MPI_Session_finalize`
119+
* :ref:`MPI_Session_init`

0 commit comments

Comments
 (0)