deal.II version GIT relicensing2076g6b43d56a25 20241102 12:30:00+00:00

#include <deal.II/lac/petsc_vector.h>
Public Types  
using  size_type = types::global_dof_index 
using  value_type = PetscScalar 
using  real_type = PetscReal 
using  reference = internal::VectorReference 
using  const_reference = const internal::VectorReference 
Static Public Member Functions  
static ::ExceptionBase &  ExcInUse (int arg1, std::string arg2, std::string arg3) 
static ::ExceptionBase &  ExcNoSubscriber (std::string arg1, std::string arg2) 
Protected Member Functions  
virtual void  create_vector (const MPI_Comm comm, const size_type n, const size_type locally_owned_size) 
virtual void  create_vector (const MPI_Comm comm, const size_type n, const size_type locally_owned_size, const IndexSet &ghostnodes) 
void  do_set_add_operation (const size_type n_elements, const size_type *indices, const PetscScalar *values, const bool add_values) 
void  determine_ghost_indices () 
Protected Attributes  
Vec  vector 
bool  ghosted 
IndexSet  ghost_indices 
VectorOperation::values  last_action 
Private Types  
using  map_value_type = decltype(counter_map)::value_type 
using  map_iterator = decltype(counter_map)::iterator 
Private Member Functions  
void  check_no_subscribers () const noexcept 
Private Attributes  
std::atomic< unsigned int >  counter 
std::map< std::string, unsigned int >  counter_map 
std::vector< std::atomic< bool > * >  validity_pointers 
const std::type_info *  object_info 
Static Private Attributes  
static std::mutex  mutex 
Related Symbols  
(Note that these are not member symbols.)  
void  swap (Vector &u, Vector &v) noexcept 
void  swap (VectorBase &u, VectorBase &v) noexcept 
Implementation of a parallel vector class based on PETSC and using MPI communication to synchronize distributed operations. All the functionality is actually in the base class, except for the calls to generate a parallel vector. This is possible since PETSc only works on an abstract vector type and internally distributes to functions that do the actual work depending on the actual vector type (much like using virtual functions). Only the functions creating a vector of specific type differ, and are implemented in this particular class.
The parallel functionality of PETSc is built on top of the Message Passing Interface (MPI). MPI's communication model is built on collective communications: if one process wants something from another, that other process has to be willing to accept this communication. A process cannot query data from another process by calling a remote function, without that other process expecting such a transaction. The consequence is that most of the operations in the base class of this class have to be called collectively. For example, if you want to compute the l2 norm of a parallel vector, all processes across which this vector is shared have to call the l2_norm
function. If you don't do this, but instead only call the l2_norm
function on one process, then the following happens: This one process will call one of the collective MPI functions and wait for all the other processes to join in on this. Since the other processes don't call this function, you will either get a timeout on the first process, or, worse, by the time the next a call to a PETSc function generates an MPI message on the other processes, you will get a cryptic message that only a subset of processes attempted a communication. These bugs can be very hard to figure out, unless you are wellacquainted with the communication model of MPI, and know which functions may generate MPI messages.
One particular case, where an MPI message may be generated unexpectedly is discussed below.
PETSc does allow read access to individual elements of a vector, but in the distributed case only to elements that are stored locally. We implement this through calls like d=vec(i)
. However, if you access an element outside the locally stored range, an exception is generated.
In contrast to read access, PETSc (and the respective deal.II wrapper classes) allow to write (or add) to individual elements of vectors, even if they are stored on a different process. You can do this writing, for example, vec(i)=d
or vec(i)+=d
, or similar operations. There is one catch, however, that may lead to very confusing error messages: PETSc requires application programs to call the compress() function when they switch from adding, to elements to writing to elements. The reasoning is that all processes might accumulate addition operations to elements, even if multiple processes write to the same elements. By the time we call compress() the next time, all these additions are executed. However, if one process adds to an element, and another overwrites to it, the order of execution would yield nondeterministic behavior if we don't make sure that a synchronization with compress() happens in between.
In order to make sure these calls to compress() happen at the appropriate time, the deal.II wrappers keep a state variable that store which is the presently allowed operation: additions or writes. If it encounters an operation of the opposite kind, it calls compress() and flips the state. This can sometimes lead to very confusing behavior, in code that may for example look like this:
This code can run into trouble: by the time we see the first addition operation, we need to flush the overwrite buffers for the vector, and the deal.II library will do so by calling compress(). However, it will only do so for all processes that actually do an addition – if the condition is never true for one of the processes, then this one will not get to the actual compress() call, whereas all the other ones do. This gets us into trouble, since all the other processes hang in the call to flush the write buffers, while the one other process advances to the call to compute the l2 norm. At this time, you will get an error that some operation was attempted by only a subset of processes. This behavior may seem surprising, unless you know that write/addition operations on single elements may trigger this behavior.
The problem described here may be avoided by placing additional calls to compress(), or making sure that all processes do the same type of operations at the same time, for example by placing zero additions if necessary.
Definition at line 157 of file petsc_vector.h.
Declare type for container size.
Definition at line 163 of file petsc_vector.h.

inherited 
Declare some of the standard types used in all containers. These types parallel those in the C++
standard libraries vector<...>
class.
Definition at line 259 of file petsc_vector_base.h.

inherited 
Definition at line 260 of file petsc_vector_base.h.

inherited 
Definition at line 262 of file petsc_vector_base.h.

inherited 
Definition at line 263 of file petsc_vector_base.h.

privateinherited 
The data type used in counter_map.
Definition at line 238 of file enable_observer_pointer.h.

privateinherited 
The iterator type used in counter_map.
Definition at line 243 of file enable_observer_pointer.h.
Default constructor. Initialize the vector as empty.
Definition at line 38 of file petsc_parallel_vector.cc.

explicit 
Constructor. Set dimension to n
and initialize all elements with zero.
The constructor is made explicit to avoid accidents like this: v=0;
. Presumably, the user wants to set every element of the vector to zero, but instead, what happens is this call: v=Vector<number>(0);
, i.e. the vector is replaced by one of length zero.
Definition at line 48 of file petsc_parallel_vector.cc.

explicit 
Copyconstructor from deal.II vectors. Sets the dimension to that of the given vector, and copies all elements.
Vector< Number >::Vector  (  const IndexSet &  local, 
const IndexSet &  ghost,  
const MPI_Comm  communicator  
) 
Construct a new parallel ghosted PETSc vector from IndexSets.
Note that local
must be ascending and 1:1, see IndexSet::is_ascending_and_one_to_one(). In particular, the DoFs in local
need to be contiguous, meaning you can only create vectors from a DoFHandler with several finite element components if they are not reordered by component (use a PETScWrappers::BlockVector otherwise). The global size of the vector is determined by local.size(). The global indices in ghost
are supplied as ghost indices so that they can be read locally.
Note that the ghost
IndexSet may be empty and that any indices already contained in local
are ignored during construction. That way, the ghost parameter can equal the set of locally relevant degrees of freedom, see step32.
Definition at line 57 of file petsc_parallel_vector.cc.
Construct a new parallel PETSc vector without ghost elements from an IndexSet.
Note that local
must be ascending and 1:1, see IndexSet::is_ascending_and_one_to_one(). In particular, the DoFs in local
need to be contiguous, meaning you can only create vectors from a DoFHandler with several finite element components if they are not reordered by component (use a PETScWrappers::BlockVector otherwise).
Definition at line 93 of file petsc_parallel_vector.cc.
Copy constructor.
Definition at line 75 of file petsc_parallel_vector.cc.
Release all memory and return to a state just like after having called the default constructor.
Reimplemented from PETScWrappers::VectorBase.
Definition at line 145 of file petsc_parallel_vector.cc.
Vector & Vector< Number >::operator=  (  const Vector &  v  ) 
Copy the given vector. Resize the present vector if necessary. Also take over the MPI communicator of v
.
The semantics of this operator are complex. If the two vectors have the same size, and if either the left or right hand side vector of the assignment (i.e., either the input vector on the right hand side, or the calling vector to the left of the assignment operator) currently has ghost elements, then the left hand side vector will also have ghost values and will consequently be a readonly vector (see also the glossary entry on the issue). Otherwise, the left hand vector will be a writable vector after this operation. These semantics facilitate having a vector with ghost elements on the left hand side of the assignment, and a vector without ghost elements on the right hand side, with the resulting left hand side vector having the correct values in both its locally owned and its ghost elements.
On the other hand, if the left hand side vector does not have the correct size yet, or is perhaps an entirely uninitialized vector, then the assignment is simply a copy operation in the usual sense: In that case, if the right hand side has no ghost elements (i.e., is a completely distributed vector), then the left hand side will have no ghost elements either. And if the right hand side has ghost elements (and is consequently readonly), then the left hand side will have these same properties after the operation.
Definition at line 103 of file petsc_parallel_vector.cc.
Set all components of the vector to the given number s
. Simply pass this down to the base class, but we still need to declare this function to make the example given in the discussion about making the constructor explicit work.
Vector & PETScWrappers::MPI::Vector::operator=  (  const ::Vector< number > &  v  ) 
Copy the values of a deal.II vector (as opposed to those of the PETSc vector wrapper class) into this object.
Contrary to the case of sequential vectors, this operators requires that the present vector already has the correct size, since we need to have a partition and a communicator present which we otherwise can't get from the source vector.
void Vector< Number >::reinit  (  const MPI_Comm  communicator, 
const size_type  N,  
const size_type  locally_owned_size,  
const bool  omit_zeroing_entries = false 

) 
Change the dimension of the vector to N
. It is unspecified how resizing the vector affects the memory allocation of this object; i.e., it is not guaranteed that resizing it to a smaller size actually also reduces memory consumption, or if for efficiency the same amount of memory is used
locally_owned_size
denotes how many of the N
values shall be stored locally on the present process. for less data.
communicator
denotes the MPI communicator henceforth to be used for this vector.
If omit_zeroing_entries
is false, the vector is filled by zeros. Otherwise, the elements are left an unspecified state.
Definition at line 155 of file petsc_parallel_vector.cc.
Change the dimension to that of the vector v
, and also take over the partitioning into local sizes as well as the MPI communicator. The same applies as for the other reinit
function.
The elements of v
are not copied, i.e. this function is the same as calling reinit(v.size(), v.locally_owned_size(), omit_zeroing_entries)
.
Definition at line 196 of file petsc_parallel_vector.cc.
void Vector< Number >::reinit  (  const IndexSet &  local, 
const IndexSet &  ghost,  
const MPI_Comm  communicator  
) 
Reinit as a vector with ghost elements. See the constructor with same signature for more details.
Definition at line 219 of file petsc_parallel_vector.cc.
Reinit as a vector without ghost elements. See constructor with same signature for more details.
Definition at line 235 of file petsc_parallel_vector.cc.
void Vector< Number >::reinit  (  const std::shared_ptr< const Utilities::MPI::Partitioner > &  partitioner, 
const bool  make_ghosted = true 

) 
Initialize the vector given to the parallel partitioning described in partitioner
.
You can decide whether your vector will contain ghost elements with make_ghosted
.
Definition at line 246 of file petsc_parallel_vector.cc.
void Vector< Number >::print  (  std::ostream &  out, 
const unsigned int  precision = 3 , 

const bool  scientific = true , 

const bool  across = true 

)  const 
Print to a stream. precision
denotes the desired precision with which values shall be printed, scientific
whether scientific notation shall be used. If across
is true
then the vector is printed in a line, while if false
then the elements are printed on a separate line each.
Definition at line 364 of file petsc_parallel_vector.cc.
Return whether the vector contains only elements with value zero. This is a collective operation. This function is expensive, because potentially all elements have to be checked.
Definition at line 352 of file petsc_parallel_vector.cc.

protectedvirtual 
Create a vector of length n
. For this class, we create a parallel vector. n
denotes the total size of the vector to be created. locally_owned_size
denotes how many of these elements shall be stored locally.
Definition at line 269 of file petsc_parallel_vector.cc.

protectedvirtual 
Create a vector of global length n
, local size locally_owned_size
and with the specified ghost indices. Note that you need to call update_ghost_values() before accessing those.
Definition at line 289 of file petsc_parallel_vector.cc.
PETScWrappers::VectorBase::VectorBase  (  ) 
Import VectorBase constructors, including from a PETSc Vec object.
Definition at line 269 of file petsc_vector_base.cc.
PETScWrappers::VectorBase::VectorBase  (  const VectorBase &  v  ) 
Import VectorBase constructors, including from a PETSc Vec object.
Definition at line 275 of file petsc_vector_base.cc.

explicit 
Import VectorBase constructors, including from a PETSc Vec object.
Definition at line 281 of file petsc_vector_base.cc.
void PETScWrappers::VectorBase::reinit  (  Vec  v  ) 
This method associates the PETSc Vec to the instance of the class. This is particularly useful when performing PETSc to Deal.II operations since it allows to reuse the Deal.II VectorBase and the PETSc Vec without incurring in memory copies.
Definition at line 337 of file petsc_vector_base.cc.

inherited 
Compress the underlying representation of the PETSc object, i.e. flush the buffers of the vector object if it has any. This function is necessary after writing into a vector elementbyelement and before anything else can be done on it.
See Compressing distributed objects for more information.
Definition at line 538 of file petsc_vector_base.cc.

inherited 
Test for equality. This function assumes that the present vector and the one to compare with have the same size already, since comparing vectors of different sizes makes not much sense anyway.
Definition at line 397 of file petsc_vector_base.cc.

inherited 
Test for inequality. This function assumes that the present vector and the one to compare with have the same size already, since comparing vectors of different sizes makes not much sense anyway.
Definition at line 411 of file petsc_vector_base.cc.

overridevirtualinherited 
Return the global dimension of the vector.
Implements ReadVector< PetscScalar >.
Definition at line 425 of file petsc_vector_base.cc.

inherited 
Return the local dimension of the vector, i.e. the number of elements stored on the present MPI process. For sequential vectors, this number is the same as size(), but for parallel vectors it may be smaller.
To figure out which elements exactly are stored locally, use local_range() or locally_owned_elements().
Definition at line 437 of file petsc_vector_base.cc.

inherited 
Return a pair of indices indicating which elements of this vector are stored locally. The first number is the index of the first element stored, the second the index of the one past the last one that is stored locally. If this is a sequential vector, then the result will be the pair (0,N), otherwise it will be a pair (i,i+n), where n=locally_owned_size()
.
Definition at line 449 of file petsc_vector_base.cc.
Return whether index
is in the local range or not, see also local_range().

inherited 
Return an index set that describes which elements of this vector are owned by the current processor. Note that this index set does not include elements this vector may store locally as ghost elements but that are in fact owned by another processor. As a consequence, the index sets returned on different processors if this is a distributed vector will form disjoint sets that add up to the complete index set. Obviously, if a vector is created on only one processor, then the result would satisfy

inherited 
Return if the vector contains ghost elements.
Return the IndexSet of ghost elements.

inherited 
Update ghosted elements.
Provide access to a given element, both read and write.
Provide readonly access to an element.
Provide access to a given element, both read and write.
Exactly the same as operator().
Provide readonly access to an element.
Exactly the same as operator().

inherited 
A collective set operation: instead of setting individual elements of a vector, this function allows to set a whole set of elements at once. The indices of the elements to be set are stated in the first argument, the corresponding values in the second.
Definition at line 462 of file petsc_vector_base.cc.

inherited 
Instead of getting individual elements of a vector via operator(), this function allows getting a whole set of elements at once. The indices of the elements to be read are stated in the first argument, the corresponding values are returned in the second.
If the current vector is called v
, then this function is the equivalent to the code
indices
and values
arrays must be identical.

overridevirtualinherited 
Extract a range of elements all at once.
Implements ReadVector< PetscScalar >.

inherited 
Instead of getting individual elements of a vector via operator(), this function allows getting a whole set of elements at once. In contrast to the previous function, this function obtains the indices of the elements by dereferencing all elements of the iterator range provided by the first two arguments, and puts the vector values into memory locations obtained by dereferencing a range of iterators starting at the location pointed to by the third argument.
If the current vector is called v
, then this function is the equivalent to the code
values_begin
as there are iterators between indices_begin
and indices_end
.

inherited 
A collective add operation: This function adds a whole set of values stored in values
to the vector components specified by indices
.
Definition at line 473 of file petsc_vector_base.cc.

inherited 
This is a second collective add operation. As a difference, this function takes a deal.II vector of values.
Definition at line 484 of file petsc_vector_base.cc.

inherited 
Take an address where n_elements
are stored contiguously and add them into the vector. Handles all cases which are not covered by the other two add()
functions above.
Definition at line 495 of file petsc_vector_base.cc.
Addition of s
to all components. Note that s
is a scalar and not a vector.
Definition at line 857 of file petsc_vector_base.cc.

inherited 
Simple addition of a multiple of a vector, i.e. *this += a*V
.
Definition at line 869 of file petsc_vector_base.cc.

inherited 
Multiple addition of scaled vectors, i.e. *this += a*V+b*W
.
Definition at line 881 of file petsc_vector_base.cc.

inherited 
Return the scalar product of two vectors. The vectors must have the same size.
For complex valued vector, this gives \(\left(v^\ast,vec\right)\).
Definition at line 505 of file petsc_vector_base.cc.

inherited 
Return the square of the \(l_2\)norm.
Definition at line 602 of file petsc_vector_base.cc.

inherited 
Return the mean value of the elements of this vector.
Definition at line 611 of file petsc_vector_base.cc.

inherited 
\(l_1\)norm of the vector. The sum of the absolute values.
Definition at line 664 of file petsc_vector_base.cc.

inherited 
\(l_2\)norm of the vector. The square root of the sum of the squares of the elements.
Definition at line 677 of file petsc_vector_base.cc.

inherited 
\(l_p\)norm of the vector. The pth root of the sum of the pth powers of the absolute values of the elements.
Definition at line 690 of file petsc_vector_base.cc.

inherited 
\(l_\infty\)norm of the vector. Return the value of the vector element with the maximum absolute value.
Definition at line 734 of file petsc_vector_base.cc.

inherited 
Performs a combined operation of a vector addition and a subsequent inner product, returning the value of the inner product. In other words, the result of this function is the same as if the user called
The reason this function exists is for compatibility with deal.II's own vector classes which can implement this functionality with less memory transfer. However, for PETSc vectors such a combined operation is not natively supported and thus the cost is completely equivalent as calling the two methods separately.
For complexvalued vectors, the scalar product in the second step is implemented as \(\left<v,w\right>=\sum_i v_i \bar{w_i}\).
Definition at line 527 of file petsc_vector_base.cc.

inherited 
Multiply the entire vector by a fixed factor.
Definition at line 802 of file petsc_vector_base.cc.

inherited 
Divide the entire vector by a fixed factor.
Definition at line 816 of file petsc_vector_base.cc.

inherited 
Add the given vector to the present one.
Definition at line 833 of file petsc_vector_base.cc.

inherited 
Subtract the given vector from the present one.
Definition at line 845 of file petsc_vector_base.cc.

inherited 
Scaling and simple vector addition, i.e. this = s(*this)+V
.
Definition at line 900 of file petsc_vector_base.cc.

inherited 
Scaling and simple addition, i.e. this = s(*this)+a*V
.
Definition at line 912 of file petsc_vector_base.cc.

inherited 
Scale each element of this vector by the corresponding element in the argument. This function is mostly meant to simulate multiplication (and immediate reassignment) by a diagonal scaling matrix.
Definition at line 930 of file petsc_vector_base.cc.

inherited 
Assignment *this = a*V
.
Definition at line 940 of file petsc_vector_base.cc.

inherited 
Prints the PETSc vector object values using PETSc internal vector viewer function VecView
. The default format prints the vector's contents, including indices of vector elements. For other valid view formats, consult http://www.mcs.anl.gov/petsc/petsccurrent/docs/manualpages/Vec/VecView.html
Definition at line 954 of file petsc_vector_base.cc.
Write the data of this object to a stream for the purpose of serialization using the BOOST serialization library.
Read the data of this object from a stream for the purpose of serialization using the BOOST serialization library.

inherited 
Write and read the data of this object from a stream for the purpose of serialization using the BOOST serialization library.

noexceptinherited 
Swap the contents of this vector and the other vector v
. One could do this operation with a temporary variable and copying over the data elements, but this function is significantly more efficient since it only swaps the pointers to the data of the two vectors and therefore does not need to allocate temporary storage and move data around.
This function is analogous to the swap
function of all C++ standard containers. Also, there is a global function swap(u,v)
that simply calls u.swap(v)
, again in analogy to standard functions.
Definition at line 1021 of file petsc_vector_base.cc.

inherited 
Conversion operator to gain access to the underlying PETSc type. If you do this, you cut this class off some information it may need, so this conversion operator should only be used if you know what you do. In particular, it should only be used for readonly operations into the vector.
Definition at line 1033 of file petsc_vector_base.cc.

inherited 
Return a reference to the underlying PETSc type. It can be used to modify the underlying data, so use it only when you know what you are doing.
Definition at line 1040 of file petsc_vector_base.cc.

inherited 
Estimate for the memory consumption (not implemented for this class).
Definition at line 1047 of file petsc_vector_base.cc.

inherited 
Return the underlying MPI communicator.

protectedinherited 
Collective set or add operation: This function is invoked by the collective set
and add
with the add_values
flag set to the corresponding value.
Definition at line 1069 of file petsc_vector_base.cc.

protectedinherited 
Determine ghost indices from the internal PETSc Vec
Definition at line 240 of file petsc_vector_base.cc.

inherited 
Subscribes a user of the object by storing the pointer validity
. The subscriber may be identified by text supplied as identifier
.
Definition at line 131 of file enable_observer_pointer.cc.

inherited 
Unsubscribes a user from the object.
identifier
and the validity
pointer must be the same as the one supplied to subscribe(). Definition at line 151 of file enable_observer_pointer.cc.

inlineinherited 
Return the present number of subscriptions to this object. This allows to use this class for reference counted lifetime determination where the last one to unsubscribe also deletes the object.
Definition at line 322 of file enable_observer_pointer.h.

inlineinherited 
List the subscribers to the input stream
.
Definition at line 339 of file enable_observer_pointer.h.

inherited 
List the subscribers to deallog
.
Definition at line 199 of file enable_observer_pointer.cc.

privatenoexceptinherited 
Check that there are no objects subscribing to this object. If this check passes then it is safe to destroy the current object. It this check fails then this function will either abort or print an error message to deallog (by using the AssertNothrow mechanism), but will not throw an exception.
Definition at line 53 of file enable_observer_pointer.cc.
Global function swap
which overloads the default implementation of the C++ standard library which uses a temporary object. The function simply exchanges the data of the two vectors.
Definition at line 449 of file petsc_vector.h.

related 
Global function swap
which overloads the default implementation of the C++ standard library which uses a temporary object. The function simply exchanges the data of the two vectors.
Definition at line 868 of file petsc_vector_base.h.

protectedinherited 
A generic vector object in PETSc. The actual type, a sequential vector, is set in the constructor.
Definition at line 812 of file petsc_vector_base.h.

protectedinherited 
Denotes if this vector has ghost indices associated with it. This means that at least one of the processes in a parallel program has at least one ghost index.
Definition at line 819 of file petsc_vector_base.h.

protectedinherited 
This vector contains the global indices of the ghost values. The location in this vector denotes the local numbering, which is used in PETSc.
Definition at line 826 of file petsc_vector_base.h.

mutableprotectedinherited 
Store whether the last action was a write or add operation. This variable is mutable
so that the accessor classes can write to it, even though the vector object they refer to is constant.
Definition at line 833 of file petsc_vector_base.h.

mutableprivateinherited 
Store the number of objects which subscribed to this object. Initially, this number is zero, and upon destruction it shall be zero again (i.e. all objects which subscribed should have unsubscribed again).
The creator (and owner) of an object is counted in the map below if HE manages to supply identification.
We use the mutable
keyword in order to allow subscription to constant objects also.
This counter may be read from and written to concurrently in multithreaded code: hence we use the std::atomic
class template.
Definition at line 227 of file enable_observer_pointer.h.

mutableprivateinherited 
In this map, we count subscriptions for each different identification string supplied to subscribe().
Definition at line 233 of file enable_observer_pointer.h.

mutableprivateinherited 
In this vector, we store pointers to the validity bool in the ObserverPointer objects that subscribe to this class.
Definition at line 249 of file enable_observer_pointer.h.

mutableprivateinherited 
Pointer to the typeinfo object of this object, from which we can later deduce the class name. Since this information on the derived class is neither available in the destructor, nor in the constructor, we obtain it in between and store it here.
Definition at line 257 of file enable_observer_pointer.h.

staticprivateinherited 
A mutex used to ensure data consistency when accessing the mutable
members of this class. This lock is used in the subscribe() and unsubscribe() functions, as well as in list_subscribers()
.
Definition at line 280 of file enable_observer_pointer.h.