Manual domain decomposition in parallel PHOENICS
In most cases, it is sufficient to use the automatic domain decompostion which is provided
for within the parallel PHOENICS solver. In some cases though, the user may prefer to choose
how the domain is partitioned, ie apply a manual decompostion. To do this the user should
supply a file PARDAT in the working directory which describes how the domain is to be split
between the processes.
File PARDAT serves the manual decomposition in parallel run of PHOENICS. If it is absent then
the automatic decomposition is made.
The format of file PARDAT depends on kind of decomposition:
 Decomposition on equal sizes in selected directions of one domain
 Decomposition on arbitrary sizes in selected directions of one domain
 Decomposition of MultiBlock domain
1. Decomposition on equal sizes in selected directions of one domain
First type of decomposition has very simple format of file PARDAT. User should set three lines,
for example:
NXSD = 1
NYSD = 1
NZSD = 2
First line shows that number of subdomains in Xdirection is one, domain is not split in
Xdirection. Second line shows that number of subdomains in Ydirection is one too. Third line
shows that number of subdomains in Ydirection is two, domain is split in Z direction on two
equal (or almost equal) parts.
This simple type of manual decomposition may alternatively be achieved by adding settings to
the Q1 file.
For example, to split the domain into 8 subdomains (2 in each direction), the following arrays
must be set in the Q1 file:
LG(2)=T
IG(1)=2
IG(2)=2
IG(3)=2
The logical LG(2) will instruct the splitter to bypass the automatic domain decomposition,
and split the domain according to the settings defined in the IG array as follows.
IG(1) specifies the number of subdomains in the xdirection;
IG(2) specifies the number of subdomains in the ydirection;
IG(3) specifies the number of subdomains in the zdirection.
2. Decomposition on arbitrary sizes in selected directions of one domain
Second type of decomposition allows dividing of domain on nonequal parts. It is helpful
sometimes when user has PCs with nonequal parameters connected by network. Therefore for better
performance user wants to set more cells to more powerful PC. In this case user should set three
lines, for example:
NXSD = 1 38
NYSD = 2 36 16
NZSD = 1 28
The first numbers after sign ‘=’ are numbers of subdomains in current direction. Next number
(or numbers) is number of cells in current direction for each subdomain. Sum of cells for all
subdomains should be equal the total number of cells in each direction. The sample demonstrates
case when there is not splitting in X and Z direction (number of cells in Xdirection is 38,
number of cells in Zdirection is 28) and domain is divided in Y direction on two subdomain,
first has 36 cells and second has 16 cells in Y direction.
3. Decomposition of MultiBlock domain
Third type of decomposition deals with MultiBlock cases. In this case the first line should be
MULTIBLOCK = T
The comment line should be started from sign’*’. Next lines demonstrates the sample of
wellcommented PARDAT file:
NUMBLOCKS = 5
NUMPROCS = 3
* description of first block
* firstly we set current block
* CUR_BLOCK =
CUR_BLOCK = 1 2
* secondly we enumerate process for current block
* PROC =
PROC = 1 0
* for each process we should set global BOUND BOX (for current block)
* BOUND_BOX =
BOUND_BOX = 1 16 1 16 1 5
PROC = 2 1
BOUND_BOX = 1 16 1 16 6 9
*
* next blocks here
* block 2 in process 2
CUR_BLOCK = 2 2
PROC = 1 1
BOUND_BOX = 1 16 1 16 11 14
PROC = 2 2
BOUND_BOX = 1 16 1 16 15 19
* block 3 in process 0 and 1
CUR_BLOCK = 3 1
PROC = 1 1
BOUND_BOX = 1 4 1 4 21 30
* block 4 in processes 0, 1 and 2
CUR_BLOCK = 4 1
PROC = 1 1
BOUND_BOX = 1 4 1 4 32 61
* block 5 in processes 1 and 2
CUR_BLOCK = 5 1
PROC = 1 1
BOUND_BOX = 1 4 1 4 63 72
