Manual domain decomposition in parallel PHOENICS
In most cases, it is sufficient to use the automatic domain decompostion which is provided for within the parallel PHOENICS solver. In some cases though, the user may prefer to choose how the domain is partitioned, ie apply a manual decompostion. To do this the user should supply a file PARDAT in the working directory which describes how the domain is to be split between the processes.
Description of file PARDAT
File PARDAT serves the manual decomposition in parallel run of PHOENICS. If it is absent then the automatic decomposition is made.
The format of file PARDAT depends on kind of decomposition:
1. Decomposition on equal sizes in selected directions of one domain
First type of decomposition has very simple format of file PARDAT. User should set three lines,
NXSD = 1 NYSD = 1 NZSD = 2
First line shows that number of sub-domains in X-direction is one, domain is not split in X-direction. Second line shows that number of sub-domains in Y-direction is one too. Third line shows that number of sub-domains in Y-direction is two, domain is split in Z direction on two equal (or almost equal) parts.
This simple type of manual decomposition may alternatively be achieved by adding settings to the Q1 file.
For example, to split the domain into 8 sub-domains (2 in each direction), the following arrays must be set in the Q1 file:
LG(2)=T IG(1)=2 IG(2)=2 IG(3)=2
The logical LG(2) will instruct the splitter to by-pass the automatic domain decomposition, and split the domain according to the settings defined in the IG array as follows.
IG(1) specifies the number of sub-domains in the x-direction;
2. Decomposition on arbitrary sizes in selected directions of one domain
Second type of decomposition allows dividing of domain on non-equal parts. It is helpful
sometimes when user has PCs with non-equal parameters connected by network. Therefore for better
performance user wants to set more cells to more powerful PC. In this case user should set three
lines, for example:
NXSD = 1 38 NYSD = 2 36 16 NZSD = 1 28
The first numbers after sign ‘=’ are numbers of sub-domains in current direction. Next number (or numbers) is number of cells in current direction for each sub-domain. Sum of cells for all sub-domains should be equal the total number of cells in each direction. The sample demonstrates case when there is not splitting in X and Z direction (number of cells in X-direction is 38, number of cells in Z-direction is 28) and domain is divided in Y direction on two sub-domain, first has 36 cells and second has 16 cells in Y direction.
3. Decomposition of Multi-Block domain
Third type of decomposition deals with Multi-Block cases. In this case the first line should be
MULTIBLOCK = T
The comment line should be started from sign’*’. Next lines demonstrates the sample of well-commented PARDAT file:
NUMBLOCKS = 5 NUMPROCS = 3 * description of first block * firstly we set current block * CUR_BLOCK =