[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: DAcreate2d process layout order

Barry Smith wrote:


Bill, does the MPI standard dictate this decomposition or
could different implementations do it the opposite way?
Then we'd have to make the DA logic a bit more complicated.

I don't have a copy of the standard, but to quote page 255 of "MPI, the complete reference" by Snir et al:
"Row-major numbering is always used for the processes in a Cartesian structure".
Their diagram in figure 6.1 matches my code output for coords couplets (i,j):

 0      1       2       3
(0,0)  (0,1)   (0,2)   (0,3)

 4      5       6       7
(1,0)  (1,1)   (1,2)   (1,3)

 8      9      10      11
(2,0)  (2,1)   (2,2)   (2,3)


By the way I agree with you, I *should* be able to swap the x and y myself. Just haven't had much luck yet in that regard.

Sean



On Thu, 18 May 2006, Sean Dettrick wrote:

Barry Smith wrote:

On Thu, 18 May 2006, Sean Dettrick wrote:

Hi Barry,
the order is determined by MPI_Cart_create.




   Do you mean that MPI_Cart_create() orders across the 2nd (y-axis)
fastest and then the first (x-axis)? Hmmm, maybe we should change the
DA? Changing it once and for all (not supporting both) is probably
not a big deal and shouldn't break much (I hope).



Hi Barry,

it depends, what do you call x and what do you call y?
MPI_Cart_coords returns a vector, coords - I tend to say x is coords[0], y is coords[1] and z is coords[2]. For what it's worth, there's a short code appended to this email, which produces:

rank = 0 has Cartesian coords = { 0, 0 }
rank = 1 has Cartesian coords = { 0, 1 }
rank = 2 has Cartesian coords = { 1, 0 }
rank = 3 has Cartesian coords = { 1, 1 }
rank = 0 has DA range x=[0,50) and y=[0,50)
rank = 1 has DA range x=[50,100) and y=[0,50)
rank = 2 has DA range x=[0,50) and y=[50,100)
rank = 3 has DA range x=[50,100) and y=[50,100)

I don't completely understand what goes wrong. Is it because YOUR
application orders the processors related to geometry in the following way?

    ^ y direction
    |
       2   5  8
       1   4  7
       0   3  6

                    -> x direction

Or is this something inherent in MPI_Cart_create?



For my interpretation of x and y, MPI_Cart_create produces the above layout. But if I said x=coords[1] and y=coords[0], then it would match the one below.


PETSc does it so

    ^ y direction
    |
       6   7  8
       3   4  5
       0   1  2

                    -> x direction




Code and makefile attached ... hopefully within the message size limit.
Just make cartcommtest.

Sean