[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: DAcreate2d process layout order


I would say your original interpretation is correct. The DA
distribution style is not compatible with the way MPI_cart_create()
works.

Once I get Bill's confirmation I'll change the DA; for now I
think my suggestion of just swapping the meaning of x and y (i and j)
when dealing with DAs will have the same effect.

  Barry

Bill, does the MPI standard dictate this decomposition or
could different implementations do it the opposite way?
Then we'd have to make the DA logic a bit more complicated.


On Thu, 18 May 2006, Sean Dettrick wrote:

Barry Smith wrote:

On Thu, 18 May 2006, Sean Dettrick wrote:

Hi Barry,
the order is determined by MPI_Cart_create.



   Do you mean that MPI_Cart_create() orders across the 2nd (y-axis)
fastest and then the first (x-axis)? Hmmm, maybe we should change the
DA? Changing it once and for all (not supporting both) is probably
not a big deal and shouldn't break much (I hope).


Hi Barry,

it depends, what do you call x and what do you call y?
MPI_Cart_coords returns a vector, coords - I tend to say x is coords[0], y is coords[1] and z is coords[2]. For what it's worth, there's a short code appended to this email, which produces:

rank = 0 has Cartesian coords = { 0, 0 }
rank = 1 has Cartesian coords = { 0, 1 }
rank = 2 has Cartesian coords = { 1, 0 }
rank = 3 has Cartesian coords = { 1, 1 }
rank = 0 has DA range x=[0,50) and y=[0,50)
rank = 1 has DA range x=[50,100) and y=[0,50)
rank = 2 has DA range x=[0,50) and y=[50,100)
rank = 3 has DA range x=[50,100) and y=[50,100)

I don't completely understand what goes wrong. Is it because YOUR
application orders the processors related to geometry in the following way?

    ^ y direction
    |
       2   5  8
       1   4  7
       0   3  6

                    -> x direction

Or is this something inherent in MPI_Cart_create?


For my interpretation of x and y, MPI_Cart_create produces the above layout. But if I said x=coords[1] and y=coords[0], then it would match the one below.


PETSc does it so

    ^ y direction
    |
       6   7  8
       3   4  5
       0   1  2

                    -> x direction




Code and makefile attached ... hopefully within the message size limit.
Just make cartcommtest.

Sean