[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: GMRES left-preconditioned with ILU1 versus ILU0


When monkeying with ILU it is always useful to run tests with - ksp_monitor_true_residual, because bad pivots in ILU
can produce wildly scaled preconditioners so the preconditioned residual can be at a different scale then the actual residual.

The block versions of the factorizations don't support the pc_factor_shiftXXXX options. You could try to add it.

That is one nasty matrix, I don't know if you'll ever get reasonable performance with ILU. You may want to stick to
direct solvers on each process (it may be slow and use lots of memory but at least it might work).

  Barry


On May 16, 2008, at 11:22 AM, Stephane Aubert wrote:

Hi,
I tried to improve the convergence of a rather badly conditioned linear system by increasing the ILU(k) level from k=0 to k=1.
And, whereas I got a convergence for k=0, I ended up with an explosive divergence for k=1!!!
The PETCS version is 2.3.2-p8.
The matrix type is MPIBAIJ (block size=7,mesh points=23010,non-empty blocks=297454)
The common command line options are:

* KSP="-ksp_type gmres -ksp_max_it 800 -ksp_gmres_restart 800
-ksp_rtol 1.0e-12 -ksp_left_pc -ksp_gmres_modifiedgramschmidt
-ksp_gmres_cgs_refinement_type REFINE_NEVER -ksp_singmonitor
-ksp_compute_eigenvalues": I'm forcing 800 krylovs without restart
to get the condition number of the pre-conditioned system (if I
understand correctly the man page of -ksp_singmonitor)
* PC="-pc_type asm -pc_asm_overlap 2": I'm planning to run with more
than 1 partition, but for the time being, only one partition is used.
* BLK_KSP="-sub_ksp_type preonly": Because of GMRES+ILU

For ILU(0), I'm using:

* BLK_PC="-sub_pc_type ilu -sub_pc_factor_levels 0
-sub_pc_factor_fill 1.00 -sub_pc_factor_shift_nonzero
-sub_pc_factor_mat_ordering_type rcm - sub_pc_factor_pivot_in_blocks"

and I got for convergence:
0 KSP Residual norm 1.687258996558e+00 % max 1 min 1 max/min 1
1 KSP Residual norm 1.687132576728e+00 % max 67.8829 min 67.8829 max/ min 1
2 KSP Residual norm 1.685760733293e+00 % max 3496.78 min 19.5582 max/ min 178.789
3 KSP Residual norm 1.668552043995e+00 % max 3604.26 min 12.0073 max/ min 300.174
4 KSP Residual norm 1.578511835381e+00 % max 3639.92 min 7.35118 max/ min 495.148
....
795 KSP Residual norm 1.465607932165e-09 % max 18209.9 min 0.00612973 max/min 2.97075e+06
796 KSP Residual norm 1.390602265424e-09 % max 18227.3 min 0.00612913 max/min 2.97388e+06
797 KSP Residual norm 1.320529491862e-09 % max 18231.9 min 0.0061286 max/min 2.97489e+06
798 KSP Residual norm 1.253371917713e-09 % max 18234.8 min 0.00612856 max/min 2.97538e+06
799 KSP Residual norm 1.188955299647e-09 % max 18278.5 min 0.00612594 max/min 2.98378e+06
800 KSP Residual norm 1.118294486519e-09 % max 18278.5 min 0.00612475 max/min 2.98437e+06

and the iterative solution compares very well with the one computed using complete LU factorization.

For ILU(1), I'm using:

* BLK_PC="-sub_pc_type ilu -sub_pc_factor_levels 1
-sub_pc_factor_fill 3.81 -sub_pc_factor_shift_nonzero
-sub_pc_factor_mat_ordering_type rcm
-sub_pc_factor_pivot_in_blocks": RCM gives the smallest fill value.

and I got for "convergence":
0 KSP Residual norm 7.095990612421e+126 % max 1 min 1 max/min 1
1 KSP Residual norm 3.313547979190e+123 % max 1.68012e+135 min 1.68012e+135 max/min 1
2 KSP Residual norm 1.257750994639e+119 % max 6.34518e+135 min 3.55953e+131 max/min 17825.9
3 KSP Residual norm 5.233083258710e+118 % max 1.25538e+136 min 1.42732e+127 max/min 8.79538e+08
4 KSP Residual norm 1.938981257595e+118 % max 1.44472e+136 min 4.82369e+125 max/min 2.99506e+10
5 KSP Residual norm 3.371270926617e+116 % max 1.45841e+136 min 1.79839e+125 max/min 8.1095e+10
6 KSP Residual norm 2.179293254483e+115 % max 1.45842e+136 min 5.16422e+122 max/min 2.82408e+13
7 KSP Residual norm 2.120598006522e+115 % max 1.46024e+136 min 3.67337e+122 max/min 3.97521e+13
8 KSP Residual norm 1.486820601733e+115 % max 1.461e+136 min 1.02249e +122 max/min 1.42886e+14
9 KSP Residual norm 7.653834441859e+114 % max 1.46138e+136 min 4.93314e+121 max/min 2.96237e+14
10 KSP Residual norm 5.001204920218e+114 % max 1.47243e+136 min 4.89675e+121 max/min 3.00694e+14

My guess is that the ILU(1) is singular (zero as diagonal elements?), but I thought that the options "- sub_pc_factor_shift_nonzero -sub_pc_factor_pivot_in_blocks" were taking care of that... I got lost in the source files trying to find out who at the end is computing and applying the ILU for MPIBAIJ format (replaced by SEQBAIJ with only one partition, I'm guessing).

The question is: What am I doing wrong? I never heard that ILU(1) was worst than ILU(0)!
Stef.

--
___________________________________________________________
Dr. Stephane AUBERT, CEO & CTO
FLUOREM s.a.s
Centre Scientifique Auguste MOIROUX
64 chemin des MOUILLES
F-69130 ECULLY, FRANCE
International:      fax: +33 4.78.33.99.39      tel: +33 4.78.33.99.35
France:             fax: 04.78.33.99.39         tel: 04.78.33.99.35
email: stephane.aubert@xxxxxxxxxxx
web: www.fluorem.com