Re: not enough memory?!

[Barry Smith <bsmith@xxxxxxxxxxx>]

In the output:

ICC: factor fill ratio allocated 1

but

ICC: 2 levels of fill

You should use the option -pc_factor_fill <a> (or -sub_pc_factor_fill  
if using block Jacobi or ASM in parallel)
and set <a> to be the expected ratio of nonzeros in the factored  
matrix and the unfactored matrix;
I suggest starting with at least 3 since you are using 2 levels of  
fill. See the manual page for PCFactorSetFill()

   Barry


On Nov 2, 2007, at 7:54 AM, Zhifeng Sheng wrote:

> Dear all
>
> I have a problem for memory management.
>
> I implemented 3d FEM code and at this moment SeqSBAIJ is used to  
> store the system matrix, then I used CG method and ICC(k)  
> preconditioner.
>
> The test configuration is not very big, I succeeded in constructing  
> the system matrix and construct the preconditioner. But when I need  
> to setup the KSP solver, I got the error message below ( I also  
> dumped some information about solver and system matrix):
>
> KSP Object:
> type: cg
> maximum iterations=10000, initial guess is zero
> tolerances:  relative=1e-09, absolute=1e-50, divergence=10000
> left preconditioning
> PC Object:
> type: icc
>   ICC: 2 levels of fill
>   ICC: factor fill ratio allocated 1
> linear system matrix = precond matrix:
> Matrix Object:
>   type=seqsbaij, rows=435450, cols=435450
>   total: nonzeros=9470450, allocated nonzeros=21772580
>       block size is 1
> [0]PETSC ERROR: --------------------- Error Message  
> ------------------------------------
> [0]PETSC ERROR: Out of memory. This could be due to allocating
> [0]PETSC ERROR: too large an object or bleeding by not properly
> [0]PETSC ERROR: destroying unneeded objects.
> [0]PETSC ERROR: Memory allocated 375435120 Memory used by process  
> 428597248
> [0]PETSC ERROR: Try running with -malloc_dump or -malloc_log for info.
> [0]PETSC ERROR: Memory requested 3108507040!
> [0]PETSC ERROR:  
> ------------------------------------------------------------------------
> [0]PETSC ERROR: Petsc Release Version 2.3.2, Patch 10, Wed Mar 28  
> 19:13:22 CDT 2007 HG revision:  
> d7298c71db7f5e767f359ae35d33cab3bed44428
> [0]PETSC ERROR: See docs/changes/index.html for recent updates.
> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [0]PETSC ERROR: See docs/index.html for manual pages.
> [0]PETSC ERROR:  
> ------------------------------------------------------------------------
> [0]PETSC ERROR: bin/main on a linux-gnu named callisto by zhifeng  
> Fri Nov  2 13:26:39 2007
> [0]PETSC ERROR: Libraries linked from /u/01/01/zhifeng/install/lib/ 
> linux-gnu-c-debug
> [0]PETSC ERROR: Configure run at Wed Aug  8 13:46:26 2007
> [0]PETSC ERROR: Configure options --with-cc=gcc --with-fc=g77 -- 
> download-f-blas-lapack=1 --download-mpich=1 --prefix=/u/01/01/ 
> zhifeng/install --with-shared=0
> [0]PETSC ERROR:  
> ------------------------------------------------------------------------
> [0]PETSC ERROR: PetscMallocAlign() line 61 in src/sys/memory/mal.c
> [0]PETSC ERROR: PetscTrMallocDefault() line 194 in src/sys/memory/ 
> mtr.c
> [0]PETSC ERROR: PetscFreeSpaceGet() line 14 in src/mat/utils/ 
> freespace.c
> [0]PETSC ERROR: MatICCFactorSymbolic_SeqSBAIJ() line 1648 in src/mat/ 
> impls/sbaij/seq/sbaijfact2.c
> [0]PETSC ERROR: MatICCFactorSymbolic() line 4611 in src/mat/ 
> interface/matrix.c
> [0]PETSC ERROR: PCSetup_ICC() line 117 in src/ksp/pc/impls/factor/ 
> icc/icc.c
> [0]PETSC ERROR: PCSetUp() line 801 in src/ksp/pc/interface/precon.c
> [0]PETSC ERROR: KSPSetUp() line 234 in src/ksp/ksp/interface/itfunc.c
> [0]PETSC ERROR: KSPSolve() line 338 in src/ksp/ksp/interface/itfunc.c
> =====================the solver convergence info======================
> Convergence in 0 iterations.
> =====================================================================
> writing solution to file temp_H.vtk ...
> number of unknowns >>435450
> finishing the numeric solver ...
> LSFIM hybrid method for four domain problem
> deallocating memory of domain class ...
>
>
> I debug the code, and the error is dumped by the funciton KSPSetUp()  
> which should not take so much memory, but still 3108507040 memory  
> was requested .... I really could not figure out where I need them...
>
> Any help will be appreciated.
>
> Thank you all
> Best regards
> Zhifeng