Re: not enough memory?!

[Zhifeng Sheng <z.sheng@xxxxxxxxxxxxxx>]

Matthew Knepley wrote:

> The routine which calculates ICC requested 3G, so it seems that the matrix
> is too dense or large to be factored on this machine.
>
>  Matt
>
> On Nov 2, 2007 7:54 AM, Zhifeng Sheng <z.sheng@xxxxxxxxxxxxxx> wrote:
>  
>
> > Dear all
> >
> > I have a problem for memory management.
> >
> > I implemented 3d FEM code and at this moment SeqSBAIJ is used to store
> > the system matrix, then I used CG method and ICC(k) preconditioner.
> >
> > The test configuration is not very big, I succeeded in constructing the
> > system matrix and construct the preconditioner. But when I need to setup
> > the KSP solver, I got the error message below ( I also dumped some
> > information about solver and system matrix):
> >
> > KSP Object:
> >  type: cg
> >  maximum iterations=10000, initial guess is zero
> >  tolerances:  relative=1e-09, absolute=1e-50, divergence=10000
> >  left preconditioning
> > PC Object:
> >  type: icc
> >    ICC: 2 levels of fill
> >    ICC: factor fill ratio allocated 1
> >  linear system matrix = precond matrix:
> >  Matrix Object:
> >    type=seqsbaij, rows=435450, cols=435450
> >    total: nonzeros=9470450, allocated nonzeros=21772580
> >        block size is 1
> > [0]PETSC ERROR: --------------------- Error Message
> > ------------------------------------
> > [0]PETSC ERROR: Out of memory. This could be due to allocating
> > [0]PETSC ERROR: too large an object or bleeding by not properly
> > [0]PETSC ERROR: destroying unneeded objects.
> > [0]PETSC ERROR: Memory allocated 375435120 Memory used by process 428597248
> > [0]PETSC ERROR: Try running with -malloc_dump or -malloc_log for info.
> > [0]PETSC ERROR: Memory requested 3108507040!
> > [0]PETSC ERROR:
> > ------------------------------------------------------------------------
> > [0]PETSC ERROR: Petsc Release Version 2.3.2, Patch 10, Wed Mar 28
> > 19:13:22 CDT 2007 HG revision: d7298c71db7f5e767f359ae35d33cab3bed44428
> > [0]PETSC ERROR: See docs/changes/index.html for recent updates.
> > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> > [0]PETSC ERROR: See docs/index.html for manual pages.
> > [0]PETSC ERROR:
> > ------------------------------------------------------------------------
> > [0]PETSC ERROR: bin/main on a linux-gnu named callisto by zhifeng Fri
> > Nov  2 13:26:39 2007
> > [0]PETSC ERROR: Libraries linked from
> > /u/01/01/zhifeng/install/lib/linux-gnu-c-debug
> > [0]PETSC ERROR: Configure run at Wed Aug  8 13:46:26 2007
> > [0]PETSC ERROR: Configure options --with-cc=gcc --with-fc=g77
> > --download-f-blas-lapack=1 --download-mpich=1
> > --prefix=/u/01/01/zhifeng/install --with-shared=0
> > [0]PETSC ERROR:
> > ------------------------------------------------------------------------
> > [0]PETSC ERROR: PetscMallocAlign() line 61 in src/sys/memory/mal.c
> > [0]PETSC ERROR: PetscTrMallocDefault() line 194 in src/sys/memory/mtr.c
> > [0]PETSC ERROR: PetscFreeSpaceGet() line 14 in src/mat/utils/freespace.c
> > [0]PETSC ERROR: MatICCFactorSymbolic_SeqSBAIJ() line 1648 in
> > src/mat/impls/sbaij/seq/sbaijfact2.c
> > [0]PETSC ERROR: MatICCFactorSymbolic() line 4611 in
> > src/mat/interface/matrix.c
> > [0]PETSC ERROR: PCSetup_ICC() line 117 in src/ksp/pc/impls/factor/icc/icc.c
> > [0]PETSC ERROR: PCSetUp() line 801 in src/ksp/pc/interface/precon.c
> > [0]PETSC ERROR: KSPSetUp() line 234 in src/ksp/ksp/interface/itfunc.c
> > [0]PETSC ERROR: KSPSolve() line 338 in src/ksp/ksp/interface/itfunc.c
> > =====================the solver convergence info======================
> > Convergence in 0 iterations.
> > =====================================================================
> > writing solution to file temp_H.vtk ...
> > number of unknowns >>435450
> > finishing the numeric solver ...
> > LSFIM hybrid method for four domain problem
> > deallocating memory of domain class ...
> >
> >
> > I debug the code, and the error is dumped by the funciton KSPSetUp()
> > which should not take so much memory, but still 3108507040 memory was
> > requested .... I really could not figure out where I need them...
> >
> > Any help will be appreciated.
> >
> > Thank you all
> > Best regards
> > Zhifeng
> >
> >
> >    
>
>
>
>  
So ... computation of the preconditioner is actually called by 
KSPSetUp() instead of PCGetFactoredMatrix(prec,&M)?

Could you please tell me when the system matrix be factorized 
(incompletely)? thank you.

Zhifeng