450 likes | 649 Views
weilib: wave-equation imaging. Paul Sava. My goal. Quick introduction to the library Not much about the imaging science Ask the right questions. Library goals. Flexible research Code recycling Efficiency vs. flexibility Cluster access Parallelization with MPI, OMP Large problems
E N D
weilib: wave-equation imaging Paul Sava paul@sep.stanford.edu
My goal • Quick introduction to the library • Not much about the imaging science • Ask the right questions paul@sep.stanford.edu
Library goals • Flexible research • Code recycling • Efficiency vs. flexibility • Cluster access • Parallelization with MPI, OMP • Large problems • 3-D data paul@sep.stanford.edu
Characteristics • Programming • Fortran 90 • OO • Problem types • Shot-receiver migration • Shot-profile migration • Versions paul@sep.stanford.edu
Linear operators m d L call operator_init(…) stat=operator(adj,add,m,d) paul@sep.stanford.edu
WE imaging Large datasets (m,d) Data partition Complicated operators (L) Operator partition paul@sep.stanford.edu
The problem DATA (d) w z IMAGE (m) paul@sep.stanford.edu
Cluster distribution MPI w z paul@sep.stanford.edu
Outline • Overview • Objects • Partitioning • I/O • Operators • WE migration • WE MVA • Examples paul@sep.stanford.edu
WEI initialization … call sep_init() call weipar_init() call weipar_report() call weitag_init() call weipro_init() call weipro_bcast() … paul@sep.stanford.edu
WEI objects & procedures Objects Data Image Slowness Wavefield Procedures file tag dimensions I/O MPI communication paul@sep.stanford.edu
WEI data object type wedata character(len=128) :: tag character(len=128) :: from integer :: bites,esize type(axis) :: amx,amy,ahx,ahy,aw complex, pointer :: bin(:,:,:,:,:) end type wedata paul@sep.stanford.edu
WEI data object DD%tag=wtag%D call weiD_init(DD) call weiD_report(DD) call weiD_allocate(DD) call weiD_read(DD) call weiD_write(DD) call weiD_mpisprd(DD) call weiD_mpigthr(DD) … paul@sep.stanford.edu
WEI migration workflow MPI MPI data image slowness Node 1 data data image slowness Node 2 image data image slowness slowness Node 3 data image slowness Node 4 paul@sep.stanford.edu
WEI workflow example … call weiS_mpicopy(SS) call weiR_mpizero(RR) call weiD_mpisprd(DD) … call weimigof(…) … call weiR_mpisumm(RR) … paul@sep.stanford.edu
I/O blocking w z paul@sep.stanford.edu
I/O blocking example … do iwb=1,nwb call weiD_read(bW,iwb) do izb=1,nzb call weiR_read(RR,izb) call weiS_read(SS,izb) … call weiR_write(RR,izb) end do end do … paul@sep.stanford.edu
Shared memory parallelization w z OMP paul@sep.stanford.edu
Shared memory parallelization !$OMP PARALLEL !$OMP DO & !$OMP$ PRIVATE(iws,izs,ith) do iws=1,nws ith=omp_get_thread_num()+1 do izs=1,nzs … end do end do !$OMP END DO !$OMP END PARALLEL paul@sep.stanford.edu
Outline • Overview • Objects • Partitioning • I/O • Operators • WE migration • WE MVA • Examples paul@sep.stanford.edu
WE migration DATA (d) frequency extrapolation imaging IMAGE (m) depth paul@sep.stanford.edu
Extrapolation frequency Wz Wz+Dz depth paul@sep.stanford.edu
Imaging frequency Rz+Dz Wz+Dz depth paul@sep.stanford.edu
WE migration paul@sep.stanford.edu
WCop: wavefield continuation operator • Single reference slowness • Multi reference slowness • Wavefield interpolation paul@sep.stanford.edu
SLop: slowness selection operator • Min/Mean/Median • Lloyd paul@sep.stanford.edu
FKop: phase-shift operator • Common-azimuth • Narrow-azimuth • Full prestack paul@sep.stanford.edu
FXop: finite-difference operator • Phase-shift • Split-step Fourier • Generalized Screen Propagator paul@sep.stanford.edu
IGop: imaging operator • Offset • Ray parameter paul@sep.stanford.edu
Outline • Overview • Objects • Partitioning • I/O • Operators • WE migration • WE MVA • Examples paul@sep.stanford.edu
WE MVA paul@sep.stanford.edu
SCop: scattering operator paul@sep.stanford.edu
Outline • Overview • Objects • Partitioning • I/O • Operators • WE migration • WE MVA • Examples paul@sep.stanford.edu
WEI datuming (survey sinking) Slowness operator Continuation operator f-k operator f-x operator call weidtm_init( SLin=weiop_slo1_init, WCin=weiop_mwc1_init, FKin=weiop_wem3_init, FXin=weiop_ssf3_init) stat = weidtm(adj,add, D, U, SLop=weiop_slo1, WCop=weiop_mwc1, FKop=weiop_wem3, FXop=weiop_ssf3) paul@sep.stanford.edu
WEI migration: example 1 call weimig_init( SLin=weiop_slo1_init, WCin=weiop_mwc1_init, FKin=weiop_wem3_init, FXin=weiop_ssf3_init, IGin=weiop_pcig_init) stat = weimig(adj,add, R, D, SLop=weiop_slo1, WCop=weiop_mwc1, FKop=weiop_wem3, FXop=weiop_ssf3, IGop=weiop_pcig) Imaging operator paul@sep.stanford.edu
WEI migration: example 2 call weimig_init( SLin=weiop_sllN_init, WCin=weiop_mwcN_init, FKin=weiop_cam3_init, FXin=weiop_ssf3_init, IGin=weiop_hcig_init) stat = weimig(adj,add, R, D, SLop=weiop_sllN, WCop=weiop_mwcN, FKop=weiop_cam3, FXop=weiop_ssf3, IGop=weiop_hcig) paul@sep.stanford.edu
WEI MVA call weimva_init( SLin=weiop_sllN_init, SCin=weiop_bor1_init, WCin=weiop_mwcN_init, FKin=weiop_wem3_init, FXin=weiop_ssf3_init, IGin=weiop_hcig_init) stat = weimva(adj,add, dS,dR, SLop=weiop_sllN, SCop=weiop_bor1, WCop=weiop_mwcN, FKop=weiop_wem3, FXop=weiop_ssf3, IGop=weiop_hcig) Scattering operator paul@sep.stanford.edu
Summary • Flexible, reusable f90 code • Cluster ready (MPI,OMP) • Standard operator interface • Applications • WE datuming • WE migration • WE MVA paul@sep.stanford.edu
Resources (name@sep.stanford.edu) • Developers • Paul, Bob • Manual • Marie • Users • Biondo, Antoine, Morgan, Brad, Nick, Alejandro paul@sep.stanford.edu