Hybrid Message-Passing Interface-Open Multiprocessing Accelerated Euler-Lagrange Simulations of Microbubble Enhanced HIFU for Tumor Ablation.

Microbubble enhanced high intensity focused ultrasound (HIFU) is of great interest to tissue ablation for solid tumor treatments such as in liver and brain cancers, in which contrast agents/microbubbles are injected into the targeted region to promote heating and reduce prefocal tissue damage. A compressible Euler-Lagrange coupled model has been developed to accurately characterize the acoustic and thermal fields during this process. This employs a compressible Navier-Stokes solver for the ultrasound acoustic field and a discrete singularities model for bubble dynamics. To address the demanding computational cost relevant to practical medical applications, a multilevel hybrid message-passing interface (MPI)-open multiprocessing (OpenMP) parallelization scheme is developed to take advantage of both scalability of MPI and load balancing of OpenMP. At the first level, the Eulerian computational domain is divided into multiple subdomains and the bubbles are subdivided into groups based on which subdomain they fall into. At the next level, in each subdomain containing bubbles, multiple OpenMP threads are activated to speed up the computations of the bubble dynamics. For improved throughput, the OpenMP threads are more heavily distributed to subdomains where the bubbles are clustered. By doing this, MPI load imbalance issue due to uneven bubble distribution is mitigated by OpenMP speedup locally for those subdomains hosting more bubbles than others. The hybrid MPI-OpenMP Euler-Lagrange solver is used to conduct simulations and physical studies of bubble-enhanced HIFU problems containing a large number of microbubbles. The phenomenon of acoustic shadowing caused by the bubble cloud is then analyzed and discussed. Efficiency tests on two different machines with 48 processors are conducted and indicate 2-3 times speedup with the same hardware by introducing an OpenMP parallelization in combination with the MPI parallelization.

Message Passing Interface Parallelization for Two-Way Coupled Euler-Lagrange Simulation of Microbubble Enhanced HIFU.

Microbubble enhanced high intensity focused ultrasound (HIFU) is of great interest to tissue ablation for tumor treatment such as in liver and brain cancers. To accurately characterize the acoustic and thermal fields during this process, a coupled Euler-Lagrange model is used. The ultrasound field is modeled using compressible Navier-Stokes equations on an Eulerian grid, while the microbubbles are tracked in a Lagrangian fashion. The coupling is realized through the void fraction computed from the instantaneous bubble volumes. To speed up the computations, an message passing interface parallelization scheme based on domain decomposition is herein proposed. During each time-step, message passing interface processors, each handling one subdomain, are first used to execute the fluid computation, and then the bubble computations. This is followed by the coupling procedure. The coupling is challenging as the effect of the bubbles through the void fraction at an Eulerian point near a subdomain border will require information from bubbles located in different subdomains, and vice versa. This is addressed by a special utilization of ghost cells surrounding each fluid subdomain, which allows bubbles to spread their void fraction effects across subdomain edges without the need of exchanging directly bubble information between subdomains and significantly increasing overhead. After a careful verification of gas effects conservation, this parallelization scheme is validated and illustrated on a typical microbubble enhanced HIFU problem, followed by parallelization scaling tests and efficiency analysis.

Numerical study of acoustically driven bubble cloud dynamics near a rigid wall.

The dynamics of a bubble cloud excited by a sinusoidal pressure field near a rigid wall is studied using a novel Eulerian/Lagrangian two-phase flow model. The effects of key parameters such as the amplitude and frequency of the excitation pressure, the cloud and bubble sizes, the void fraction, and the initial standoff distance on the bubbles' collective behavior and the resulting pressure loads on the nearby wall are investigated. The study shows that nonlinear bubble cloud dynamics becomes more pronounced and results in higher pressure loading at the wall as the excitation pressure amplitude increases. The strongest collective bubble behavior occurs at a preferred resonance frequency. At this resonance frequency, pressure peaks orders of magnitudes higher than the excitation pressure result from the bubble interaction when the amplitude of the pressure excitation is high. The numerically obtained resonance frequency is significantly different from the reported natural frequency of a spherical cloud derived from linear theory, which assumes small amplitude oscillations in an unbounded medium. At high amplitudes of the excitation, the resonance frequency decreases almost linearly with the ratio of excitation pressure amplitude to ambient pressure until the ratio is larger than one.

A physics based multiscale modeling of cavitating flows.

Numerical modeling of cavitating bubbly flows is challenging due to the wide range of characteristic lengths of the physics at play: from micrometers (e.g., bubble nuclei radius) to meters (e.g., propeller diameter or sheet cavity length). To address this, we present here a multiscale approach which integrates a Discrete Singularities Model (DSM) for dispersed microbubbles and a two-phase Navier Stokes solver for the bubbly medium, which includes a level set approach to describe large cavities or gaseous pockets. Inter-scale schemes are used to smoothly bridge the two transitioning subgrid DSM bubbles into larger discretized cavities. This approach is demonstrated on several problems including cavitation inception and vapor core formation in a vortex flow, sheet-to-cloud cavitation over a hydrofoil, cavitation behind a blunt body, and cavitation on a propeller. These examples highlight the capabilities of the developed multiscale model in simulating various form of cavitation.

Numerical study of gravity effects on phase separation in a swirl chamber.

The effects of gravity on a phase separator are studied numerically using an Eulerian/Lagrangian two-phase flow approach. The separator utilizes high intensity swirl to separate bubbles from the liquid. The two-phase flow enters tangentially a cylindrical swirl chamber and rotate around the cylinder axis. On earth, as the bubbles are captured by the vortex formed inside the swirl chamber due to the centripetal force, they also experience the buoyancy force due to gravity. In a reduced or zero gravity environment buoyancy is reduced or inexistent and capture of the bubbles by the vortex is modified. The present numerical simulations enable study of the relative importance of the acceleration of gravity on the bubble capture by the swirl flow in the separator. In absence of gravity, the bubbles get stratified depending on their sizes, with the larger bubbles entering the core region earlier than the smaller ones. However, in presence of gravity, stratification is more complex as the two acceleration fields - due to gravity and to rotation - compete or combine during the bubble capture.