Enhancing predictive capabilities in data-driven dynamical modeling with automatic differentiation: Koopman and neural ODE approaches.

Data-driven approximations of the Koopman operator are promising for predicting the time evolution of systems characterized by complex dynamics. Among these methods, the approach known as extended dynamic mode decomposition with dictionary learning (EDMD-DL) has garnered significant attention. Here, we present a modification of EDMD-DL that concurrently determines both the dictionary of observables and the corresponding approximation of the Koopman operator. This innovation leverages automatic differentiation to facilitate gradient descent computations through the pseudoinverse. We also address the performance of several alternative methodologies. We assess a "pure" Koopman approach, which involves the direct time-integration of a linear, high-dimensional system governing the dynamics within the space of observables. Additionally, we explore a modified approach where the system alternates between spaces of states and observables at each time step-this approach no longer satisfies the linearity of the true Koopman operator representation. For further comparisons, we also apply a state-space approach (neural ordinary differential equations). We consider systems encompassing two- and three-dimensional ordinary differential equation systems featuring steady, oscillatory, and chaotic attractors, as well as partial differential equations exhibiting increasingly complex and intricate behaviors. Our framework significantly outperforms EDMD-DL. Furthermore, the state-space approach offers superior performance compared to the "pure" Koopman approach where the entire time evolution occurs in the space of observables. When the temporal evolution of the Koopman approach alternates between states and observables at each time step, however, its predictions become comparable to those of the state-space approach.

Marginated aberrant red blood cells induce pathologic vascular stress fluctuations in a computational model of hematologic disorders.

Red blood cell (RBC) disorders such as sickle cell disease affect billions worldwide. While much attention focuses on altered properties of aberrant RBCs and corresponding hemodynamic changes, RBC disorders are also associated with vascular dysfunction, whose origin remains unclear and which provoke severe consequences including stroke. Little research has explored whether biophysical alterations of RBCs affect vascular function. We use a detailed computational model of blood that enables characterization of cell distributions and vascular stresses in blood disorders and compare simulation results with experimental observations. Aberrant RBCs, with their smaller size and higher stiffness, concentrate near vessel walls (marginate) because of contrasts in physical properties relative to normal cells. In a curved channel exemplifying the geometric complexity of the microcirculation, these cells distribute heterogeneously, indicating the importance of geometry. Marginated cells generate large transient stress fluctuations on vessel walls, indicating a mechanism for the observed vascular inflammation.

Marginated aberrant red blood cells induce pathologic vascular stress fluctuations in a computational model of hematologic disorders.

Red blood cell (RBC) disorders affect billions worldwide. While alterations in the physical properties of aberrant RBCs and associated hemodynamic changes are readily observed, in conditions such as sickle cell disease and iron deficiency, RBC disorders can also be associated with vascular dysfunction. The mechanisms of vasculopathy in those diseases remain unclear and scant research has explored whether biophysical alterations of RBCs can directly affect vascular function. Here we hypothesize that the purely physical interactions between aberrant RBCs and endothelial cells, due to the margination of stiff aberrant RBCs, play a key role in this phenomenon for a range of disorders. This hypothesis is tested by direct simulations of a cellular scale computational model of blood flow in sickle cell disease, iron deficiency anemia, COVID-19, and spherocytosis. We characterize cell distributions for normal and aberrant RBC mixtures in straight and curved tubes, the latter to address issues of geometric complexity that arise in the microcirculation. In all cases aberrant RBCs strongly localize near the vessel walls (margination) due to contrasts in cell size, shape, and deformability from the normal cells. In the curved channel, the distribution of marginated cells is very heterogeneous, indicating a key role for vascular geometry. Finally, we characterize the shear stresses on the vessel walls; consistent with our hypothesis, the marginated aberrant cells generate large transient stress fluctuations due to the high velocity gradients induced by their near-wall motions. The anomalous stress fluctuations experienced by endothelial cells may be responsible for the observed vascular inflammation.

Deep learning delay coordinate dynamics for chaotic attractors from partial observable data.

A common problem in time-series analysis is to predict dynamics with only scalar or partial observations of the underlying dynamical system. For data on a smooth compact manifold, Takens' theorem proves a time-delayed embedding of the partial state is diffeomorphic to the attractor, although for chaotic and highly nonlinear systems, learning these delay coordinate mappings is challenging. We utilize deep artificial neural networks (ANNs) to learn discrete time maps and continuous time flows of the partial state. Given training data for the full state, we also learn a reconstruction map. Thus, predictions of a time series can be made from the current state and several previous observations with embedding parameters determined from time-series analysis. The state space for time evolution is of comparable dimension to reduced order manifold models. These are advantages over recurrent neural network models, which require a high-dimensional internal state or additional memory terms and hyperparameters. We demonstrate the capacity of deep ANNs to predict chaotic behavior from a scalar observation on a manifold of dimension three via the Lorenz system. We also consider multivariate observations on the Kuramoto-Sivashinsky equation, where the observation dimension required for accurately reproducing dynamics increases with the manifold dimension via the spatial extent of the system.

Pathologic mechanobiological interactions between red blood cells and endothelial cells directly induce vasculopathy in iron deficiency anemia.

The correlation between cardiovascular disease and iron deficiency anemia (IDA) is well documented but poorly understood. Using a multi-disciplinary approach, we explore the hypothesis that the biophysical alterations of red blood cells (RBCs) in IDA, such as variable degrees of microcytosis and decreased deformability may directly induce endothelial dysfunction via mechanobiological mechanisms. Using a combination of atomic force microscopy and microfluidics, we observed that subpopulations of IDA RBCs (idRBCs) are significantly stiffer and smaller than both healthy RBCs and the remaining idRBC population. Furthermore, computational simulations demonstrated that the smaller and stiffer idRBC subpopulations marginate toward the vessel wall causing aberrant shear stresses. This leads to increased vascular inflammation as confirmed with perfusion of idRBCs into our "endothelialized" microfluidic systems. Overall, our multifaceted approach demonstrates that the altered biophysical properties of idRBCs directly lead to vasculopathy, suggesting that the IDA and cardiovascular disease association extends beyond correlation and into causation.

Data-driven reduced-order modeling of spatiotemporal chaos with neural ordinary differential equations.

Dissipative partial differential equations that exhibit chaotic dynamics tend to evolve to attractors that exist on finite-dimensional manifolds. We present a data-driven reduced-order modeling method that capitalizes on this fact by finding a coordinate representation for this manifold and then a system of ordinary differential equations (ODEs) describing the dynamics in this coordinate system. The manifold coordinates are discovered using an undercomplete autoencoder-a neural network (NN) that reduces and then expands dimension. Then, the ODE, in these coordinates, is determined by a NN using the neural ODE framework. Both of these steps only require snapshots of data to learn a model, and the data can be widely and/or unevenly spaced. Time-derivative information is not needed. We apply this framework to the Kuramoto-Sivashinsky equation for domain sizes that exhibit chaotic dynamics with again estimated manifold dimensions ranging from 8 to 28. With this system, we find that dimension reduction improves performance relative to predictions in the ambient space, where artifacts arise. Then, with the low-dimensional model, we vary the training data spacing and find excellent short- and long-time statistical recreation of the true dynamics for widely spaced data (spacing of ∼ 0.7 Lyapunov times). We end by comparing performance with various degrees of dimension reduction and find a "sweet spot" in terms of performance vs dimension.

Symmetry reduction for deep reinforcement learning active control of chaotic spatiotemporal dynamics.

Deep reinforcement learning (RL) is a data-driven, model-free method capable of discovering complex control strategies for macroscopic objectives in high-dimensional systems, making its application toward flow control promising. Many systems of flow control interest possess symmetries that, when neglected, can significantly inhibit the learning and performance of a naive deep RL approach. Using a test-bed consisting of the Kuramoto-Sivashinsky equation (KSE), equally spaced actuators, and a goal of minimizing dissipation and power cost, we demonstrate that by moving the deep RL problem to a symmetry-reduced space, we can alleviate limitations inherent in the naive application of deep RL. We demonstrate that symmetry-reduced deep RL yields improved data efficiency as well as improved control policy efficacy compared to policies found by naive deep RL. Interestingly, the policy learned by the symmetry aware control agent drives the system toward an equilibrium state of the forced KSE that is connected by continuation to an equilibrium of the unforced KSE, despite having been given no explicit information regarding its existence. That is, to achieve its goal, the RL algorithm discovers and stabilizes an equilibrium state of the system. Finally, we demonstrate that the symmetry-reduced control policy is robust to observation and actuation signal noise, as well as to system parameters it has not observed before.

Discovering multiscale and self-similar structure with data-driven wavelets.

Many materials, processes, and structures in science and engineering have important features at multiple scales of time and/or space; examples include biological tissues, active matter, oceans, networks, and images. Explicitly extracting, describing, and defining such features are difficult tasks, at least in part because each system has a unique set of features. Here, we introduce an analysis method that, given a set of observations, discovers an energetic hierarchy of structures localized in scale and space. We call the resulting basis vectors a "data-driven wavelet decomposition." We show that this decomposition reflects the inherent structure of the dataset it acts on, whether it has no structure, structure dominated by a single scale, or structure on a hierarchy of scales. In particular, when applied to turbulence-a high-dimensional, nonlinear, multiscale process-the method reveals self-similar structure over a wide range of spatial scales, providing direct, model-free evidence for a century-old phenomenological picture of turbulence. This approach is a starting point for the characterization of localized hierarchical structures in multiscale systems, which we may think of as the building blocks of these systems.

Coil-stretch-like transition of elastic sheets in extensional flows.

The conformation of a long linear polymer dissolved in fluid and exposed to an extensional flow is well-known to exhibit a "coil-stretch" transition, which for sufficiently long chains can lead to bistability. The present work reports computations indicating that an analogous "compact-stretched" transition arises in the dynamics of a thin elastic sheet. Sheets of nominally circular, square or rectangular shape are simulated in planar and biaxial flows using a finite element method for the sheet conformations and a regularized Stokeslet method for the fluid flow. If a neo-Hookean constitutive model is used for the sheet elasticity, the sheets will stretch without bound once a critical extension rate, as characterized nondimensionally by a capillary number, is exceeded. Nonlinear elasticity, represented with the Yeoh model, arrests the stretching, leading to a highly-stretched steady state once the critical capillary number is exceeded. For all shapes and in both planar and biaxial extension, a parameter regime exists in which both weakly stretched (compact) and strongly stretched states can be found, depending on initial conditions. I.e. this parameter regime displays bistability. As in the long-chain polymer case, the bistable behavior arises from the hydrodynamic interaction between distant elements of the sheet, and vanishes if these interactions are artificially screened by use of a Brinkman model for the fluid motion. While the sheets can transiently display wrinkled shapes, all final shapes in planar and biaxial extension are planar.

Low- and High-Drag Intermittencies in Turbulent Channel Flows.

Recent direct numerical simulations (DNS) and experiments in turbulent channel flow have found intermittent low- and high-drag events in Newtonian fluid flows, at Reτ=uτh/ν between 70 and 100, where uτ, h and ν are the friction velocity, channel half-height and kinematic viscosity, respectively. These intervals of low-drag and high-drag have been termed "hibernating" and "hyperactive", respectively, and in this paper, a further investigation of these intermittent events is conducted using experimental and numerical techniques. For experiments, simultaneous measurements of wall shear stress and velocity are carried out in a channel flow facility using hot-film anemometry (HFA) and laser Doppler velocimetry (LDV), respectively, for Reτ between 70 and 250. For numerical simulations, DNS of a channel flow is performed in an extended domain at Reτ = 70 and 85. These intermittent events are selected by carrying out conditional sampling of the wall shear stress data based on a combined threshold magnitude and time-duration criteria. The use of three different scalings (so-called outer, inner and mixed) for the time-duration criterion for the conditional events is explored. It is found that if the time-duration criterion is kept constant in inner units, the frequency of occurrence of these conditional events remain insensitive to Reynolds number. There exists an exponential distribution of frequency of occurrence of the conditional events with respect to their duration, implying a potentially memoryless process. An explanation for the presence of a spike (or dip) in the ensemble-averaged wall shear stress data before and after the low-drag (or high-drag) events is investigated. During the low-drag events, the conditionally-averaged streamwise velocities get closer to Virk's maximum drag reduction (MDR) asymptote, near the wall, for all Reynolds numbers studied. Reynolds shear stress (RSS) characteristics during these conditional events are investigated for Reτ = 70 and 85. Except very close to the wall, the conditionally-averaged RSS is higher than the time-averaged value during the low-drag events.

Deep learning to discover and predict dynamics on an inertial manifold.

A data-driven framework is developed to represent chaotic dynamics on an inertial manifold (IM) and applied to solutions of the Kuramoto-Sivashinsky equation. A hybrid method combining linear and nonlinear (neural-network) dimension reduction transforms between coordinates in the full state space and on the IM. Additional neural networks predict time evolution on the IM. The formalism accounts for translation invariance and energy conservation, and substantially outperforms linear dimension reduction, reproducing very well key dynamic and statistical features of the attractor.

Multiplicity of stable orbits for deformable prolate capsules in shear flow.

This work investigates the orbital dynamics of a fluid-filled deformable prolate capsule in unbounded simple shear flow at zero Reynolds number using direct simulations. The motion of the capsule is simulated using a model that incorporates shear elasticity, area dilatation, and bending resistance. Here the deformability of the capsule is characterized by the nondimensional capillary number Ca, which represents the ratio of viscous stresses to elastic restoring stresses on the capsule. For a capsule with small bending stiffness, at a given Ca, the orientation converges over time towards a unique stable orbit independent of the initial orientation. With increasing Ca, four dynamical modes are found for the stable orbit, namely, rolling, wobbling, oscillating-swinging, and swinging. On the other hand, for a capsule with large bending stiffness, multiplicity in the orbit dynamics is observed. When the viscosity ratio λ ≲ 1, the long-axis of the capsule always tends towards a stable orbit in the flow-gradient plane, either tumbling or swinging, depending on Ca. When λ ≳ 1, the stable orbit of the capsule is a tumbling motion at low Ca, irrespective of the initial orientation. Upon increasing Ca, there is a symmetry-breaking bifurcation away from the tumbling orbit, and the capsule is observed to adopt multiple stable orbital modes including nonsymmetric precessing and rolling, depending on the initial orientation. As Ca further increases, the nonsymmetric stable orbit loses existence at a saddle-node bifurcation, and rolling becomes the only attractor at high Ca, whereas the rolling state coexists with the nonsymmetric state at intermediate values of Ca. A symmetry-breaking bifurcation away from the rolling orbit is also found upon decreasing Ca. The regime with multiple attractors becomes broader as the aspect ratio of the capsule increases, while narrowing as viscosity ratio increases. We also report the particle contribution to the stress, which also displays multiplicity.

Flow-induced segregation and dynamics of red blood cells in sickle cell disease.

Blood flow in sickle cell disease (SCD) can substantially differ from normal blood flow due to significant alterations in the physical properties of the red blood cells (RBCs). Chronic complications, such as inflammation of the endothelial cells lining blood vessel walls, are associated with SCD, for reasons that are unclear. Here, detailed boundary integral simulations are performed to investigate an idealized model flow flow in SCD, a binary suspension of flexible biconcave discoidal fluid-filled capsules and stiff curved prolate capsules that represent healthy and sickle RBCs, respectively, subjected to pressure-driven flow in a planar slit. The stiff component is dilute. The key observation is that, unlike healthy RBCs that concentrate around the center of the channel and form an RBC-depleted layer (i.e. cell-free layer) next to the walls, sickle cells are largely drained from the bulk of the suspension and aggregate inside the cell-free layer, displaying strong margination. These cells are found to undergo a rigid-body-like rolling orbit near the walls. A binary suspension of flexible biconcave discoidal capsules and stiff straight (non-curved) prolate capsules is also considered for comparison, and the curvature of the stiff component is found to play a minor role in the behavior. Additionally, by considering a mixture of flexible and stiff biconcave discoids, we reveal that rigidity difference by itself is sufficient to induce the segregation behavior in a binary suspension. Furthermore, the additional shear stress on the walls induced by the presence of cells is computed for the various cases. Compared to the small fluctuations in wall shear stress for a suspension of healthy RBCs, large local peaks in wall shear stress are observed for the binary suspensions, due to the proximity of the marginated stiff cells to the walls. This effect is most marked for the straight prolate capsules. As endothelial cells are known to mechanotransduce physical forces such as aberrations in shear stress and convert them to physiological processes such as activation of inflammatory signals, these results may aid in understanding mechanisms for endothelial dysfunction associated with SCD.

Dynamics of deformable straight and curved prolate capsules in simple shear flow.

This work investigates the motion of neutrally-buoyant, slightly deformable straight and curved prolate fluid-filled capsules in unbounded simple shear flow at zero Reynolds number using direct simulations. The curved capsules serve as a model for the typical crescent-shaped sickle red blood cells in sickle cell disease (SCD). The effects of deformability and curvature on the dynamics are revealed. We show that with low deformability, straight prolate spheroidal capsules exhibit tumbling in the shear plane as their unique asymptotically stable orbit. This result contrasts with that for rigid spheroids, where infinitely many neutrally stable Jeffery orbits exist. The dynamics of curved prolate capsules are more complicated due to a combined effect of deformability and curvature. At short times, depending on the initial orientation, slightly deformable curved prolate capsules exhibit either a Jeffery-like motion such as tumbling or kayaking, or a non-Jeffery-like behavior in which the director (end-to-end vector) of the capsule crosses the shear-gradient plane back and forth. At long times, however, a Jeffery-like quasiperiodic orbit is taken regardless of the initial orientation. We further show that the average of the long-time trajectory can be well approximated using the analytical solution for Jeffery orbits with an effective orbit constant C eff and aspect ratio ℓ eff. These parameters are useful for characterizing the dynamics of curved capsules as a function of given deformability and curvature. As the capsule becomes more deformable or curved, C eff decreases, indicating a shift of the orbit towards log-rolling motion, while ℓ eff increases weakly as the degree of curvature increases but shows negligible dependency on deformability. These features are not changed substantially as the viscosity ratio between the inner and outer fluids is changed from 1 to 5. As cell deformability, cell shape, and cell-cell interactions are all pathologically altered in blood disorders such as SCD, these results will have clear implications on improving our understanding of the pathophysiology of hematologic disease.

Critical-Layer Structures and Mechanisms in Elastoinertial Turbulence.

Simulations of elastoinertial turbulence (EIT) of a polymer solution at low Reynolds number are shown to display localized polymer stretch fluctuations. These are very similar to structures arising from linear stability (Tollmien-Schlichting modes) and resolvent analyses, i.e., critical-layer structures localized where the mean fluid velocity equals the wave speed. Computations of self-sustained nonlinear Tollmien-Schlichting waves reveal that the critical layer exhibits stagnation points that generate sheets of large polymer stretch. These kinematics may be the genesis of similar structures in EIT.

Buckling Instabilities and Complex Trajectories in a Simple Model of Uniflagellar Bacteria.

Observations of uniflagellar bacteria show that buckling instabilities of the hook protein connecting the cell body and flagellum play a role in locomotion. To understand this phenomenon, we develop models at varying levels of description with a particular focus on the parameter dependence of the buckling instability. A key dimensionless group called the flexibility number measures the hook flexibility relative to the thrust exerted by the flagellum; this parameter and the geometric parameters of the cell determine the stability of straight swimming. Two very simple models amenable to analytical treatment are developed to examine buckling in stationary (pinned) and moving swimmers. We then consider a more detailed model incorporating a helical flagellum and the rotational degrees of freedom of the cell body and flagellum, and we use numerical simulations to map out the parameter dependence of the buckling instability. In all models, a bifurcation occurs as the flexibility number increases, separating equilibrium configurations into straight or bent, and for the full model, separating trajectories into straight or helical. More specifically for the latter, the critical flexibility marks the transition from periodicity to quasi-periodicity in the behavior of variables determining configuration. We also find that for a given body geometry, there is a specific flagellar geometry that minimizes the critical flexibility number at which buckling occurs. These results highlight the role of flexibility in the biology of real organisms and the engineering of artificial microswimmers.

Dynamics of Miura-patterned foldable sheets in shear flow.

We study the dynamics of piecewise rigid sheets containing predefined crease lines in shear flow. The crease lines act like hinge joints along which the sheet may fold rigidly, i.e. without bending any other crease line. We choose the crease lines such that they tessellate the sheet into a two-dimensional array of parallelograms. Specifically, we focus on a particular arrangement of crease lines known as a Miura-pattern in the origami community. When all the hinges are fully open the sheet is planar, whereas when all are closed the sheet folds over itself to form a compact flat structure. Due to rigidity constraints, the folded state of a Miura-sheet can be described using a single fold angle. The hinged sheet is modeled using the framework of constrained multibody systems in the absence of inertia. The hydrodynamic drag on each of the rigid panels is calculated based on an inscribed elliptic disk, but intra-panel hydrodynamic interactions are neglected. We find that when the motion of a sheet remains symmetric with respect to the flow-gradient plane, after a sufficiently long time, the sheet either exhibits asymptotically periodic tumbling and breathing, indicating approach to a limit cycle; or it reaches a steady state by completely unfolding, which we show to be a half-stable node in the phase space. In the case of asymmetric motion of the sheet with respect to the flow-gradient plane, we find that the terminal state of motion is one of - (i) steady state with a fully unfolded or fully folded configuration, (ii) asymptotically periodic tumbling, indicating approach to a limit cycle, (iii) cyclic tumbling without repetition, indicating a quasiperiodic orbit, or (iv) cyclic tumbling with repetition after several cycles, indicating a resonant quasiperiodic orbit. No chaotic behavior was found.

Low-dimensional representations of exact coherent states of the Navier-Stokes equations from the resolvent model of wall turbulence.

We report that many exact invariant solutions of the Navier-Stokes equations for both pipe and channel flows are well represented by just a few modes of the model of McKeon and Sharma [J. Fluid Mech. 658, 336 (2010)]. This model provides modes that act as a basis to decompose the velocity field, ordered by their amplitude of response to forcing arising from the interaction between scales. The model was originally derived from the Navier-Stokes equations to represent turbulent flows and has been used to explain coherent structure and to predict turbulent statistics. This establishes a surprising new link between the two distinct approaches to understanding turbulence.

Cellular softening mediates leukocyte demargination and trafficking, thereby increasing clinical blood counts.

Leukocytes normally marginate toward the vascular wall in large vessels and within the microvasculature. Reversal of this process, leukocyte demargination, leads to substantial increases in the clinical white blood cell and granulocyte count and is a well-documented effect of glucocorticoid and catecholamine hormones, although the underlying mechanisms remain unclear. Here we show that alterations in granulocyte mechanical properties are the driving force behind glucocorticoid- and catecholamine-induced demargination. First, we found that the proportions of granulocytes from healthy human subjects that traversed and demarginated from microfluidic models of capillary beds and veins, respectively, increased after the subjects ingested glucocorticoids. Also, we show that glucocorticoid and catecholamine exposure reorganizes cellular cortical actin, significantly reducing granulocyte stiffness, as measured with atomic force microscopy. Furthermore, using simple kinetic theory computational modeling, we found that this reduction in stiffness alone is sufficient to cause granulocyte demargination. Taken together, our findings reveal a biomechanical answer to an old hematologic question regarding how glucocorticoids and catecholamines cause leukocyte demargination. In addition, in a broader sense, we have discovered a temporally and energetically efficient mechanism in which the innate immune system can simply alter leukocyte stiffness to fine tune margination/demargination and therefore leukocyte trafficking in general. These observations have broad clinically relevant implications for the inflammatory process overall as well as hematopoietic stem cell mobilization and homing.

Shape-mediated margination and demargination in flowing multicomponent suspensions of deformable capsules.

We present detailed simulations and theory for flow-induced segregation in suspensions of deformable fluid-filled capsules with different shapes during simple shear flow in a planar slit. This system is an idealized model for transport for blood cells and/or drug carriers in the microcirculation or in microfluidic devices. For the simulations, an accelerated implementation of the boundary integral method was employed. We studied the binary mixtures of spherical and ellipsoidal capsules, varying the aspect ratio κ of the ellipsoid while keeping constant either (a) equatorial radius or (b) volume. Effects of a variety of parameters was studied, including κ, volume fraction and number fraction of the spherical capsules in the mixture. In suspensions where the ellipsoids have the same equatorial radius as the spheres, capsules with lower κ marginate. In suspension where the ellipsoids have the same volume as the spheres, ellipsoidal (both oblate and prolate) capsules are seen to demarginate in a mixture of primarily spherical capsules. To understand these results, a mechanistic framework based on the competition between wall-induced migration and shear-induced collisions is presented. A simplified drift-diffusion theory based on this framework shows excellent qualitative agreement with simulation results.