Deep Learning and Symbolic Regression for Discovering Parametric Equations.

Symbolic regression is a machine learning technique that can learn the equations governing data and thus has the potential to transform scientific discovery. However, symbolic regression is still limited in the complexity and dimensionality of the systems that it can analyze. Deep learning, on the other hand, has transformed machine learning in its ability to analyze extremely complex and high-dimensional datasets. We propose a neural network architecture to extend symbolic regression to parametric systems where some coefficient may vary, but the structure of the underlying governing equation remains constant. We demonstrate our method on various analytic expressions and partial differential equations (PDEs) with varying coefficients and show that it extrapolates well outside of the training domain. The proposed neural-network-based architecture can also be enhanced by integrating with other deep learning architectures such that it can analyze high-dimensional data while being trained end-to-end. To this end, we demonstrate the scalability of our architecture by incorporating a convolutional encoder to analyze 1-D images of varying spring systems.

Discovering conservation laws using optimal transport and manifold learning.

Conservation laws are key theoretical and practical tools for understanding, characterizing, and modeling nonlinear dynamical systems. However, for many complex systems, the corresponding conserved quantities are difficult to identify, making it hard to analyze their dynamics and build stable predictive models. Current approaches for discovering conservation laws often depend on detailed dynamical information or rely on black box parametric deep learning methods. We instead reformulate this task as a manifold learning problem and propose a non-parametric approach for discovering conserved quantities. We test this new approach on a variety of physical systems and demonstrate that our method is able to both identify the number of conserved quantities and extract their values. Using tools from optimal transport theory and manifold learning, our proposed method provides a direct geometric approach to identifying conservation laws that is both robust and interpretable without requiring an explicit model of the system nor accurate time information.

Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discovery.

Symbolic regression is a powerful technique to discover analytic equations that describe data, which can lead to explainable models and the ability to predict unseen data. In contrast, neural networks have achieved amazing levels of accuracy on image recognition and natural language processing tasks, but they are often seen as black-box models that are difficult to interpret and typically extrapolate poorly. In this article, we use a neural network-based architecture for symbolic regression called the equation learner (EQL) network and integrate it with other deep learning architectures such that the whole system can be trained end-to-end through backpropagation. To demonstrate the power of such systems, we study their performance on several substantially different tasks. First, we show that the neural network can perform symbolic regression and learn the form of several functions. Next, we present an MNIST arithmetic task where a convolutional network extracts the digits. Finally, we demonstrate the prediction of dynamical systems where an unknown parameter is extracted through an encoder. We find that the EQL-based architecture can extrapolate quite well outside of the training data set compared with a standard neural network-based architecture, paving the way for deep learning to be applied in scientific exploration and discovery.

Extraordinary optical transmission inside a waveguide: spatial mode dependence.

We study the influence of the input spatial mode on the extraordinary optical transmission (EOT) effect. By placing a metal screen with a 1D array of subwavelength holes inside a terahertz (THz) parallel-plate waveguide (PPWG), we can directly compare the transmission spectra with different input waveguide modes. We observe that the transmitted spectrum depends strongly on the input mode. A conventional description of EOT based on the excitation of surface plasmons is not predictive in all cases. Instead, we utilize a formalism based on impedance matching, which accurately predicts the spectral resonances for both TEM and non-TEM input modes.

Energy loss at propagating jamming fronts in granular gas clusters.

We explore the initial moments of impact between two dense granular clusters in a two-dimensional geometry. The particles are composed of solid CO(2) and are levitated on a hot surface. Upon collision, the propagation of a dynamic "jamming front" produces a distinct regime for energy dissipation in a granular gas in which the translational kinetic energy decreases by over 90%. Experiments and associated simulations show that the initial loss of kinetic energy obeys a power law in time ΔE = -Kt(3/2), a form that can be predicted from kinetic arguments.

Collision dynamics of particle clusters in a two-dimensional granular gas.

In a granular gas, inelastic collisions produce an instability in which the constituent particles cluster heterogeneously. These clusters then interact with each other, further decreasing their kinetic energy. We report experiments of the free collisions of dense clusters of particles in a two-dimensional geometry. The particles are composed of solid CO(2), which float nearly frictionlessly on a hot surface due to sublimated vapor. After two dense clusters of ≈100 particles collide, there are two distinct stages of evolution. First, the translational kinetic energy rapidly decreases by over 90% as a "jamming front" sweeps across each cluster. Subsequently, the kinetic energy decreases more slowly as the particles approach the container boundaries. In this regime, the measured velocity distributions are non-Gaussian with long tails. Finally, we compare our experiments to computer simulations of colliding, two-dimensional, granular clusters composed of circular, viscoelastic particles with friction.

Steps taken to protect and rescue cryopreserved embryos during Hurricane Katrina.

Hurricane Katrina was a devastating storm that brought lessons in emergency preparedness. All IVF programs should have a plan to protect fresh and cryopreserved embryos in case of natural or human-made disaster.

Risk factors for high-order multiple pregnancy and multiple birth after controlled ovarian hyperstimulation: results of 4,062 intrauterine insemination cycles.

OBJECTIVE: To determine factors responsible for high-order multiple pregnancy (HOMP) and high-order multiple births when multiple cycles of controlled ovarian hyperstimulation-IUI (COH-IUI) are performed. DESIGN: Retrospective analysis. SETTING: Private infertility clinic. PATIENT(S): Women (n = 2,272) who underwent 4,067 consecutive COH-IUI cycles. INTERVENTION(S): None. MAIN OUTCOME MEASURE(S): High-order multiple pregnancy rate, pregnancy rate (PR), and birth rate (PR) per cycle. RESULT(S): High-order multiple pregnancy was related to number of follicles of diameter > or = 10 mm, age, and treatment cycle. For age <32 years, HOMP was 6% for three to six follicles and 20% for seven or more follicles. For ages 32 to 37 years, HOMP was 5% for three to six follicles and 12% for seven or more follicles. In the first COH-IUI cycle, HOMP was 8% for three to six follicles and 15% for seven or more follicles. In the second cycle, HOMP did not occur unless there were more than six follicles. No HOMP occurred after the second cycle. Pregnancy rate did not increase significantly when there were more than four follicles. Continuing COH-IUI past the third cycle resulted in additional pregnancies in patients with one to eight follicles. CONCLUSION(S): High-order multiple pregnancy can be predicted by age and number of follicles of diameter > or = 10 mm. Controlled ovarian hyperstimulation is not necessary to achieve satisfactory overall pregnancy rates if ovulation induction is continued past the third cycle in low responders.

Clomiphene citrate intrauterine insemination (IUI) before gonadotropin IUI affects the pregnancy rate and the rate of high-order multiple pregnancies.

OBJECTIVE: To determine if previous treatment with clomiphene citrate intrauterine insemination (CC-IUI) affects pregnancy and high-order multiple pregnancy (HOMP) rates in subsequent hMG-IUI or FSH-IUI cycles. DESIGN: Retrospective cohort study. SETTING: Private infertility clinic. PATIENT(S): Five hundred fifty-one patients (age <38 years) without tubal factor infertility, treated with 918 cycles of hMG/FSH-IUI after one or more unsuccessful cycles of CC-IUI; 908 patients treated with 1459 cycles of hMG/FSH-IUI without prior CC-IUI. INTERVENTION(S): CC-IUI, hMG-IUI, FSH-IUI. MAIN OUTCOME MEASURE(S): Pregnancy rate per cycle, HOMP (three or more gestational sacs). RESULT(S): Pregnancy rates during the first three hMG-IUI or FSH-IUI cycles averaged 21.8 +/- 1.1% without previous CC-IUI, 19.6 +/- 1.3% after one to four cycles of CC-IUI, and 3.6 +/- 2.6% after >or= five previous CC-IUI cycles. The HOMP rates were 8.8% without previous CC-IUI, 7.5% after one, 5.7% after two and <2.6% (0 out of 38) after >or= three previous CC-IUI cycles. CONCLUSION(S): Pregnancy rates in hMG/FSH-IUI cycles are significantly reduced after four unsuccessful CC-IUI cycles. High-order multiple pregnancies due to hMG/HMG-IUI are reduced following previous unsuccessful CC-IUI cycles.

Effect of diagnosis, age, sperm quality, and number of preovulatory follicles on the outcome of multiple cycles of clomiphene citrate-intrauterine insemination.

OBJECTIVE: To determine how diagnosis, age, sperm quality, and number of preovulatory follicles affect pregnancy rates when multiple cycles of clomiphene citrate (CC)-IUI are performed. DESIGN: Fifteen-year prospective observational study. SETTING: Private infertility clinic. PATIENT(S): Three thousand, three hundred eighty-one cycles of husband or donor IUI. INTERVENTION(S): Ovulation induction with CC and IUI. MAIN OUTCOME MEASURE(S): Per-cycle pregnancy rate (PR), cumulative pregnancy rate (CPR). RESULT(S): Pregnancy rates remained constant through four cycles, then fell significantly for diagnoses other than ovulatory dysfunction. Mean PRs for cycles 1-4 were significantly lower for patients with the following characteristics: age >/=43 years, poor semen quality, single preovulatory follicles, and diagnoses other than ovulatory dysfunction. Additional cycles of CC-IUI compensated for low PRs because of age, semen quality, or number of follicles. After four cycles, CPRs were 46% for ovulatory dysfunction; 38% for cervical factor, male factor, and unexplained infertility; 34% for endometriosis; and 26% for tubal factor. After six cycles, CPRs were 65% for ovulation dysfunction, 35% for endometriosis, and unchanged for other diagnoses. CONCLUSION(S): Clomiphene citrate-intrauterine insemination should be performed for a minimum of four cycles. Additional cycles of CC-IUI can compensate for low pregnancy rates due to age, semen quality, or follicle number in patients with ovulation dysfunction.

Spontaneous reduction of multiple pregnancy: incidence and effect on outcome.

OBJECTIVE: Our objective was to determine the incidence of spontaneous reduction in multiple pregnancies during the first 12 gestational weeks and determine the outcome of the surviving fetuses. STUDY DESIGN: Analysis of prospectively collected ultrasound and birth information on 709 multiple and 5962 singleton pregnancies conceived at a private infertility clinic. RESULTS: Spontaneous reduction of one or more gestational sacs and or embryos occurred before the 12th week of gestation in 36% of twin (95% CI, 32%-40%), 53% of triplet (95% CI, 44%-61%), and 65% of quadruplet (95% CI, 46%-85%) pregnancies. Reduction was less frequent after ovulation induction than after spontaneous ovulation. In general, pregnancy duration and birth weight were inversely related to the initial gestational sac number irrespective of the final birth number. CONCLUSIONS: More than 50% of patients with 3 or more gestational sacs had spontaneous reduction before 12 weeks. The surviving fetuses weighed less and were born earlier than unreduced pregnancies with the same initial number of fetuses.