A Tempered Particle Filter to Enhance the Assimilation of SAR-Derived Flood Extent Maps Into Flood Forecasting Models.

Data assimilation (DA) is a powerful tool to optimally combine uncertain model simulations and observations. Among DA techniques, the particle filter (PF) has gained attention for its capacity to deal with nonlinear systems and for its relaxation of the Gaussian assumption. However, the PF may suffer from degeneracy and sample impoverishment. In this study, we propose an innovative approach, based on a tempered particle filter (TPF), aiming at mitigating PFs issues, thus extending over time the assimilation benefits. Probabilistic flood maps derived from synthetic aperture radar data are assimilated into a flood forecasting model through an iterative process including a particle mutation in order to keep diversity within the ensemble. Results show an improvement of the model forecasts accuracy, with respect to the Open Loop: on average the root mean square error (RMSE) of water levels decrease by 80% at the assimilation time and by 60% 2 days after the assimilation. A comparison with the Sequential Importance Sampling (SIS) is carried out showing that although SIS performances are generally comparable to the TPF ones at the assimilation time, they tend to decrease more quickly. For instance, on average TPF-based RMSE are 20% lower compared to the SIS-based ones 2 days after the assimilation. The application of the TPF determines higher critical success index values compared to the SIS. On average the increase in performances lasts for almost 3 days after the assimilation. Our study provides evidence that the application of the variant of the TPF enables more persistent benefits compared to the SIS.

Dynamics and Predictability of Tropical Cyclone Rapid Intensification in Ensemble Simulations of Hurricane Patricia (2015).

Hurricane Patricia (2015) over the eastern Pacific was a record-breaking tropical cyclone (TC) under a very favorable environment during its rapid intensification (RI) period, which makes it an optimal real case for studying RI dynamics and predictability. In this study, we performed ensemble Kalman filter analyses at Patricia's early development stage using both traditional observations and the Office of Naval Research Tropical Cyclone Intensity (TCI) field campaign data. It is shown that assimilating the inner-core TCI observations produces a stronger initial vortex and significantly improves the prediction of RI. Analysis of observation sensitivity experiments shows that the deep-layer dropsonde observations have high impact on both the primary and secondary circulations for the entire troposphere while the radar observations have the most impact on the primary circulations near aircraft flight level. A wide range of intensification scenarios are obtained through two sets of ensemble forecasts initialized with and without assimilating the TCI data prior to the RI onset. Verification of the ensemble forecasts against the TCI observations during the RI period shows that forecast errors toward later stages can originate from two different error sources at early stages of the vortex structure: One is a timing error from a delayed vortex development such that the TC evolution is the same but shifted in time; the other is due to a totally different storm such that there is no moment in time the simulated storm can obtain a correct TC structure.

Rain-Induced Stratification of the Equatorial Indian Ocean and Its Potential Feedback to the Atmosphere.

Surface freshening through precipitation can act to stably stratify the upper ocean, forming a rain layer (RL). RLs inhibit subsurface vertical mixing, isolating deeper ocean layers from the atmosphere. This process has been studied using observations and idealized simulations. The present ocean modeling study builds upon this body of work by incorporating spatially resolved and realistic atmospheric forcing. Fine-scale observations of the upper ocean collected during the Dynamics of the Madden-Julian Oscillation field campaign are used to verify the General Ocean Turbulence Model (GOTM). Spatiotemporal characteristics of equatorial Indian Ocean RLs are then investigated by forcing a 2D array of GOTM columns with realistic and well-resolved output from an existing regional atmospheric simulation. RL influence on the ocean-atmosphere system is evaluated through analysis of RL-induced modification to surface fluxes and sea surface temperature (SST). This analysis demonstrates that RLs cool the ocean surface on time scales longer than the associated precipitation event. A second simulation with identical atmospheric forcing to that in the first, but with rainfall set to zero, is performed to investigate the role of rain temperature and salinity stratification in maintaining cold SST anomalies within RLs. Approximately one third, or 0.1°C, of the SST reduction within RLs can be attributed to rain effects, while the remainder is attributed to changes in atmospheric temperature and humidity. The prolonged RL-induced SST anomalies enhance SST gradients that have been shown to favor the initiation of atmospheric convection. These findings encourage continued research of RL feedbacks to the atmosphere.

A particle flow filter for high-dimensional system applications.

A novel particle filter proposed recently, the particle flow filter (PFF), avoids the long-existing weight degeneracy problem in particle filters and, therefore, has great potential to be applied in high-dimensional systems. The PFF adopts the idea of a particle flow, which sequentially pushes the particles from the prior to the posterior distribution, without changing the weight of each particle. The essence of the PFF is that it assumes the particle flow is embedded in a reproducing kernel Hilbert space, so that a practical solution for the particle flow is obtained. The particle flow is independent of the choice of kernel in the limit of an infinite number of particles. Given a finite number of particles, we have found that a scalar kernel fails in high-dimensional and sparsely observed settings. A new matrix-valued kernel is proposed that prevents the collapse of the marginal distribution of observed variables in a high-dimensional system. The performance of the PFF is tested and compared with a well-tuned local ensemble transform Kalman filter (LETKF) using the 1,000-dimensional Lorenz 96 model. It is shown that the PFF is comparable to the LETKF for linear observations, except that explicit covariance inflation is not necessary for the PFF. For nonlinear observations, the PFF outperforms LETKF and is able to capture the multimodal likelihood behavior, demonstrating that the PFF is a viable path to fully nonlinear geophysical data assimilation.

Effects of mis-specified time-correlated model error in the (ensemble) Kalman Smoother.

Data assimilation is often performed under the perfect model assumption. Although there is an increasing amount of research accounting for model errors in data assimilation, the impact of an incorrect specification of the model errors on the data assimilation results has not been thoroughly assessed. We investigate the effect that an inaccurate time correlation in the model error description can have on data assimilation results, deriving analytical results using a Kalman Smoother for a one-dimensional system. The analytical results are evaluated numerically to generate useful illustrations. For a higher-dimensional system, we use an ensemble Kalman Smoother. Strong dependence on observation density is found. For a single observation at the end of the window, the posterior variance is a concave function of the guessed decorrelation time-scale used in the data assimilation process. This is due to an increasing prior variance with that time-scale, combined with a decreasing tendency from larger observation influence. With an increasing number of observations, the posterior variance decreases with increasing guessed decorrelation time-scale because the prior variance effect becomes less important. On the other hand, the posterior mean-square error has a convex shape as a function of the guessed time-scale with a minimum where the guessed time-scale is equal to the real decorrelation time-scale. With more observations, the impact of the difference between two decorrelation time-scales on the posterior mean-square error reduces. Furthermore, we show that the correct model error decorrelation time-scale can be estimated over several time windows using state augmentation in the ensemble Kalman Smoother. Since model errors are significant and significantly time correlated in real geophysical systems such as the atmosphere, this contribution opens up a next step in improving prediction of these systems.

Observational Constraints on Warm Cloud Microphysical Processes Using Machine Learning and Optimization Techniques.

We introduce new parameterizations for autoconversion and accretion rates that greatly improve representation of the growth processes of warm rain. The new parameterizations capitalize on machine-learning and optimization techniques and are constrained by in situ cloud probe measurements from the recent Atmospheric Radiation Measurement Program field campaign at Azores. The uncertainty in the new estimates of autoconversion and accretion rates is about 15% and 5%, respectively, outperforming existing parameterizations. Our results confirm that cloud and drizzle water content are the most important factors for determining accretion rates. However, for autoconversion, in addition to cloud water content and droplet number concentration, we discovered a key role of drizzle number concentration that is missing in current parameterizations. The robust relation between autoconversion rate and drizzle number concentration is surprising but real, and furthermore supported by theory. Thus, drizzle number concentration should be considered in parameterizations for improved representation of the autoconversion process.

A framework for causal discovery in non-intervenable systems.

Many frameworks exist to infer cause and effect relations in complex nonlinear systems, but a complete theory is lacking. A new framework is presented that is fully nonlinear, provides a complete information theoretic disentanglement of causal processes, allows for nonlinear interactions between causes, identifies the causal strength of missing or unknown processes, and can analyze systems that cannot be represented on directed acyclic graphs. The basic building blocks are information theoretic measures such as (conditional) mutual information and a new concept called certainty that monotonically increases with the information available about the target process. The framework is presented in detail and compared with other existing frameworks, and the treatment of confounders is discussed. While there are systems with structures that the framework cannot disentangle, it is argued that any causal framework that is based on integrated quantities will miss out potentially important information of the underlying probability density functions. The framework is tested on several highly simplified stochastic processes to demonstrate how blocking and gateways are handled and on the chaotic Lorentz 1963 system. We show that the framework provides information on the local dynamics but also reveals information on the larger scale structure of the underlying attractor. Furthermore, by applying it to real observations related to the El-Nino-Southern-Oscillation system, we demonstrate its power and advantage over other methodologies.

Particle filters for high-dimensional geoscience applications: A review.

Particle filters contain the promise of fully nonlinear data assimilation. They have been applied in numerous science areas, including the geosciences, but their application to high-dimensional geoscience systems has been limited due to their inefficiency in high-dimensional systems in standard settings. However, huge progress has been made, and this limitation is disappearing fast due to recent developments in proposal densities, the use of ideas from (optimal) transportation, the use of localization and intelligent adaptive resampling strategies. Furthermore, powerful hybrids between particle filters and ensemble Kalman filters and variational methods have been developed. We present a state-of-the-art discussion of present efforts of developing particle filters for high-dimensional nonlinear geoscience state-estimation problems, with an emphasis on atmospheric and oceanic applications, including many new ideas, derivations and unifications, highlighting hidden connections, including pseudo-code, and generating a valuable tool and guide for the community. Initial experiments show that particle filters can be competitive with present-day methods for numerical weather prediction, suggesting that they will become mainstream soon.

An ensemble framework for time delay synchronization.

Synchronization based state estimation tries to synchronize a model with the true evolution of a system via the observations. In practice, an extra term is added to the model equations which hampers growth of instabilities transversal to the synchronization manifold. Therefore, there is a very close connection between synchronization and data assimilation. Recently, synchronization with time-delayed observations has been proposed, in which observations at future times are used to help synchronize a system that does not synchronize using only present observations, with remarkable successes. Unfortunately, these schemes are limited to small-dimensional problems. In this article, we lift that restriction by proposing an ensemble-based synchronization scheme. Tests were performed using the Lorenz'96 model for 20-, 100- and 1000-dimension systems. Results show global synchronization errors stabilizing at values of at least an order of magnitude lower than the observation errors, suggesting that the scheme is a promising tool to steer model states to the truth. While this framework is not a complete data assimilation method, we develop this methodology as a potential choice for a proposal density in a more comprehensive data assimilation method, like a fully nonlinear particle filter.

Time-correlated model error in the (ensemble) Kalman smoother.

Data assimilation is often performed in a perfect-model scenario, where only errors in initial conditions and observations are considered. Errors in model equations are increasingly being included, but typically using rather adhoc approximations with limited understanding of how these approximations affect the solution and how these approximations interfere with approximations inherent in finite-size ensembles. We provide the first systematic evaluation of the influence of approximations to model errors within a time window of weak-constraint ensemble smoothers. In particular, we study the effects of prescribing temporal correlations in the model errors incorrectly in a Kalman smoother, and in interaction with finite-ensemble-size effects in an ensemble Kalman smoother. For the Kalman smoother we find that an incorrect correlation time-scale for additive model errors can have substantial negative effects on the solutions, and we find that overestimating of the correlation time-scale leads to worse results than underestimating. In the ensemble Kalman smoother case, the resulting ensemble-based space-time gain can be written as the true gain multiplied by two factors, a linear factor containing the errors due to both time-correlation errors and finite ensemble effects, and a nonlinear factor related to the inverse part of the gain. Assuming that both errors are relatively small, we are able to disentangle the contributions from the different approximations. The analysis mean is affected by the time-correlation errors, but also substantially by finite-ensemble effects, which was unexpected. The analysis covariance is affected by both time-correlation errors and an in-breeding term. This first thorough analysis of the influence of time-correlation errors and finite-ensemble-size errors on weak-constraint ensemble smoothers will aid further development of these methods and help to make them robust for e.g. numerical weather prediction.

Data Assimilation in the Solar Wind: Challenges and First Results.

Data assimilation (DA) is used extensively in numerical weather prediction (NWP) to improve forecast skill. Indeed, improvements in forecast skill in NWP models over the past 30 years have directly coincided with improvements in DA schemes. At present, due to data availability and technical challenges, DA is underused in space weather applications, particularly for solar wind prediction. This paper investigates the potential of advanced DA methods currently used in operational NWP centers to improve solar wind prediction. To develop the technical capability, as well as quantify the potential benefit, twin experiments are conducted to assess the performance of the Local Ensemble Transform Kalman Filter (LETKF) in the solar wind model ENLIL. Boundary conditions are provided by the Wang-Sheeley-Arge coronal model and synthetic observations of density, temperature, and momentum generated every 4.5 h at 0.6 AU. While in situ spacecraft observations are unlikely to be routinely available at 0.6 AU, these techniques can be applied to remote sensing of the solar wind, such as with Heliospheric Imagers or interplanetary scintillation. The LETKF can be seen to improve the state at the observation location and advect that improvement toward the Earth, leading to an improvement in forecast skill in near-Earth space for both the observed and unobserved variables. However, sharp gradients caused by the analysis of a single observation in space resulted in artificial wavelike structures being advected toward Earth. This paper is the first attempt to apply DA to solar wind prediction and provides the first in-depth analysis of the challenges and potential solutions.