The STONE Curve: A ROC-Derived Model Performance Assessment Tool.

A new model validation and performance assessment tool is introduced, the sliding threshold of observation for numeric evaluation (STONE) curve. It is based on the relative operating characteristic (ROC) curve technique, but instead of sorting all observations in a categorical classification, the STONE tool uses the continuous nature of the observations. Rather than defining events in the observations and then sliding the threshold only in the classifier/model data set, the threshold is changed simultaneously for both the observational and model values, with the same threshold value for both data and model. This is only possible if the observations are continuous and the model output is in the same units and scale as the observations, that is, the model is trying to exactly reproduce the data. The STONE curve has several similarities with the ROC curve-plotting probability of detection against probability of false detection, ranging from the (1,1) corner for low thresholds to the (0,0) corner for high thresholds, and values above the zero-intercept unity-slope line indicating better than random predictive ability. The main difference is that the STONE curve can be nonmonotonic, doubling back in both the x and y directions. These ripples reveal asymmetries in the data-model value pairs. This new technique is applied to modeling output of a common geomagnetic activity index as well as energetic electron fluxes in the Earth's inner magnetosphere. It is not limited to space physics applications but can be used for any scientific or engineering field where numerical models are used to reproduce observations.

Direct evidence of nonstationary collisionless shocks in space plasmas.

Collisionless shocks are ubiquitous throughout the universe: around stars, supernova remnants, active galactic nuclei, binary systems, comets, and planets. Key information is carried by electromagnetic emissions from particles accelerated by high Mach number collisionless shocks. These shocks are intrinsically nonstationary, and the characteristic physical scales responsible for particle acceleration remain unknown. Quantifying these scales is crucial, as it affects the fundamental process of redistributing upstream plasma kinetic energy into other degrees of freedom-particularly electron thermalization. Direct in situ measurements of nonstationary shock dynamics have not been reported. Thus, the model that best describes this process has remained unknown. Here, we present direct evidence demonstrating that the transition to nonstationarity is associated with electron-scale field structures inside the shock ramp.

Space Radiation and Plasma Effects on Satellites and Aviation: Quantities and Metrics for Tracking Performance of Space Weather Environment Models.

The Community Coordinated Modeling Center has been leading community-wide space science and space weather model validation projects for many years. These efforts have been broadened and extended via the newly launched International Forum for Space Weather Modeling Capabilities Assessment (https://ccmc.gsfc.nasa.gov/assessment/). Its objective is to track space weather models' progress and performance over time, a capability that is critically needed in space weather operations and different user communities in general. The Space Radiation and Plasma Effects Working Team of the aforementioned International Forum works on one of the many focused evaluation topics and deals with five different subtopics (https://ccmc.gsfc.nasa.gov/assessment/topics/radiation-all.php) and varieties of particle populations: Surface Charging from tens of eV to 50-keV electrons and internal charging due to energetic electrons from hundreds keV to several MeVs. Single-event effects from solar energetic particles and galactic cosmic rays (several MeV to TeV), total dose due to accumulation of doses from electrons (>100 keV) and protons (>1 MeV) in a broad energy range, and radiation effects from solar energetic particles and galactic cosmic rays at aviation altitudes. A unique aspect of the Space Radiation and Plasma Effects focus area is that it bridges the space environments, engineering, and user communities. The intent of the paper is to provide an overview of the current status and to suggest a guide for how to best validate space environment models for operational/engineering use, which includes selection of essential space environment and effect quantities and appropriate metrics.