Insights into the predictions of a machine learning classifier for gravitational-wave events

Nayyer Raza ( Université McGill )

GWSkyNet-Multi is a machine learning model developed for classification of candidate gravitational wave event sources as detected by the LIGO, Virgo, and KAGRA (LVK) detectors. The model uses the limited information released in the early public alerts for events, and produces prediction scores for whether the source is a real merger of two black holes, a real merger involving a neutron star, or a non-astrophysical instrumental glitch event. With test set accuracies of more than 90% for all source types, it is a powerful tool that can facilitate astronomers in deciding whether to perform time-sensitive electromagnetic follow-up observations of candidate events in the upcoming fourth LVK observing run. But what exactly is the model learning and how is it leveraging the limited information available to make these accurate predictions? As a deep learning neural network the inner workings of the model are essentially a black-box and difficult to interpret, affecting our trust in the model’s validity and robustness. In our work we tackle this issue by systematically testing and perturbing the model and its inputs to explain what underlying features and correlations it has learned, if any, for distinguishing the sources. We show that the localization area of the 2D sky maps and the computed coherence versus incoherence Bayes factors are used as strong predictors for distinguishing between real events and glitches, and that the estimated distance to the source can further be used to discriminate between binary black hole mergers and mergers involving neutron stars. Our results increase our trust in the model’s applicability to future observations, and inform potential avenues for further optimization.