Disappointing agreement in the interpretation of neurological adverse events following immunization with SARS-CoV-2 vaccines. A case-series study

Background: A sizable proportion of the world's population has been vaccinated to prevent SARS-CoV-2 infections. In clinical practice, however, almost any neurological manifestation occurring after vaccination has been attributed to the vaccine, generating doubts on their safety. In 2013, the WHO created the Adverse Event Following Immunization (AEFI) criteria to establish the relationship between a vaccine and side effects, but they seem not to dispel doubts regarding severity and causality of neurological events following SARS-CoV-2 vaccination. This study aims to analyze consistency of the AEFI to designate probable side effects of SARSCoV-2 vaccines among patients who develop neurological symptoms after vaccination. Methods: We measured the level of agreement using the Fleiss kappa methodology. Seventeen observers (five neurologists, four infectious disease specialists, and eight internal medicine residents) independently rated 11 cases treated in our service presenting neurological symptoms within 21 days after SARS-CoV-2 vaccination according to the AEFI criteria. Results: We found low concordance agreements among the 17 raters regarding severity (k=0.088) and causality (k=0.025). When analyzing by raters' subgroups, agreement for severity was moderately higher among neurologists (k=0.383) and for causality among internal medicine residents (k=0.117). Conclusions: AEFI criteria do not allow, by themselves, to establish the relationship between a vaccine and adverse neurological effects. Therefore, it is essential to create more useful tools that allow doctors and lay people to be more confident in this challenge.