Language Log

Laurent Mottron, "A radical change in our autism research strategy is needed: Back to prototypes", Autism Research 6/2/2021:

ABSTRACT: The evolution of autism diagnosis, from its discovery to its current delineation using standardized instruments, has been paralleled by a steady increase in its prevalence and heterogeneity. In clinical settings, the diagnosis of autism is now too vague to specify the type of support required by the concerned individuals. In research, the inclusion of individuals categorically defined by over-inclusive, polythetic criteria in autism cohorts results in a population whose heterogeneity runs contrary to the advancement of scientific progress. Investigating individuals sharing only a trivial resemblance produces a large-scale type-2 error (not finding differences between autistic and dominant population) rather than detecting mechanistic differences to explain their phenotypic divergences. The dimensional approach of autism proposed to cure the disease of its categorical diagnosis is plagued by the arbitrariness of the dimensions under study. Here, we argue that an emphasis on the reliability rather than specificity of diagnostic criteria and the misuse of diagnostic instruments, which ignore the recognition of a prototype, leads to confound autism with the entire range of neurodevelopmental conditions and personality variants. We propose centering research on cohorts in which individuals are selected based on their expert judged prototypicality to advance the theoretical and practical pervasive issues pertaining to autism diagnostic thresholds. Reversing the current research strategy by giving more weight to specificity than reliability should increase our ability to discover the mechanisms of autism.

A colleague pointed me to Terje Holmlund et al., "Applying speech technologies to assess verbal memory in patients with serious mental illness", NPJ digital medicine 2020:

Verbal memory deficits are some of the most profound neurocognitive deficits associated with schizophrenia and serious mental illness in general. As yet, their measurement in clinical settings is limited to traditional tests that allow for limited administrations and require substantial resources to deploy and score. Therefore, we developed a digital ambulatory verbal memory test with automated scoring, and repeated self-administration via smart devices. One hundred and four adults participated, comprising 25 patients with serious mental illness and 79 healthy volunteers. The study design was successful with high quality speech recordings produced to 92% of prompts (Patients: 86%, Healthy: 96%). The story recalls were both transcribed and scored by humans, and scores generated using natural language processing on transcriptions were comparable to human ratings (R = 0.83, within the range of human-to-human correlations of R = 0.73–0.89). A fully automated approach that scored transcripts generated by automatic speech recognition produced comparable and accurate scores (R = 0.82), with very high correlation to scores derived from human transcripts (R = 0.99). This study demonstrates the viability of leveraging speech technologies to facilitate the frequent assessment of verbal memory for clinical monitoring purposes in psychiatry.

This is great work, but over-interpretation of such results is likely to be a problem. At this stage in the development of the technologies, experimenting with with speech-to-text in such applications is a very good idea, but relying on it without accurate human-corrected transcripts is a very bad idea.