Replication Data for: Modeling monophthongal versus diphthongal /aɪ/ in sung vocal performance with interpretable machine learning

Use email button above to contact.

This dataset contains replication data for the study "Modeling monophthongal versus diphthongal /aɪ/ in sung vocal performance with interpretable machine learning" (De Timmerman & Verbeke, 2026). The study investigates how binary perceptual annotations of monophthongal versus diphthongal /aɪ/ relate to measurable acoustic variation in sung vocal performance. Using a corpus of studio-recorded blues music processed with vocal-instrumental source separation, F1 and F2 formant trajectories were extracted for 1,004 /aɪ/ tokens, each perceptually categorized by a trained annotator as either monophthongal or diphthongal. These data were used to train and interpret two machine-learning models predicting the perceptual labels: a gradient boosted decision tree (XGBoost) trained on a set of engineered acoustic features, and a multilayer perceptron (MLP) trained directly on the raw, time-aligned formant trajectories.

This study combines modern source-separation techniques with interpretable machine learning methods to investigate how binary perceptual annotations of diphthongal and monophthongal /aɪ/ relate to measurable acoustic variation in sung vocal performance. Using a corpus of studio-recorded music processed with vocal-instrumental source separation, we extract F1 and F2 trajectories for 1,004 /aɪ/ tokens, each perceptually categorized as either monophthongal or diphthongal. These trajectories were modeled using two approaches: (i) a gradient boosted decision tree trained on engineered acoustic features, and (ii) a multilayer perceptron trained directly on raw formant trajectories. Both models achieved high accuracy and AUC-ROC, indicating that perceptual labels can reliably be predicted from both raw and engineered acoustic input. Moreover, a SHAP-based feature importance analysis showed that features such as Delta F1/Delta F2, cubic spline coefficients and trajectory derivatives captured systematic differences between perceived monophthongal and diphthongal tokens, highlighting the value of dynamic representations over static distance-based measures. The results indicate that perceptual annotations of diphthongal and monophthongal /aɪ/ correspond with quantifiable acoustic information and demonstrate how explainable machine learning can help map gradient vowel dynamics onto binary perceptual categories. The study further emphasizes how recent advances in source separation make sung performance a valuable new domain for phonetic research.

GitHub repository containing python scripts/notebooks which were used to analyze this data:

TROLLing repository containing original corpus data from which the vowel tokens were sampled:

This dataset is dedicated to the public domain under CC0.