Replication data for: Who needs particles? A challenge to the classification of particles as a part of speech in Russiandoi:10.18710/700FNVDataverseNO2016-03-291Endresen, Anna; Janda, Laura A.; Reynolds, Robert; Tyers, Francis M., 2016, "Replication data for: Who needs particles? A challenge to the classification of particles as a part of speech in Russian", https://doi.org/10.18710/700FNV, DataverseNO, V1Replication data for: Who needs particles? A challenge to the classification of particles as a part of speech in Russiandoi:10.18710/700FNVEndresen, AnnaJanda, Laura A.Reynolds, RobertTyers, Francis M.UiT The Arctic University of Norway2016222506DataverseNOThe Tromsø Repository of Language and Linguistics (TROLLing)Janda, Laura A.2016-03-282016Arts and HumanitiesRussianHidden Markov ModelField: LexisTime-depth: synchronicTopic: particlesIn 1985, Zwicky argued that “particle” is a pretheoretical notion that should be eliminated from linguistic analysis. We propose a reclassification of Russian particles that implements Zwicky’s directive. Russian particles lack a coherent conceptual basis as a category and many are ambiguous with respect to part of speech. Our corpus analysis of Russian particles addresses theoretical questions about the cognitive status of parts of speech and practical concerns about how particles should be represented in computational models. We focus on nine high-frequency words commonly classed as particles: ešče, tak, ved’, slovno, daže, že, li, da, net. We show that current tagging of particles in the manually disambiguated Morphological Standard of the Russian National Corpus (RNC) is not entirely consistent, and that this can create challenges for training a part-of-speech tagger. We offer an alternative tagging scheme that eliminates the category of “particle” altogether. We show that our enriched scheme makes it possible for a part-of-speech tagger to achieve more useful results. Our analysis of particles provides a detailed account of various sub-uses that correspond to different parts of speech, their relationships, and relative distribution. In this sense, our study also contributes to the study of words that exhibit part-of-speech ambigu
ities. We construct a database by extracting from the RNC gold standard 100 random sentences for each of the nine focus words. This database is used for both training and testing a Hidden Markov Model (HMM) trigram tagger (Halácsy et al. 2007), which is the standard model for training part-of-speech tagging. This is done in two rounds: in Experiment 1 we use the tagging of the nine words as in the RNC, including the use of “particle” as a tag; in Experiment 2 we use our own tagging scheme which eliminates “particle” as a tag. In both experiments we partition our database into ten chunks and perform a ten-fold cross-validation, each time using 90 sentences as the training set and 10 sentences as the test set. This means that each part of the total set is tested in the course of the ten repetitions of training and testing.RussiaRussiacorpusEndresen, A., Janda, L. A., Reynolds, R., & Tyers, F. M. (2016). Who needs particles? A challenge to the classification of particles as a part of speech in Russian / Кому нужны частицы? Стоит ли определять частицы как отдельную часть речи в русском языке? Russian Linguistics, 40(2), 103–132. http://www.jstor.org/stable/43945159https://www.jstor.org/stable/43945159Endresen, A., Janda, L. A., Reynolds, R., & Tyers, F. M. (2016). Who needs particles? A challenge to the classification of particles as a part of speech in Russian / Кому нужны частицы? Стоит ли определять частицы как отдельную часть речи в русском языке? Russian Linguistics, 40(2), 103–132. http://www.jstor.org/stable/43945159DATABASE particles.csvThis is a spreadsheet of our database, which was used in both Experiment 1 and Experiment 2.text/plain; charset=UTF-8experiment.tar.gzThis file contains all the data and code needed to run Experiment 1 and Experiment 2.application/x-gzipReadme file for DATABASE particles.txtThis file describes the columns and values in those columns for the DATABASE particles.cvs file.text/plain; charset=UTF-8Readme file for experiment.txtThis file explains the contents of the experiment.tar.gz file.text/plain; charset=UTF-8