|
View: |
Part 1: Document Description
|
|
Citation |
|
|---|---|
|
Title: |
Biber et al.'s (2016) set of 150 BNC items for the analysis of dispersion measures: Dataset for "Evaluation of text-level measures of lexical dispersion" |
|
Identification Number: |
doi:10.18710/ATCQZW |
|
Distributor: |
DataverseNO |
|
Date of Distribution: |
2025-01-07 |
|
Version: |
1 |
|
Bibliographic Citation: |
Sönning, Lukas, 2025, "Biber et al.'s (2016) set of 150 BNC items for the analysis of dispersion measures: Dataset for "Evaluation of text-level measures of lexical dispersion"", https://doi.org/10.18710/ATCQZW, DataverseNO, V1 |
|
Citation |
|
|
Title: |
Biber et al.'s (2016) set of 150 BNC items for the analysis of dispersion measures: Dataset for "Evaluation of text-level measures of lexical dispersion" |
|
Identification Number: |
doi:10.18710/ATCQZW |
|
Authoring Entity: |
Sönning, Lukas (University of Bamberg) |
|
Producer: |
University of Bamberg |
|
Software used in Production: |
rcqp (R package) |
|
Distributor: |
DataverseNO |
|
Distributor: |
The Tromsø Repository of Language and Linguistics (TROLLing) |
|
Access Authority: |
Sönning, Lukas |
|
Depositor: |
Sönning, Lukas |
|
Date of Deposit: |
2022-07-08 |
|
Holdings Information: |
https://doi.org/10.18710/ATCQZW |
|
Study Scope |
|
|
Keywords: |
Arts and Humanities, BNC, dispersion measures, lexical dispersion, word importance, vocabulary lists, word frequency lists, corpus, corpus linguistics, corpus design, methodology, frequency, text-level analysis, Juilland's D, Gries' DP, DA, English |
|
Abstract: |
This dataset contains frequencies for a set of 150 word forms in the BNC. The set of items was compiled by Biber et al. (2016) for the purpose of analyzing the behavior of dispersion measures in different distributional settings. It was therefore assembled to cover a broad range of frequency and dispersion levels. For each form, the dataset lists (i) the number occurrences in each of the 4049 text files in the BNC, including zero counts; and (ii) the length of each text file, i.e. the number of word tokens it contains. |
|
Time Period: |
1975-1993 |
|
Date of Collection: |
1991-1993 |
|
Country: |
United Kingdom |
|
Kind of Data: |
corpus data |
|
Kind of Data: |
observational data |
|
Methodology and Processing |
|
|
Sources Statement |
|
|
Data Sources: |
The British National Corpus, version 3 (BNC XML Edition). 2007. Distributed by Bodleian Libraries, University of Oxford, on behalf of the BNC Consortium. URL: <a href="http://www.natcorp.ox.ac.uk/" title="BNC" target="_blank">http://www.natcorp.ox.ac.uk/</a>. |
|
Data Access |
|
|
Notes: |
<p>This Dataset, Biber et al.'s (2016) set of 150 BNC items for the analysis of dispersion measures: Dataset for "Evaluation of text-level measures of lexical dispersion" (hereafter: Dataset), may be reused according to the <strong>CLARIN PUB+BY+LRT</strong> license as described below:</p> <p></p> <p><strong>Copyright holder: </strong>Lukas Sönning</p> <p></p> <p><strong>Resource:</strong> Biber et al.'s (2016) set of 150 BNC items for the analysis of dispersion measures: Dataset for "Evaluation of text-level measures of lexical dispersion"</p> <p></p> <p>The Copyright holders grant the End-User a free, non-exclusive and perpetual (for the duration of the copyright) right to use and make copies of the Resource, distribute copies and present the Resource in public as such, as modified, or as part of a compilation or derived work. The permission applies to all known or future modes and means of communication and includes a right to make modifications enabling the use of the Resource on other devices and in other formats.</p> <p></p> <p>Additional license terms as defined in the <em>Condition Definitions</em> section below:<br> <ul> <li><strong>General Use conditions:</strong> BY, LRT</li> </ul> <p></p> <p><strong>Condition Definitions:</strong><br> <ul> <li><strong>BY:</strong> Attribution, i.e. acknowledgement of authorship, is required.</li> <li><strong>LRT:</strong> The content is available only for language research and technology development.</li> </ul></p> <p></p> <p>This license has been made in compliance with copyright agreements by WIPO – the World Intellectual Property Organization. The rights granted in this license shall be so interpreted that in case applicable intellectual property laws grant rights not mentioned in this license, they are also regarded as part of the rights to be licensed; the purpose of this license is not to restrict any rights intended to be licensed within different legal systems. Additional rights to the Resource may be agreed separately in writing.</p> <p></p> <p><strong>Reused sources:</strong></p> <p></p> <p>This Dataset contains analyses of data, but no actual text excerpts, from the following source:<br> <ul> <li><strong>BNC:</strong> British National Corpus (XML edition; <a href="http://www.natcorp.ox.ac.uk/" title="BNC" target="_blank">http://www.natcorp.ox.ac.uk/</a>). The BNC User Licence (cf. <a href="http://www.natcorp.ox.ac.uk/docs/licence.html" title="BNC User Licence" target="_blank">http://www.natcorp.ox.ac.uk/docs/licence.html</a>) states among other things that the use of BNC is only allowed "for the purposes of linguistic research and/or the development of language products".</li> </ul> </p> |
|
Other Study Description Materials |
|
|
Related Publications |
|
|
Citation |
|
|
Title: |
Sönning, L. (2022). Evaluation of text-level measures of lexical dispersion: Robustness and consistency. PsyArXiv. https://doi.org/10.31234/osf.io/h9mvs |
|
Identification Number: |
10.31234/osf.io/h9mvs |
|
Bibliographic Citation: |
Sönning, L. (2022). Evaluation of text-level measures of lexical dispersion: Robustness and consistency. PsyArXiv. https://doi.org/10.31234/osf.io/h9mvs |
|
Citation |
|
|
Title: |
Biber, Douglas, Randi Reppen, Erin Schnur & Romy Ghanem. 2016. On the (non)utility of Juilland’s D to measure lexical dispersion in large corpora. International Journal of Corpus Linguistics 21(4), 439–464. |
|
Identification Number: |
10.1075/ijcl.21.4.01bib |
|
Bibliographic Citation: |
Biber, Douglas, Randi Reppen, Erin Schnur & Romy Ghanem. 2016. On the (non)utility of Juilland’s D to measure lexical dispersion in large corpora. International Journal of Corpus Linguistics 21(4), 439–464. |
|
Citation |
|
|
Title: |
Burch, Brent, Jesse Egbert & Douglas Biber. 2017. Measuring and interpreting lexical dispersion in corpus linguistics. Journal of Research Design and Statistics in Linguistics and Communication Science 3(2), 189–216. |
|
Identification Number: |
10.1558/jrds.33066 |
|
Bibliographic Citation: |
Burch, Brent, Jesse Egbert & Douglas Biber. 2017. Measuring and interpreting lexical dispersion in corpus linguistics. Journal of Research Design and Statistics in Linguistics and Communication Science 3(2), 189–216. |
|
Citation |
|
|
Title: |
Egbert, Jesse, Brent Burch & Douglas Biber. 2020. Lexical dispersion and corpus design. International Journal of Corpus Linguistics 25(1), 89–115. |
|
Identification Number: |
10.1075/ijcl.18010.egb |
|
Bibliographic Citation: |
Egbert, Jesse, Brent Burch & Douglas Biber. 2020. Lexical dispersion and corpus design. International Journal of Corpus Linguistics 25(1), 89–115. |
|
Label: |
00_ReadMe_BNC150.txt |
|
Text: |
Documentation file describing the dataset |
|
Notes: |
text/plain |
|
Label: |
01_R_script_BNC150_extraction.Rmd |
|
Text: |
R script documenting data retrieval from the BNC |
|
Notes: |
application/octet-stream |
|
Label: |
able.csv |
|
Notes: |
text/csv |
|
Label: |
a.csv |
|
Notes: |
text/csv |
|
Label: |
actually.csv |
|
Notes: |
text/csv |
|
Label: |
after.csv |
|
Notes: |
text/csv |
|
Label: |
against.csv |
|
Notes: |
text/csv |
|
Label: |
aha.csv |
|
Notes: |
text/csv |
|
Label: |
ah.csv |
|
Notes: |
text/csv |
|
Label: |
all.csv |
|
Notes: |
text/csv |
|
Label: |
among.csv |
|
Notes: |
text/csv |
|
Label: |
an.csv |
|
Notes: |
text/csv |
|
Label: |
and.csv |
|
Notes: |
text/csv |
|
Label: |
another.csv |
|
Notes: |
text/csv |
|
Label: |
anybody.csv |
|
Notes: |
text/csv |
|
Label: |
at.csv |
|
Notes: |
text/csv |
|
Label: |
aye.csv |
|
Notes: |
text/csv |
|
Label: |
became.csv |
|
Notes: |
text/csv |
|
Label: |
be.csv |
|
Notes: |
text/csv |
|
Label: |
been.csv |
|
Notes: |
text/csv |
|
Label: |
began.csv |
|
Notes: |
text/csv |
|
Label: |
bet.csv |
|
Notes: |
text/csv |
|
Label: |
between.csv |
|
Notes: |
text/csv |
|
Label: |
bloke.csv |
|
Notes: |
text/csv |
|
Label: |
both.csv |
|
Notes: |
text/csv |
|
Label: |
bringing.csv |
|
Notes: |
text/csv |
|
Label: |
brought.csv |
|
Notes: |
text/csv |
|
Label: |
but.csv |
|
Notes: |
text/csv |
|
Label: |
charles.csv |
|
Notes: |
text/csv |
|
Label: |
claimed.csv |
|
Notes: |
text/csv |
|
Label: |
cor.csv |
|
Notes: |
text/csv |
|
Label: |
corp.csv |
|
Notes: |
text/csv |
|
Label: |
cos.csv |
|
Notes: |
text/csv |
|
Label: |
da.csv |
|
Notes: |
text/csv |
|
Label: |
day.csv |
|
Notes: |
text/csv |
|
Label: |
decided.csv |
|
Notes: |
text/csv |
|
Label: |
did.csv |
|
Notes: |
text/csv |
|
Label: |
do.csv |
|
Notes: |
text/csv |
|
Label: |
doo.csv |
|
Notes: |
text/csv |
|
Label: |
during.csv |
|
Notes: |
text/csv |
|
Label: |
each.csv |
|
Notes: |
text/csv |
|
Label: |
economic.csv |
|
Notes: |
text/csv |
|
Label: |
eh.csv |
|
Notes: |
text/csv |
|
Label: |
eighty.csv |
|
Notes: |
text/csv |
|
Label: |
england.csv |
|
Notes: |
text/csv |
|
Label: |
er.csv |
|
Notes: |
text/csv |
|
Label: |
erm.csv |
|
Notes: |
text/csv |
|
Label: |
etcetera.csv |
|
Notes: |
text/csv |
|
Label: |
everybody.csv |
|
Notes: |
text/csv |
|
Label: |
fall.csv |
|
Notes: |
text/csv |
|
Label: |
fig.csv |
|
Notes: |
text/csv |
|
Label: |
for.csv |
|
Notes: |
text/csv |
|
Label: |
forty.csv |
|
Notes: |
text/csv |
|
Label: |
found.csv |
|
Notes: |
text/csv |
|
Label: |
from.csv |
|
Notes: |
text/csv |
|
Label: |
full.csv |
|
Notes: |
text/csv |
|
Label: |
get.csv |
|
Notes: |
text/csv |
|
Label: |
government.csv |
|
Notes: |
text/csv |
|
Label: |
ha.csv |
|
Notes: |
text/csv |
|
Label: |
had.csv |
|
Notes: |
text/csv |
|
Label: |
has.csv |
|
Notes: |
text/csv |
|
Label: |
have.csv |
|
Notes: |
text/csv |
|
Label: |
having.csv |
|
Notes: |
text/csv |
|
Label: |
held.csv |
|
Notes: |
text/csv |
|
Label: |
hello.csv |
|
Notes: |
text/csv |
|
Label: |
himself.csv |
|
Notes: |
text/csv |
|
Label: |
hm.csv |
|
Notes: |
text/csv |
|
Label: |
however.csv |
|
Notes: |
text/csv |
|
Label: |
hundred.csv |
|
Notes: |
text/csv |
|
Label: |
ibm.csv |
|
Notes: |
text/csv |
|
Label: |
i.csv |
|
Notes: |
text/csv |
|
Label: |
if.csv |
|
Notes: |
text/csv |
|
Label: |
important.csv |
|
Notes: |
text/csv |
|
Label: |
inc.csv |
|
Notes: |
text/csv |
|
Label: |
including.csv |
|
Notes: |
text/csv |
|
Label: |
in.csv |
|
Notes: |
text/csv |
|
Label: |
international.csv |
|
Notes: |
text/csv |
|
Label: |
into.csv |
|
Notes: |
text/csv |
|
Label: |
it.csv |
|
Notes: |
text/csv |
|
Label: |
just.csv |
|
Notes: |
text/csv |
|
Label: |
know.csv |
|
Notes: |
text/csv |
|
Label: |
large.csv |
|
Notes: |
text/csv |
|
Label: |
later.csv |
|
Notes: |
text/csv |
|
Label: |
latter.csv |
|
Notes: |
text/csv |
|
Label: |
let's.csv |
|
Notes: |
text/csv |
|
Label: |
life.csv |
|
Notes: |
text/csv |
|
Label: |
ltd.csv |
|
Notes: |
text/csv |
|
Label: |
made.csv |
|
Notes: |
text/csv |
|
Label: |
may.csv |
|
Notes: |
text/csv |
|
Label: |
methods.csv |
|
Notes: |
text/csv |
|
Label: |
mhm.csv |
|
Notes: |
text/csv |
|
Label: |
minus.csv |
|
Notes: |
text/csv |
|
Label: |
mm.csv |
|
Notes: |
text/csv |
|
Label: |
most.csv |
|
Notes: |
text/csv |
|
Label: |
mr.csv |
|
Notes: |
text/csv |
|
Label: |
mum.csv |
|
Notes: |
text/csv |
|
Label: |
new.csv |
|
Notes: |
text/csv |
|
Label: |
nineteen.csv |
|
Notes: |
text/csv |
|
Label: |
ninety.csv |
|
Notes: |
text/csv |
|
Label: |
nodded.csv |
|
Notes: |
text/csv |
|
Label: |
nought.csv |
|
Notes: |
text/csv |
|
Label: |
oh.csv |
|
Notes: |
text/csv |
|
Label: |
okay.csv |
|
Notes: |
text/csv |
|
Label: |
on.csv |
|
Notes: |
text/csv |
|
Label: |
ooh.csv |
|
Notes: |
text/csv |
|
Label: |
out.csv |
|
Notes: |
text/csv |
|
Label: |
pence.csv |
|
Notes: |
text/csv |
|
Label: |
percent.csv |
|
Notes: |
text/csv |
|
Label: |
political.csv |
|
Notes: |
text/csv |
|
Label: |
presence.csv |
|
Notes: |
text/csv |
|
Label: |
provides.csv |
|
Notes: |
text/csv |
|
Label: |
put.csv |
|
Notes: |
text/csv |
|
Label: |
really.csv |
|
Notes: |
text/csv |
|
Label: |
reckon.csv |
|
Notes: |
text/csv |
|
Label: |
say.csv |
|
Notes: |
text/csv |
|
Label: |
seemed.csv |
|
Notes: |
text/csv |
|
Label: |
seriously.csv |
|
Notes: |
text/csv |
|
Label: |
sixty.csv |
|
Notes: |
text/csv |
|
Label: |
smiled.csv |
|
Notes: |
text/csv |
|
Label: |
social.csv |
|
Notes: |
text/csv |
|
Label: |
so.csv |
|
Notes: |
text/csv |
|
Label: |
somebody.csv |
|
Notes: |
text/csv |
|
Label: |
system.csv |
|
Notes: |
text/csv |
|
Label: |
take.csv |
|
Notes: |
text/csv |
|
Label: |
talking.csv |
|
Notes: |
text/csv |
|
Label: |
than.csv |
|
Notes: |
text/csv |
|
Label: |
the.csv |
|
Notes: |
text/csv |
|
Label: |
they.csv |
|
Notes: |
text/csv |
|
Label: |
thing.csv |
|
Notes: |
text/csv |
|
Label: |
think.csv |
|
Notes: |
text/csv |
|
Label: |
thirteen.csv |
|
Notes: |
text/csv |
|
Label: |
though.csv |
|
Notes: |
text/csv |
|
Label: |
thus.csv |
|
Notes: |
text/csv |
|
Label: |
time.csv |
|
Notes: |
text/csv |
|
Label: |
tt.csv |
|
Notes: |
text/csv |
|
Label: |
tv.csv |
|
Notes: |
text/csv |
|
Label: |
twenty.csv |
|
Notes: |
text/csv |
|
Label: |
uk.csv |
|
Notes: |
text/csv |
|
Label: |
under.csv |
|
Notes: |
text/csv |
|
Label: |
urgh.csv |
|
Notes: |
text/csv |
|
Label: |
usa.csv |
|
Notes: |
text/csv |
|
Label: |
us.csv |
|
Notes: |
text/csv |
|
Label: |
wants.csv |
|
Notes: |
text/csv |
|
Label: |
was.csv |
|
Notes: |
text/csv |
|
Label: |
we.csv |
|
Notes: |
text/csv |
|
Label: |
who.csv |
|
Notes: |
text/csv |
|
Label: |
with.csv |
|
Notes: |
text/csv |
|
Label: |
world.csv |
|
Notes: |
text/csv |
|
Label: |
yeah.csv |
|
Notes: |
text/csv |
|
Label: |
yes.csv |
|
Notes: |
text/csv |
|
Label: |
you.csv |
|
Notes: |
text/csv |
|
Label: |
your.csv |
|
Notes: |
text/csv |