Biber et al.'s (2016) set of 150 BNC items for the analysis of dispersion measures: Dataset for "Evaluation of text-level measures of lexical dispersion" (doi:10.18710/ATCQZW)

View:

Part 1: Document Description
Part 2: Study Description
Part 5: Other Study-Related Materials
Entire Codebook

(external link) (external link) (external link) (external link)

Document Description

Citation

Title:

Biber et al.'s (2016) set of 150 BNC items for the analysis of dispersion measures: Dataset for "Evaluation of text-level measures of lexical dispersion"

Identification Number:

doi:10.18710/ATCQZW

Distributor:

DataverseNO

Date of Distribution:

2025-01-07

Version:

1

Bibliographic Citation:

Sönning, Lukas, 2025, "Biber et al.'s (2016) set of 150 BNC items for the analysis of dispersion measures: Dataset for "Evaluation of text-level measures of lexical dispersion"", https://doi.org/10.18710/ATCQZW, DataverseNO, V1

Study Description

Citation

Title:

Biber et al.'s (2016) set of 150 BNC items for the analysis of dispersion measures: Dataset for "Evaluation of text-level measures of lexical dispersion"

Identification Number:

doi:10.18710/ATCQZW

Authoring Entity:

Sönning, Lukas (University of Bamberg)

Producer:

University of Bamberg

Software used in Production:

rcqp (R package)

Distributor:

DataverseNO

Distributor:

The Tromsø Repository of Language and Linguistics (TROLLing)

Access Authority:

Sönning, Lukas

Depositor:

Sönning, Lukas

Date of Deposit:

2022-07-08

Holdings Information:

https://doi.org/10.18710/ATCQZW

Study Scope

Keywords:

Arts and Humanities, BNC, dispersion measures, lexical dispersion, word importance, vocabulary lists, word frequency lists, corpus, corpus linguistics, corpus design, methodology, frequency, text-level analysis, Juilland's D, Gries' DP, DA, English

Abstract:

This dataset contains frequencies for a set of 150 word forms in the BNC. The set of items was compiled by Biber et al. (2016) for the purpose of analyzing the behavior of dispersion measures in different distributional settings. It was therefore assembled to cover a broad range of frequency and dispersion levels. For each form, the dataset lists (i) the number occurrences in each of the 4049 text files in the BNC, including zero counts; and (ii) the length of each text file, i.e. the number of word tokens it contains.

Time Period:

1975-1993

Date of Collection:

1991-1993

Country:

United Kingdom

Kind of Data:

corpus data

Kind of Data:

observational data

Methodology and Processing

Sources Statement

Data Sources:

The British National Corpus, version 3 (BNC XML Edition). 2007. Distributed by Bodleian Libraries, University of Oxford, on behalf of the BNC Consortium. URL: <a href="http://www.natcorp.ox.ac.uk/" title="BNC" target="_blank">http://www.natcorp.ox.ac.uk/</a>.

Data Access

Notes:

<p>This Dataset, Biber et al.'s (2016) set of 150 BNC items for the analysis of dispersion measures: Dataset for "Evaluation of text-level measures of lexical dispersion" (hereafter: Dataset), may be reused according to the <strong>CLARIN PUB+BY+LRT</strong> license as described below:</p> <p></p> <p><strong>Copyright holder: </strong>Lukas Sönning</p> <p></p> <p><strong>Resource:</strong> Biber et al.'s (2016) set of 150 BNC items for the analysis of dispersion measures: Dataset for "Evaluation of text-level measures of lexical dispersion"</p> <p></p> <p>The Copyright holders grant the End-User a free, non-exclusive and perpetual (for the duration of the copyright) right to use and make copies of the Resource, distribute copies and present the Resource in public as such, as modified, or as part of a compilation or derived work. The permission applies to all known or future modes and means of communication and includes a right to make modifications enabling the use of the Resource on other devices and in other formats.</p> <p></p> <p>Additional license terms as defined in the <em>Condition Definitions</em> section below:<br> <ul> <li><strong>General Use conditions:</strong> BY, LRT</li> </ul> <p></p> <p><strong>Condition Definitions:</strong><br> <ul> <li><strong>BY:</strong> Attribution, i.e. acknowledgement of authorship, is required.</li> <li><strong>LRT:</strong> The content is available only for language research and technology development.</li> </ul></p> <p></p> <p>This license has been made in compliance with copyright agreements by WIPO – the World Intellectual Property Organization. The rights granted in this license shall be so interpreted that in case applicable intellectual property laws grant rights not mentioned in this license, they are also regarded as part of the rights to be licensed; the purpose of this license is not to restrict any rights intended to be licensed within different legal systems. Additional rights to the Resource may be agreed separately in writing.</p> <p></p> <p><strong>Reused sources:</strong></p> <p></p> <p>This Dataset contains analyses of data, but no actual text excerpts, from the following source:<br> <ul> <li><strong>BNC:</strong> British National Corpus (XML edition; <a href="http://www.natcorp.ox.ac.uk/" title="BNC" target="_blank">http://www.natcorp.ox.ac.uk/</a>). The BNC User Licence (cf. <a href="http://www.natcorp.ox.ac.uk/docs/licence.html" title="BNC User Licence" target="_blank">http://www.natcorp.ox.ac.uk/docs/licence.html</a>) states among other things that the use of BNC is only allowed "for the purposes of linguistic research and/or the development of language products".</li> </ul> </p>

Other Study Description Materials

Related Publications

Citation

Title:

Sönning, L. (2022). Evaluation of text-level measures of lexical dispersion: Robustness and consistency. PsyArXiv. https://doi.org/10.31234/osf.io/h9mvs

Identification Number:

10.31234/osf.io/h9mvs

Bibliographic Citation:

Sönning, L. (2022). Evaluation of text-level measures of lexical dispersion: Robustness and consistency. PsyArXiv. https://doi.org/10.31234/osf.io/h9mvs

Citation

Title:

Biber, Douglas, Randi Reppen, Erin Schnur & Romy Ghanem. 2016. On the (non)utility of Juilland’s D to measure lexical dispersion in large corpora. International Journal of Corpus Linguistics 21(4), 439–464.

Identification Number:

10.1075/ijcl.21.4.01bib

Bibliographic Citation:

Biber, Douglas, Randi Reppen, Erin Schnur & Romy Ghanem. 2016. On the (non)utility of Juilland’s D to measure lexical dispersion in large corpora. International Journal of Corpus Linguistics 21(4), 439–464.

Citation

Title:

Burch, Brent, Jesse Egbert & Douglas Biber. 2017. Measuring and interpreting lexical dispersion in corpus linguistics. Journal of Research Design and Statistics in Linguistics and Communication Science 3(2), 189–216.

Identification Number:

10.1558/jrds.33066

Bibliographic Citation:

Burch, Brent, Jesse Egbert & Douglas Biber. 2017. Measuring and interpreting lexical dispersion in corpus linguistics. Journal of Research Design and Statistics in Linguistics and Communication Science 3(2), 189–216.

Citation

Title:

Egbert, Jesse, Brent Burch & Douglas Biber. 2020. Lexical dispersion and corpus design. International Journal of Corpus Linguistics 25(1), 89–115.

Identification Number:

10.1075/ijcl.18010.egb

Bibliographic Citation:

Egbert, Jesse, Brent Burch & Douglas Biber. 2020. Lexical dispersion and corpus design. International Journal of Corpus Linguistics 25(1), 89–115.

Other Study-Related Materials

Label:

00_ReadMe_BNC150.txt

Text:

Documentation file describing the dataset

Notes:

text/plain

Other Study-Related Materials

Label:

01_R_script_BNC150_extraction.Rmd

Text:

R script documenting data retrieval from the BNC

Notes:

application/octet-stream

Other Study-Related Materials

Label:

able.csv

Notes:

text/csv

Other Study-Related Materials

Label:

a.csv

Notes:

text/csv

Other Study-Related Materials

Label:

actually.csv

Notes:

text/csv

Other Study-Related Materials

Label:

after.csv

Notes:

text/csv

Other Study-Related Materials

Label:

against.csv

Notes:

text/csv

Other Study-Related Materials

Label:

aha.csv

Notes:

text/csv

Other Study-Related Materials

Label:

ah.csv

Notes:

text/csv

Other Study-Related Materials

Label:

all.csv

Notes:

text/csv

Other Study-Related Materials

Label:

among.csv

Notes:

text/csv

Other Study-Related Materials

Label:

an.csv

Notes:

text/csv

Other Study-Related Materials

Label:

and.csv

Notes:

text/csv

Other Study-Related Materials

Label:

another.csv

Notes:

text/csv

Other Study-Related Materials

Label:

anybody.csv

Notes:

text/csv

Other Study-Related Materials

Label:

at.csv

Notes:

text/csv

Other Study-Related Materials

Label:

aye.csv

Notes:

text/csv

Other Study-Related Materials

Label:

became.csv

Notes:

text/csv

Other Study-Related Materials

Label:

be.csv

Notes:

text/csv

Other Study-Related Materials

Label:

been.csv

Notes:

text/csv

Other Study-Related Materials

Label:

began.csv

Notes:

text/csv

Other Study-Related Materials

Label:

bet.csv

Notes:

text/csv

Other Study-Related Materials

Label:

between.csv

Notes:

text/csv

Other Study-Related Materials

Label:

bloke.csv

Notes:

text/csv

Other Study-Related Materials

Label:

both.csv

Notes:

text/csv

Other Study-Related Materials

Label:

bringing.csv

Notes:

text/csv

Other Study-Related Materials

Label:

brought.csv

Notes:

text/csv

Other Study-Related Materials

Label:

but.csv

Notes:

text/csv

Other Study-Related Materials

Label:

charles.csv

Notes:

text/csv

Other Study-Related Materials

Label:

claimed.csv

Notes:

text/csv

Other Study-Related Materials

Label:

cor.csv

Notes:

text/csv

Other Study-Related Materials

Label:

corp.csv

Notes:

text/csv

Other Study-Related Materials

Label:

cos.csv

Notes:

text/csv

Other Study-Related Materials

Label:

da.csv

Notes:

text/csv

Other Study-Related Materials

Label:

day.csv

Notes:

text/csv

Other Study-Related Materials

Label:

decided.csv

Notes:

text/csv

Other Study-Related Materials

Label:

did.csv

Notes:

text/csv

Other Study-Related Materials

Label:

do.csv

Notes:

text/csv

Other Study-Related Materials

Label:

doo.csv

Notes:

text/csv

Other Study-Related Materials

Label:

during.csv

Notes:

text/csv

Other Study-Related Materials

Label:

each.csv

Notes:

text/csv

Other Study-Related Materials

Label:

economic.csv

Notes:

text/csv

Other Study-Related Materials

Label:

eh.csv

Notes:

text/csv

Other Study-Related Materials

Label:

eighty.csv

Notes:

text/csv

Other Study-Related Materials

Label:

england.csv

Notes:

text/csv

Other Study-Related Materials

Label:

er.csv

Notes:

text/csv

Other Study-Related Materials

Label:

erm.csv

Notes:

text/csv

Other Study-Related Materials

Label:

etcetera.csv

Notes:

text/csv

Other Study-Related Materials

Label:

everybody.csv

Notes:

text/csv

Other Study-Related Materials

Label:

fall.csv

Notes:

text/csv

Other Study-Related Materials

Label:

fig.csv

Notes:

text/csv

Other Study-Related Materials

Label:

for.csv

Notes:

text/csv

Other Study-Related Materials

Label:

forty.csv

Notes:

text/csv

Other Study-Related Materials

Label:

found.csv

Notes:

text/csv

Other Study-Related Materials

Label:

from.csv

Notes:

text/csv

Other Study-Related Materials

Label:

full.csv

Notes:

text/csv

Other Study-Related Materials

Label:

get.csv

Notes:

text/csv

Other Study-Related Materials

Label:

government.csv

Notes:

text/csv

Other Study-Related Materials

Label:

ha.csv

Notes:

text/csv

Other Study-Related Materials

Label:

had.csv

Notes:

text/csv

Other Study-Related Materials

Label:

has.csv

Notes:

text/csv

Other Study-Related Materials

Label:

have.csv

Notes:

text/csv

Other Study-Related Materials

Label:

having.csv

Notes:

text/csv

Other Study-Related Materials

Label:

held.csv

Notes:

text/csv

Other Study-Related Materials

Label:

hello.csv

Notes:

text/csv

Other Study-Related Materials

Label:

himself.csv

Notes:

text/csv

Other Study-Related Materials

Label:

hm.csv

Notes:

text/csv

Other Study-Related Materials

Label:

however.csv

Notes:

text/csv

Other Study-Related Materials

Label:

hundred.csv

Notes:

text/csv

Other Study-Related Materials

Label:

ibm.csv

Notes:

text/csv

Other Study-Related Materials

Label:

i.csv

Notes:

text/csv

Other Study-Related Materials

Label:

if.csv

Notes:

text/csv

Other Study-Related Materials

Label:

important.csv

Notes:

text/csv

Other Study-Related Materials

Label:

inc.csv

Notes:

text/csv

Other Study-Related Materials

Label:

including.csv

Notes:

text/csv

Other Study-Related Materials

Label:

in.csv

Notes:

text/csv

Other Study-Related Materials

Label:

international.csv

Notes:

text/csv

Other Study-Related Materials

Label:

into.csv

Notes:

text/csv

Other Study-Related Materials

Label:

it.csv

Notes:

text/csv

Other Study-Related Materials

Label:

just.csv

Notes:

text/csv

Other Study-Related Materials

Label:

know.csv

Notes:

text/csv

Other Study-Related Materials

Label:

large.csv

Notes:

text/csv

Other Study-Related Materials

Label:

later.csv

Notes:

text/csv

Other Study-Related Materials

Label:

latter.csv

Notes:

text/csv

Other Study-Related Materials

Label:

let's.csv

Notes:

text/csv

Other Study-Related Materials

Label:

life.csv

Notes:

text/csv

Other Study-Related Materials

Label:

ltd.csv

Notes:

text/csv

Other Study-Related Materials

Label:

made.csv

Notes:

text/csv

Other Study-Related Materials

Label:

may.csv

Notes:

text/csv

Other Study-Related Materials

Label:

methods.csv

Notes:

text/csv

Other Study-Related Materials

Label:

mhm.csv

Notes:

text/csv

Other Study-Related Materials

Label:

minus.csv

Notes:

text/csv

Other Study-Related Materials

Label:

mm.csv

Notes:

text/csv

Other Study-Related Materials

Label:

most.csv

Notes:

text/csv

Other Study-Related Materials

Label:

mr.csv

Notes:

text/csv

Other Study-Related Materials

Label:

mum.csv

Notes:

text/csv

Other Study-Related Materials

Label:

new.csv

Notes:

text/csv

Other Study-Related Materials

Label:

nineteen.csv

Notes:

text/csv

Other Study-Related Materials

Label:

ninety.csv

Notes:

text/csv

Other Study-Related Materials

Label:

nodded.csv

Notes:

text/csv

Other Study-Related Materials

Label:

nought.csv

Notes:

text/csv

Other Study-Related Materials

Label:

oh.csv

Notes:

text/csv

Other Study-Related Materials

Label:

okay.csv

Notes:

text/csv

Other Study-Related Materials

Label:

on.csv

Notes:

text/csv

Other Study-Related Materials

Label:

ooh.csv

Notes:

text/csv

Other Study-Related Materials

Label:

out.csv

Notes:

text/csv

Other Study-Related Materials

Label:

pence.csv

Notes:

text/csv

Other Study-Related Materials

Label:

percent.csv

Notes:

text/csv

Other Study-Related Materials

Label:

political.csv

Notes:

text/csv

Other Study-Related Materials

Label:

presence.csv

Notes:

text/csv

Other Study-Related Materials

Label:

provides.csv

Notes:

text/csv

Other Study-Related Materials

Label:

put.csv

Notes:

text/csv

Other Study-Related Materials

Label:

really.csv

Notes:

text/csv

Other Study-Related Materials

Label:

reckon.csv

Notes:

text/csv

Other Study-Related Materials

Label:

say.csv

Notes:

text/csv

Other Study-Related Materials

Label:

seemed.csv

Notes:

text/csv

Other Study-Related Materials

Label:

seriously.csv

Notes:

text/csv

Other Study-Related Materials

Label:

sixty.csv

Notes:

text/csv

Other Study-Related Materials

Label:

smiled.csv

Notes:

text/csv

Other Study-Related Materials

Label:

social.csv

Notes:

text/csv

Other Study-Related Materials

Label:

so.csv

Notes:

text/csv

Other Study-Related Materials

Label:

somebody.csv

Notes:

text/csv

Other Study-Related Materials

Label:

system.csv

Notes:

text/csv

Other Study-Related Materials

Label:

take.csv

Notes:

text/csv

Other Study-Related Materials

Label:

talking.csv

Notes:

text/csv

Other Study-Related Materials

Label:

than.csv

Notes:

text/csv

Other Study-Related Materials

Label:

the.csv

Notes:

text/csv

Other Study-Related Materials

Label:

they.csv

Notes:

text/csv

Other Study-Related Materials

Label:

thing.csv

Notes:

text/csv

Other Study-Related Materials

Label:

think.csv

Notes:

text/csv

Other Study-Related Materials

Label:

thirteen.csv

Notes:

text/csv

Other Study-Related Materials

Label:

though.csv

Notes:

text/csv

Other Study-Related Materials

Label:

thus.csv

Notes:

text/csv

Other Study-Related Materials

Label:

time.csv

Notes:

text/csv

Other Study-Related Materials

Label:

tt.csv

Notes:

text/csv

Other Study-Related Materials

Label:

tv.csv

Notes:

text/csv

Other Study-Related Materials

Label:

twenty.csv

Notes:

text/csv

Other Study-Related Materials

Label:

uk.csv

Notes:

text/csv

Other Study-Related Materials

Label:

under.csv

Notes:

text/csv

Other Study-Related Materials

Label:

urgh.csv

Notes:

text/csv

Other Study-Related Materials

Label:

usa.csv

Notes:

text/csv

Other Study-Related Materials

Label:

us.csv

Notes:

text/csv

Other Study-Related Materials

Label:

wants.csv

Notes:

text/csv

Other Study-Related Materials

Label:

was.csv

Notes:

text/csv

Other Study-Related Materials

Label:

we.csv

Notes:

text/csv

Other Study-Related Materials

Label:

who.csv

Notes:

text/csv

Other Study-Related Materials

Label:

with.csv

Notes:

text/csv

Other Study-Related Materials

Label:

world.csv

Notes:

text/csv

Other Study-Related Materials

Label:

yeah.csv

Notes:

text/csv

Other Study-Related Materials

Label:

yes.csv

Notes:

text/csv

Other Study-Related Materials

Label:

you.csv

Notes:

text/csv

Other Study-Related Materials

Label:

your.csv

Notes:

text/csv