A Dataset Documenting Representations of Machine Vision Technologies in Artworks, Games and Narratives (doi:10.18710/2G0XKN)

View:

Part 1: Document Description
Part 2: Study Description
Part 5: Other Study-Related Materials
Entire Codebook

A Dataset Documenting Representations of Machine Vision Technologies in Artworks, Games and Narratives

doi:10.18710/2G0XKN

DataverseNO

2022-03-22

2

Rettberg, Jill Walker; Kronman, Linda; Solberg, Ragnhild; Gunderson, Marianne; Bjørklund, Stein Magne; Stokkedal, Linn Heidi; de Seta, Gabriele; Jacob, Kurdin; Markham, Annette, 2022, "A Dataset Documenting Representations of Machine Vision Technologies in Artworks, Games and Narratives", https://doi.org/10.18710/2G0XKN, DataverseNO, V2

A Dataset Documenting Representations of Machine Vision Technologies in Artworks, Games and Narratives

doi:10.18710/2G0XKN

Rettberg, Jill Walker (University of Bergen)

Kronman, Linda (University of Bergen)

Solberg, Ragnhild (University of Bergen)

Gunderson, Marianne (University of Bergen)

Bjørklund, Stein Magne (University of Bergen)

Stokkedal, Linn Heidi (University of Bergen)

de Seta, Gabriele (University of Bergen)

Jacob, Kurdin (University of Bergen)

Markham, Annette (Royal Melbourne Institute of Technology)

Rettberg, Jill Walker

Solberg, Ragnhild

Kronman, Linda

Gunderson, Marianne

de Seta, Gabriele

Bjørklund, Stein Magne

Lautenschlaeger, Graziele

Arce, Diana

Svihus, Edward

Waskiewicz, Milosz

Przulj, Tijana

Li, Hang On Martin

Klingenberg, Cecilie Thale

Shahpary, Milad

Hersvik, Amanda

Haugland, Ida Otilde

Sandvik, Sunniva Eirin

Retzius, Ainsley Belle

Karhio, Anne

Ostrop, Jenny

University of Bergen

Drupal

R

RStudio

771800

DataverseNO

University of Bergen

Rettberg, Jill Walker

Rettberg, Jill Walker

2022-03-17

https://doi.org/10.18710/2G0XKN

Arts and Humanities, digital humanities (discipline), digital humanities, digital culture, machine vision, computer vision, algorithmic bias, science fiction, fiction, video game, art, novels, film, motion pictures (visual works), electronic literature, television series, digital art (visual works), new media art, digital art, game studies

<p>This dataset captures cultural attitudes towards machine vision technologies as they are expressed in art, games and narratives. The dataset includes records of 500 creative works (including 77 digital games, 191 digital artworks and 236 movies, novels and other narratives) that use or represent machine vision technologies like facial recognition, deepfakes, and augmented reality. The dataset is divided into three main tables, relating to the works, to specific situations in each work involving machine vision technologies, and to the characters that interact with the technologies. Data about each work includes title, author, year and country of publication; types of machine vision technologies featured; topics the work addresses, and sentiments associated with that machine vision usage in the work. In the various works we identified 884 specific situations where machine vision is central. The dataset includes detailed data about each of these situations that describes the actions of human and non-human agents, including machine vision technologies. The dataset is the product of a digital humanities project and can be also viewed as a database at http://machine-vision.no. </p> <p>Data was collected by a team of topic experts who followed an analytical model developed to explore relationships between humans and technologies, inspired by posthumanist and feminist new materialist theories. The project team identified relevant works by searching databases, visiting exhibitions and conferences, reading scholarship, and consulting other experts. The inclusion criteria were creative works( art, games, narratives (movies, novels, etc)) where one of the following machine vision technologies was used in or represented by the work: 3D scans, AI, Augmented reality, Biometrics, Body scans, Camera, Cameraphone, Deepfake, Drones, Emotion recognition, Facial recognition, Filtering, Holograms, Image generation, Interactive panoramas Machine learning, MicroscopeOrTelescope Motion tracking, Non-Visible Spectrum Object recognition, Ocular implant, Satellite images, Surveillance cameras, UGV, Virtual reality, and Webcams.</p> <p>The dataset as well as the more detailed database can be viewed, searched, extracted, or otherwise used or reused and is considered particularly useful for humanities and social science scholars interested in the relationship between technology and culture, and by designers, artists, and scientists developing machine vision technologies.</p>

1897-01-01-2021-10-30

2019-01-01-2021-10-30

- Creative Works (i.e. novels, movies, video games, artworks) <p> - Situations in the works that involve machine vision technologies<p> - Characters in the works

Textual data

Tabular data

Descriptive data

Interpretative data

R scripts

The project identified relevant works by searching databases, visiting exhibitions and conferences, reading scholarship, and consulting other experts. The inclusion criteria were creative works (art, games, narratives) where one of the following machine vision technologies was used in or represented by the work: 3D scans, AI, Augmented reality, Biometrics, Body scans, Camera, Cameraphone, Deepfake, Drones, Emotion recognition, Facial recognition, Filtering, Holograms, Image generation, Interactive panoramas Machine learning, MicroscopeOrTelescope Motion tracking, Non-Visible Spectrum Object recognition, Ocular implant, Satellite images, Surveillance cameras, UGV, Virtual reality, and Webcams.

Works were entered into a database built in Drupal 9.

<p>The dataset consists of metadata about and structured analysis data about video games, artworks and narratives. A complete list of the primary data sources can be found in CreativeWorks.csv.</p> Narratives include:<p> - Published novels and short stories<p> - Movies and TV series screened at cinemas or film festivals and available on public broadcasting or commercial streaming services<p> - Written narratives, such as fan fiction, creepypasta, short stories and electronic literature published in online journals, websites or public forums<p> - Electronic Literature published online or presented in public exhibitions<p> - Music videos with strong narrative elements<p> Games include:<p> - Video games available for purchase or download in stores or on platforms such as Steam<p> Artworks include:<p> - Artworks publicly displayed in exhibitions or online

The dataset includes data describing 77 games, 192 artworks and 237 narratives (in total 500 Creative Works) where machine vision technologies play an important role. This includes Creative Works produced between 1891 and 2021, but with a heavy emphasis on recent works: 80% of the Works are from 2011-2021, and just over half from 2016-2021. The Creative Works are from 59 different countries, with 78,6% from North America and Europe, and 21,4% from other parts of the world.

The team iteratively went through the data and made corrections.

Database of Machine Vision in Art, Games and Narratives: http://machine-vision.no<p> Wikidata: https://www.wikidata.org/wiki/Wikidata:Main_Page

Rettberg JW, Kronman L, Solberg R, et al. (2022) Representations of Machine Vision Technologies in Artworks, Games and Narratives: Documentation of a Dataset. Data in Brief.

10.1016/j.dib.2022.108319

Rettberg JW, Kronman L, Solberg R, et al. (2022) Representations of Machine Vision Technologies in Artworks, Games and Narratives: Documentation of a Dataset. Data in Brief.

Rettberg JW (2022) Algorithmic failure as a humanities methodology: machine learning’s mispredictions identify rich cases for qualitative analysis. Big Data & Society.

10.1177/20539517221131290

Rettberg JW (2022) Algorithmic failure as a humanities methodology: machine learning’s mispredictions identify rich cases for qualitative analysis. Big Data & Society.

Label:

00_README.txt

Text:

This file contains metadata about the dataset, explains the structure of the dataset and each file, and lists the 500 creative works included in the dataset.

text/plain

Label:

01_codebook.csv

Text:

A table showing all variables used in the different data files with definitions and information about which variables are included in each file.

text/csv

Label:

02_technologies_sentiments_topics_definitions.csv

Text:

A table listing all machine vision technologies, sentiments and topics identified in the works, with the definitions used for this project.

text/comma-separated-values

Label:

characters.csv

Text:

Lists all Characters that interact with machine vision with fields describing how they are represented in the Creative Work. Each character has a separate row and no variables (columns) have multiple values.

text/csv

Label:

creativeworks.csv

Text:

Lists 500 creative works (artworks, video games, noels, movies etc) where machine vision technologies are either represented or used in the work. Variables include the title, overall genre (art, game, narrative), year of publication, country, technologies referenced and used, topics in the work, sentiment shown towards machine vision, and characters and situations in the work. The last two allow this data to be connected to the data about characters and situations. Values are repeated so each row (each combination of values) is unique, so care must be taken during analysis to avoid duplicate values.

text/csv

Label:

creators.csv

Text:

Creators (artists, authors, producers etc) of the creative works with Wikidata IDs if available. Can be joined with worksinfo.csv, which links each work to a creator.

text/csv

Label:

machinevisionscripts.R

Text:

Scripts that can be run in R to import and join files, to transform data into contingency tables, and to transform worksinfo.csv into a wide table that is more human-friendly. Use R or RStudio to run the scripts, or view the file in a text editor.

type/x-r-syntax

Label:

narrativegenres.csv

Text:

Lists narrative creative works with their subcategory, i.e. whether the narrative work is a movie, a novel, a TV series etc. Subgenres are not included in the other datasets, but the WorkID allows the data to be connected. This format of the data has a new row for each new genre or subgenre.

text/csv

Label:

situations.csv

Text:

Lists situations involving machine vision technologies in the Creative Works with details about the actions of humans, technologies, and other agents. Values are repeated so each row (each combination of values) is unique.

text/csv

Label:

situations_visual.csv

Text:

Gives information about prominent colours in machine vision situations, whether shown visually or described verbally, and about aesthetic characteristics and whether or not the situation is represented from the point of view of the machine.

text/csv

Label:

situation_description.csv

Text:

Includes textual descriptions of each situation explaining how machine vision technologies are used. Descriptions often include some interpretation and were written by the project team. The file also includes quotations (linguistic excerpts) from some of the works, particularly literary works, but also some movies, artworks and games where relevant. The SituationID and title (in the column Situation) can be used to join this data with the other files.

text/csv

Label:

worksinfo.csv

Text:

CSV file with full information about each creative work's WorkID, WikidataID if available, Title, Genre, Year of release, URL if available, whether or not is is science fiction, all Creators and all Countries affiliated with the work. The data is organised in 3 columns: WorkID, Variable and Value. The format is not very easily human-readable, but follows tidy data conventions. The file machinevisionscripts.R includes code for converting worksinfo.csv into a "wide" format where works with multiple creators or countries generate separate columns (Creator1, Creator2 etc).

text/csv