|
Description
| Turkish BIT Treaty Diffusion and Genealogy Network (Dataset 3). This dataset maps textual similarity, treaty families, and algorithmic predecessor relationships across all 141 international investment agreements in the Turkish BIT corpus. The corpus spans the 1962–2025 study period and contains agreements signed from June 1962 through April 2024. Dataset 3 is derived entirely from Dataset 1 treaty texts and Dataset 2 clause annotations; it adds no independently collected observations.
The dataset supports reproducible study of recurring drafting templates and their development over time. It should be used as a structured map of textual and coded legal similarity, not as proof that one treaty was copied from another. The canonical citation is Pehlivan, O. K. (2026), Turkish BIT Treaty Diffusion and Genealogy Network (1962–2025), DataverseNO, DOI 10.18710/WA7HEO.
Contents and table grains
similarity_matrix.csv: a complete 141 × 141 treaty-by-treaty TF-IDF cosine similarity matrix. Rows and columns use the common treaty_id key; the matrix is symmetric and its diagonal is 1.treaty_pairs.csv: one row for each of the 9,870 unordered treaty pairs. It reports cosine similarity, Jaccard lexical overlap, the number of shared Dataset 2 clause-feature packages from 0 to 9, and the absolute signature-year gap.treaty_families.csv: one row per treaty, with its deterministic family assignment, family label, family size, signature year, partner, and Pehlivan FPS Typology code.genealogy_edges.csv: one row per retained directed predecessor-to-child edge. The current table contains 88 edges, and each child has at most one retained parent.- Documentation and presentations:
00_README.txt, CODEBOOK.txt, five PNG figures, a self-contained visual-analysis page, an offline interactive network, and the third-party software licence notices. These presentation files add no observations beyond the four canonical CSV tables.
Methodology
Document construction and similarity
Each treaty document is assembled from Dataset 1 by concatenating every non-empty analytical cell in its article_extracts.csv row, in column order: the preamble, article titles, article bodies, and any protocol text. Formatting artefacts removed during Dataset 1 cleaning are excluded. Eleven protocol cells retain a main-treaty closing block that precedes an annex, so those passages also enter the corresponding analytical documents. TF-IDF vectors are generated with English stop words removed, sublinear term-frequency scaling, a maximum document-frequency threshold of 0.9, and unigrams only. Cosine similarity is then calculated for every treaty pair.
Jaccard similarity measures overlap between the two treaties' word-type sets after English stop-word removal. The pair table also counts nine new-model clause-feature packages present in both treaties, derived from Dataset 2: preambular environmental, social, or sustainable-development content; the right to regulate; denial of benefits; expropriation safeguards; general exceptions; essential security; an MFN–ISDS carve-out; operative environmental or social provisions; and ISDS transparency. This package count is not a count of identical clauses.
Treaty families
Families are produced by average-linkage agglomerative clustering on cosine distance, defined as 1 - cosine, with the canonical distance cut set to 0.65. Raw clusters are assigned F01, F02, and so on by the deterministic total key (-family_size, first_year, min_treaty_id). This rule resolves numbering ties by the smallest treaty identifier after family size and earliest member year have been considered.
Multi-member family labels summarize the modal Dataset 1 generation, membership size and year span, and modal Dataset 2 FPS type. Labels are tie-aware: every modal generation is retained in sorted order when generation counts tie, and every modal FPS code is retained in sorted order when FPS counts tie. Single-member families are labelled by their signature year.
Genealogy edges
Genealogy eligibility follows the ascending total order (signature_date, treaty_id), rather than a strictly earlier-calendar-date rule. For each child, the algorithm chooses the eligible predecessor with the highest cosine similarity; an exact cosine tie is resolved by the first candidate in the same total order. The edge is retained only when cosine similarity is at least 0.45.
This ordering makes a lower-identifier treaty signed on the same day eligible as a predecessor. TUR_BIT_018 and TUR_BIT_019 were both signed on 28 April 1992, so the published edge TUR_BIT_018 → TUR_BIT_019 is eligible and retained, with cosine similarity 0.861594 and a year gap of 0.
Current snapshot and key findings
- Coverage: 141 treaties, a 141 × 141 matrix, and all 9,870 unordered pairs.
- Cosine similarity: 0.030677–0.912594, with first quartile 0.174295, median 0.229369, mean 0.270719, and third quartile 0.316775. Twenty-four pairs have cosine similarity of at least 0.80, including two at or above 0.90.
- Jaccard similarity: 0.115232–0.993377, with first quartile 0.383069, median 0.458798, mean 0.489524, and third quartile 0.556356.
- Text and coded legal content: cosine similarity correlates positively with the number of shared clause-feature packages across all 9,870 pairs, with Pearson
r = 0.549810, reported substantively as r = 0.55.
- Families: 38 families, of which 24 are singletons and 14 have multiple members. The exact size distribution is 24 families of size 1, nine of size 2, and one family each of sizes 3, 4, 5, 41, and 46.
- Two dominant templates:
F01 is the New Turkish model with 46 treaties signed from 2009 to 2024 and dominant FPS Type C. F02 is the Liberalization model with 41 treaties signed from 1992 to 2009 and dominant FPS Type G. Together they contain 87 of the 141 treaties.
- Genealogy: 88 retained edges with cosine similarity from 0.457357 to 0.912594. Year gaps range from 0 to 17 years, with a median of 3; 18 edges connect treaties signed in the same year, and 87 of the 88 edges remain within the same textual family.
Sensitivity and interpretation caveats
Threshold sensitivity
At family-distance cuts of 0.600, 0.625, 0.650, 0.675, and 0.700, the matrix produces 51, 45, 38, 33, and 26 families, respectively, including 36, 32, 24, 21, and 16 singletons. Adjusted Rand indices against the canonical 0.65 partition are 0.825900, 0.925515, 1.000000, 0.978949, and 0.932099. The two dominant groupings persist throughout this range, although smaller-family membership changes.
The best predecessor is selected before the retention threshold is applied. Cosine cutoffs of 0.40, 0.45, and 0.50 retain 104, 88, and 78 edges; 97, 87, and 78 of those edges are within-family, corresponding to 93.27%, 98.86%, and 100.00%.
Interpretation
- A genealogy edge is an algorithmic best-predecessor relationship under the stated corpus, ordering, similarity measure, and threshold. It is not evidence by itself of copying, authorship, negotiation history, influence, or legal borrowing.
- Families are descriptive textual clusters, not authoritative legal categories. Results depend on document cleaning, English-language text or translation, vectorisation choices, linkage method, and clustering threshold.
- Jaccard overlap is lexical rather than semantic. The nine-package legal measure compresses selected Dataset 2 annotations and should not be interpreted as a count of identical provisions.
- The positive text–legal correlation is associative and does not establish causation. Dataset 2 coding definitions and provenance should be consulted before interpreting individual packages.
- Similarity values are published to six decimal places. Recalculation at full floating-point precision may differ within the documented rounding tolerance.
Reproducibility and validation
Dataset 3 is a derived release and should be regenerated rather than hand-edited. The canonical implementation is recompute_ds3.py, deposited with Dataset 1 in its code/ folder. The step-by-step procedure is documented in Steps 5 and 6 of Dataset 1's 00_REPRODUCIBILITY_PROTOCOL.txt. The process uses the current Dataset 1 article extracts and Dataset 2 annotations and writes the four Dataset 3 tables deterministically.
The current verification suite reproduces all four canonical tables exactly at their published precision. It confirms the 141 identifiers and matrix values, all 9,870 pair rows and pair-level measures, every family assignment and tie-aware label, and all 88 retained genealogy edges under the total-order and highest-cosine rules. It also checks matrix symmetry and the unit diagonal, source-derived Jaccard and clause-package values, helper columns, FPS alignment, and the absence of a missing above-threshold edge.
Ethics and licence
The dataset contains no human-participant research and no information collected through interviews, surveys, observation, or contact with individuals. No informed consent, research-ethics approval, or notification to Sikt was required. The source corpus can contain the names and official titles of state representatives as printed in officially promulgated treaty texts; they appear solely in their public official capacity. Dataset 3's analytical tables, figures, and network do not identify private individuals or contain special-category personal data.
Dataset 3 is released under the Creative Commons Attribution 4.0 International licence. The reproduction code deposited with Dataset 1 is released under the MIT licence. Bundled third-party software is governed by its own notices in THIRD_PARTY_LICENSES.txt; the offline network uses vis-network 10.1.0 under its MIT and Apache 2.0 licences.
Related datasets and identifiers
- Dataset 1: Turkish BIT Treaty Corpus. Treaty metadata, cleaned texts, article extracts, FPS classification, reproduction protocol, and code. DOI 10.18710/JX4WHH.
- Dataset 2: Turkish BIT Clause-Level Annotation Dataset. 118 substantive variables per treaty, clause extracts, and coding provenance. DOI 10.18710/FPNLKS.
- Dataset 3: Turkish BIT Treaty Diffusion and Genealogy Network. The present derived dataset. DOI 10.18710/WA7HEO.
- Development repository: https://github.com/ouzpehlivan/turkish-bit-corpus.
(2026-08-24) |
