Methods

This dashboard studies how the vocabulary of genocide appears in English Security Council records. It combines word counts, statistical comparisons and language-model analyses. Use the figures to identify patterns, then read the speeches to interpret them.

The speech collection

The source is The UNSC Meetings and Speeches by Sakamoto and Matsuoka, Harvard Dataverse v5.0. The working collection contains 167,642 speeches from 9,464 meeting records and 86,854,907 words. The source covers 17 January 1946 to 30 December 2024. Here, corpus means this collection of texts.

The transcripts are in English, including translations. The dataset does not reliably identify the language actually spoken. Results therefore concern the English records. Scanning errors, missing metadata and the source's division of records into speeches can affect the analysis.

Where the results come from

Computed from the record identifies counts or calculations based on text and metadata. Model-derived · experimental identifies results based on a language model, including similarity embeddings and classifications of passages. Computed and model-derived combines the two, for example a concordance filtered by a model-assigned case.

These labels identify the method, not its accuracy. Each figure names its source files and offers available downloads. Navigation labels cover the methods a page can use; a figure's label reflects its own sources.

Reading counts and rates

What is being counted?

An occurrence is one match for a search term. A speech repeating the term ten times contributes ten occurrences but only one speech using the term. The share of speeches divides speeches with a match by all speeches in the relevant year, period or speaker's record. Occurrences per 100,000 words divides matches by the number of words, then multiplies by 100,000.

Rates help compare periods or delegations that produced different amounts of speech. They do not adjust for differences in agenda, speaking opportunities or the political meaning of a passage. Figures state their denominators and any minimum speech or occurrence count required to display a rate. A withheld rate is not zero.

The reading-set selector identifies speeches containing genocid*, speeches containing that term or related atrocity vocabulary, or all speeches in meetings where genocid* appears. It affects the figures labelled as reading-set summaries and the reader's membership markers. Other charts and concordance results use their own filters.

Search terms and word families

The search list is defined in config/lexicon.yml. Its patterns include spelling variants: genocid*, for example, captures forms such as genocide, genocidal and genocides. This word-family count is the default across the overview, chronology and actors pages. It appears in 2.47% of speeches.

Terms are grouped into families, called registers, such as legal, preventive or commemorative language. These are research choices used to organise and colour the interface. They do not classify a speech's purpose, and the term charts show individual terms separately. A match alone cannot establish that a speaker alleged, endorsed or denied genocide.

Testing for a change over time

The change-point test compares one constant rate with two rates separated at a candidate date. It searches the allowed dates and reports the best split. Speech shares use a binomial model; occurrence counts use a Poisson model with word totals to account for speech volume.

The search is repeated 2,000 times under a no-change model. Whole meetings are reassigned between years, keeping speeches within a meeting together. This accounts for shared debate context. The independent-speech result is also shown for comparison. The p-value measures how often the no-change procedure produces a contrast at least as strong as the observed one. It is not the probability that a historical explanation is true.

A split is accepted below 2.50%. This threshold accounts for multiple tests (Bonferroni across 2 planned rate tests). Each side must span at least 4 periods. An accepted split summarises a statistical difference; an unaccepted one means insufficient evidence under this test. Historical reference dates provide context, not causal evidence. The second change-point method is exploratory.

95% Wilson intervals accompany speech shares. They indicate precision under a model treating speeches as independent observations; wider intervals usually accompany smaller totals. The observed corpus share itself is a count of available records. These intervals do not account for clustered meetings, classification errors or missing records, and comparing overlapping intervals is not a formal significance test.

Understanding the vocabulary tables

A word window contains a specified number of words before and after a term. A collocate is a word counted in those windows. Overlapping windows are merged to avoid counting the same surrounding text twice. Common function words are removed; word forms such as crime and crimes remain separate.

Log ratio

Compares a word's frequency per word of text in two sets. +1 means twice the comparison frequency, +2 four times and +3 eight times. Zero means equal rates; negative values mean a lower rate. A small adjustment permits calculation when a count is zero.

G² (log-likelihood)

Tests departure from equal word frequencies in the two sets. Larger values can reflect a larger difference, more text or both. The published tables apply a minimum G² before ranking words. G² is neither a confidence percentage nor a measure of historical importance.

logDice

Ranks the association between a term and a nearby word using their joint count relative to their individual counts. Higher scores mean a stronger association. It is a different measure from log ratio; row order and dot position can therefore differ.

Spread (DP)

Measures how unevenly a word is distributed across speeches, allowing for speech length. Values near zero mean relatively even distribution; values near one mean concentration in a small part of the text. Speech and meeting counts help identify results dominated by a few debates.

nPMI

Measures whether two terms occur in the same speech more often than expected from their individual frequencies. Zero corresponds to independence; positive values indicate association, up to 1. A high score does not require the terms to occur next to each other. Rare pairs still need careful interpretation.

Choosing comparison speeches

For the genocide vocabulary comparison, speeches using the term are paired with speeches without it that share their year, agenda item and speaker group. 3,950 of 4,133 eligible speeches found a partner (95.57%), giving 3,950 comparison speeches. The 105 groups with insufficient partners remain short.

Matching reduces differences in the recorded context but cannot hold every aspect of a debate constant. The whole-corpus comparison shows how results change without matching. Random matching is repeated 20 times to assess sensitivity to the choice of partners. The Actors page applies a separate matched comparison to each delegation's vocabulary across its speeches.

Passages and sentence boundaries

The concordance gives each match with surrounding text and a full sentence. Rules account for abbreviations such as Mr. and document symbols such as S/PV.3453. Across 83,323 concordance entries, 2,261 (2.71%) have sentences longer than 500 characters. These may be long sentences or missed boundaries caused by scanning or sentence-splitting errors. Check the full speech before quoting.

Semantic embeddings and the speech map

A language model converts each speech into a list of numbers, its embedding. The numbers represent patterns of wording and meaning learned by the model. Comparing these lists can retrieve speeches about similar subjects even when their wording differs. Long speeches are divided into overlapping sections; their embeddings are combined using weights based on section length.

Cosine similarity compares the direction of two embeddings. Scores closer to 1 indicate more similar representations; the score is not a percentage of agreement or shared meaning. The related-speech list uses approximate search over the original embeddings and can miss some of the closest speeches.

UMAP places the embeddings on a flat map while trying to preserve local neighbours. Compressing many numerical dimensions into two changes distances. The axes have no substantive units, and large gaps or apparent clusters do not establish political blocs. Map filters hide speeches without recalculating their positions.

The map reports how many neighbours are lost in a diagnostic sample and how often approximate search recovers the ten closest speeches found by exact search. Publication requires at least 80% recovery in the search check. These checks evaluate computation and distortion; interpreting similarity still requires reading the texts. The Semantic map reports whether the data are available.

Model classifications and human review

The Usage page asks a language model to classify individual mentions. A referent is the event, situation or general concept a mention concerns. Speaker position records whether the passage asserts, rejects, conditionally applies or reports a genocide claim, among other categories. These labels describe the model's interpretation of a passage; they do not establish whether an event was genocide.

A valid model response must use the permitted categories and quote text found in the speech. Those checks cannot show that the interpretation is correct. The human reference sample is coded independently by two readers; unresolved disagreements and later adjudications are kept explicit. The Usage page reports coverage and review progress for the published run.

Observed agreement is the proportion of identical labels. Kappa adjusts for agreement expected from each reader's label frequencies; PABAK here uses an alternative equal-category chance baseline. They can differ sharply when one label dominates. Agreement between models measures consistency, not accuracy.

Precision asks how many model assignments to a category match the human reference; recall asks how many reference examples the model finds. F1 combines the two, with 1 indicating a perfect match. Weighted F1 gives frequent categories more weight; macro F1 averages the eligible categories equally. Read these scores with sample sizes, exclusions and abstentions.

Analysis steps and checks

The following scripts produce the displayed analyses or related optional outputs. Automatic checks cover data consistency and implementation; they do not mean a researcher has verified every passage. Model-based results require separate interpretation and validation.

Reproducing and citing the analysis

The repository contains scripts, configuration, tests and instructions. Analytical files record their inputs and settings. An analysis_hash is a content identifier: it changes with the analytical data or declared settings, but not solely with the generation time. Retain that identifier and the filters when saving or citing a result.

The source corpus is released under CC0. The code uses the MIT licence; project-generated tables and figures use CC BY 4.0. Cite the project using CITATION.cff and cite the source corpus as well.

The steps that need GPUs — embeddings, the topic comparison, lemmatisation and the model classifications — ran on festus, the university cluster. Calculations were performed using the festus-cluster of the Bayreuth Centre for High Performance Computing (https://www.bzhpc.uni-bayreuth.de), funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - 523317330.

Word-list version 7. Data generated 2026-09-18T05:55:48Z.