Basket 0

Saved in this browser on this device. Export your basket to keep a separate copy or use it elsewhere. Anyone with access to this browser profile may be able to view it.

Nothing here yet. Add an occurrence from the concordance or the reader, and it will keep the sentence, the speaker and the record it came from.

What the word was doing

Examine which events or concepts speakers associate with genocide and whether they assert, reject or report a claim. The tables summarise language-model classifications. Open the quotations to assess them in context.

Experimental — model-derived

These are experimental model classifications. A referent is the case or concept a mention concerns; speaker position describes how the passage presents the claim. Human coding provides a separate reference for evaluation, with disagreements recorded. The progress figures below show how much has been reviewed. Word counts elsewhere do not depend on these labels.

Human validation has not started. Use these classifications to explore passages, not as validated research findings. Quotations with invalid evidence are excluded from the discourse figures.

Model
Qwen/Qwen3.8-27B · revision 1d4bf0f2ff60
Computing setup
vLLM 0.28.0 on 1 × NVIDIA H100 80GB HBM3; project-controlled hardware
Run
2026-11-03-qwen-131k · 2026-11-03
Prompt
v3 · sha256:3772c44d400f…
Reasoning setting
xhigh
Coverage
8,611 of 8,611 occurrences, over 860 requests
Uncertain labels
0 validity decisions · 22 cases or concepts · 5 speaker position
Excluded responses
0 responses without valid annotations · 0 incomplete responses · 24 invalid evidence quotations
Human reference sample
not started — 0 of 469 coded

Second opinion

A second model classified the same occurrences using the same instructions. Agreement measures consistency between the models; both can misread the same passage. The main figures use the published run. Human coding is needed to evaluate accuracy.

Second model
google/gemma-4-31B-it
Run
2026-11-03-gemma4 · 2026-11-03
Reasoning setting
high
Annotated
8,573 of 8,611 occurrences · 99.56%
Compared
8,573 carry a label from both runs
Excluded responses
14 abstentions · 33 invalid evidence quotations
How far the published run and the second opinion agree, field by field, over the occurrences both of them reached
FieldComparedAgreementKappaPABAKContestedSame model, twiceSame model, twice
verdict8,573100.00%—1.000100.00%100.00%
quotation8,57395.11%0.800.9441999.94%99.96%
concrete case8,57396.03%0.910.9534099.96%99.94%
speaker position8,57393.62%0.880.9354799.96%99.94%
referent8,57389.15%0.880.8993099.96%99.92%

A passage can receive several rhetorical-function labels. Their mean overlap between runs is 0.82 (Jaccard similarity: 1 means identical sets). Krippendorff's α is 0.72; it adjusts for chance agreement using a distance measure for partially overlapping sets (MASI). The runs assign different sets to 2,202 occurrences. The table below compares individual labels.

Observed agreement is the share of identical labels. Kappa adjusts for agreement expected from each run's label frequencies. It is withheld when fewer than 1% of labels fall outside a run's most common category. PABAK uses an alternative, equal-category chance baseline. Read both with the category counts; neither validates the interpretation.

Same model, twice compares repeated runs using identical instructions. This shows whether classifications change even without changing models. Qwen/Qwen3.8-27B returned identical labels across all compared fields for 5,199 of 5,218 occurrences.google/gemma-4-31B-it returned identical labels across all compared fields for 5,225 of 5,233 occurrences.Compare this variation with the differences between models.

How far the two runs agree that each function label applies
FunctionQwen/Qwen3.8-27Bgoogle/gemma-4-31B-itAgreementKappa
accountability1,9102,04694.10%0.83
accusation or qualification4,4494,45687.72%0.75
commemoration52071996.19%0.72
institutional title or mandate83188697.75%0.88
other53025594.62%0.39
warning or prevention1,8261,38991.71%0.73

Reporting another person's claim can resemble asserting it. Compare the reporting and assertion labels in the table, then read the disputed passages. Differences may reflect how each model interprets that boundary.

3,112 of 8,573 compared occurrences (36.30%) are read differently on at least one compared field. They are listed under The contested passages below, and marked wherever they appear in the quotations.

Which genocide is each delegation talking about when it says the word?

Model-derived · experimental
Unit
occurrences placed on a referent, published for every delegation
occurrences 0 864 no assigned mentions
Delegations down the side, referents across the top; each cell holds the occurrences that delegation placed on that referent.
Delegation

Press Tab to enter the table, then use the arrow keys to move between cells and headings. Home and End move to the first and last cells in a row.

Showing 70 of 238 affiliations with assigned mentions. The download includes 168 additional affiliations with 1,565 mentions. 32 omitted affiliations meet the share threshold; the display also has a row limit.

Cells count individual mentions or their share, according to the selected unit. Repeated mentions within one speech count separately.

Source

15_usage.py → usage/usage.json

Download

The occurrences behind a cell

Pick a cell, a delegation or a referent above and the occurrences behind it are listed here, each with the sentence it was read from and a way into the speech it came from.

When did each delegation first place the word on this genocide, first assert it, and first refuse it?

Model-derived · experimental
Placed the word on it117Refused the word for it3Asserted it115

The model assigns mentions of Rwanda to 117 affiliations. Of these, 115 have an assertion and 3 a rejection. An affiliation can appear in both curves if different passages receive those labels. 247 affiliations spoke in at least one meeting where Rwanda was named; read the curves against that number, not against every state.

Every first the curves are made of, for Rwanda, oldest first
DateDelegationMilestonePositionCumulative countOccurrence
5 May 1994Czech RepublicPlaced the word on itAsserts1SC03375-01-016#1
5 May 1994Czech RepublicAsserted itAsserts1SC03375-01-016#1
16 May 1994New ZealandPlaced the word on itAsserts2SC03377-01-015#1
16 May 1994New ZealandAsserted itAsserts2SC03377-01-015#1
8 June 1994DjiboutiPlaced the word on itAsserts3SC03388-02-002#1
8 June 1994DjiboutiAsserted itAsserts3SC03388-02-002#1
8 June 1994ArgentinaPlaced the word on itAsserts4SC03388-02-014#1
8 June 1994ArgentinaAsserted itAsserts4SC03388-02-014#1
8 June 1994United KingdomPlaced the word on itAsserts5SC03388-02-016#1
8 June 1994United KingdomAsserted itAsserts5SC03388-02-016#1
8 June 1994SpainPlaced the word on itReports without a position6SC03388-02-018#1
8 June 1994FrancePlaced the word on itAsserts7SC03388-02-023#1
8 June 1994FranceAsserted itAsserts6SC03388-02-023#1
8 June 1994United StatesPlaced the word on itAsserts8SC03388-02-026#1
8 June 1994United StatesAsserted itAsserts7SC03388-02-026#1
1 July 1994SpainAsserted itAsserts8SC03400-01-003#2
10 August 1994RussiaPlaced the word on itAsserts9SC03414-01-002#1
10 August 1994RussiaAsserted itAsserts9SC03414-01-002#1
8 November 1994BrazilPlaced the word on itAsserts10SC03453-01-010#1
8 November 1994BrazilAsserted itAsserts10SC03453-01-010#1
8 November 1994ChinaPlaced the word on itAsserts11SC03453-01-012#1
8 November 1994ChinaAsserted itAsserts11SC03453-01-012#1
8 November 1994NigeriaPlaced the word on itAsserts12SC03453-01-014#1
8 November 1994NigeriaAsserted itAsserts12SC03453-01-014#1
8 November 1994RwandaPlaced the word on itReports without a position13SC03453-01-015#1
8 November 1994RwandaAsserted itAsserts13SC03453-01-015#10
8 November 1994OmanPlaced the word on itAsserts14SC03453-01-016#1
8 November 1994OmanAsserted itAsserts14SC03453-01-016#1
30 November 1994United StatesRefused the word for itRejects1SC03473-01-013#1
18 January 1995CanadaPlaced the word on itAsserts15SC03492-02-023#1
18 January 1995CanadaAsserted itAsserts15SC03492-02-023#1
9 June 1995IndonesiaPlaced the word on itAsserts16SC03542-01-008#1
9 June 1995IndonesiaAsserted itAsserts16SC03542-01-008#1
9 June 1995BotswanaPlaced the word on itAsserts17SC03542-01-010#1
9 June 1995BotswanaAsserted itAsserts17SC03542-01-010#1
9 June 1995HondurasPlaced the word on itAsserts18SC03542-01-012#1
9 June 1995HondurasAsserted itAsserts18SC03542-01-012#1
9 June 1995GermanyPlaced the word on itAsserts19SC03542-01-034#1
9 June 1995GermanyAsserted itAsserts19SC03542-01-034#1
28 August 1995BurundiPlaced the word on itAsserts20SC03571-01-003#2
28 August 1995BurundiAsserted itAsserts20SC03571-01-003#2
7 September 1995ItalyPlaced the word on itAsserts21SC03574-01-027#1
7 September 1995ItalyAsserted itAsserts21SC03574-01-027#1
22 November 1995Bosnia and HerzegovinaPlaced the word on itAsserts22SC03595-01-024#1
22 November 1995Bosnia and HerzegovinaAsserted itAsserts22SC03595-01-024#1
20 December 1995AustraliaPlaced the word on itAsserts23SC03611-02-026#1
20 December 1995AustraliaAsserted itAsserts23SC03611-02-026#1
5 March 1996PolandPlaced the word on itAsserts24SC03639-01-030#1
5 March 1996PolandAsserted itAsserts24SC03639-01-030#1
8 March 1996ChilePlaced the word on itAsserts25SC03640-01-006#1
8 March 1996ChileAsserted itAsserts25SC03640-01-006#1
8 March 1996EgyptPlaced the word on itAsserts26SC03640-01-020#1
8 March 1996EgyptAsserted itAsserts26SC03640-01-020#1
15 November 1996DR CongoPlaced the word on itAsserts27SC03713-02-003#1
15 November 1996DR CongoAsserted itAsserts27SC03713-02-003#1
25 September 1997Organization of African UnityPlaced the word on itAsserts28SC03819-01-009#1
25 September 1997Organization of African UnityAsserted itAsserts28SC03819-01-009#1
9 April 1998PortugalPlaced the word on itAsserts29SC03870-01-010#1
9 April 1998PortugalAsserted itAsserts29SC03870-01-010#1
9 April 1998SwedenPlaced the word on itAsserts30SC03870-01-013#1
9 April 1998SwedenAsserted itAsserts30SC03870-01-013#1
9 April 1998SloveniaPlaced the word on itAsserts31SC03870-01-016#1
9 April 1998SloveniaAsserted itAsserts31SC03870-01-016#1
9 April 1998JapanPlaced the word on itAsserts32SC03870-01-022#1
9 April 1998JapanAsserted itAsserts32SC03870-01-022#1
24 April 1998BahrainPlaced the word on itAsserts33SC03875-01-015#1
24 April 1998BahrainAsserted itAsserts33SC03875-01-015#1
24 April 1998United NationsPlaced the word on itAsserts34SC03875-02-012#1
24 April 1998United NationsAsserted itAsserts34SC03875-02-012#1
30 April 1998Costa RicaPlaced the word on itAsserts35SC03877-01-008#1
30 April 1998Costa RicaAsserted itAsserts35SC03877-01-008#1
30 April 1998GabonPlaced the word on itAsserts36SC03877-01-012#1
30 April 1998GabonAsserted itAsserts36SC03877-01-012#1
21 January 1999NetherlandsPlaced the word on itAsserts37SC03968-01-011#1
21 January 1999NetherlandsAsserted itAsserts37SC03968-01-011#1
19 March 1999UgandaPlaced the word on itAsserts38SC03987-01-013#1
19 March 1999UgandaAsserted itAsserts38SC03987-01-013#1
17 September 1999IraqPlaced the word on itAsserts39SC04046-03-010#1
17 September 1999IraqAsserted itAsserts39SC04046-03-010#1
29 November 1999MalaysiaPlaced the word on itAsserts40SC04072-02-020#2
29 November 1999MalaysiaAsserted itAsserts40SC04072-02-020#2
30 November 1999LiechtensteinPlaced the word on itAsserts41SC04072-01-004#1
30 November 1999LiechtensteinAsserted itAsserts41SC04072-01-004#1
30 November 1999ZambiaPlaced the word on itAsserts42SC04072-01-006#1
30 November 1999ZambiaAsserted itAsserts42SC04072-01-006#1
26 January 2000UkrainePlaced the word on itAsserts43SC04092-03-008#1
26 January 2000UkraineAsserted itAsserts43SC04092-03-008#1
14 April 2000MaliPlaced the word on itAsserts44SC04127-01-010#1
14 April 2000MaliAsserted itAsserts44SC04127-01-010#1
14 April 2000TunisiaPlaced the word on itAsserts45SC04127-01-016#1
14 April 2000TunisiaAsserted itAsserts45SC04127-01-016#1
14 April 2000NamibiaPlaced the word on itAsserts46SC04127-01-019#1
14 April 2000NamibiaAsserted itAsserts46SC04127-01-019#1
14 April 2000BangladeshPlaced the word on itAsserts47SC04127-01-021#1
14 April 2000BangladeshAsserted itAsserts47SC04127-01-021#1
14 April 2000JamaicaPlaced the word on itAsserts48SC04127-01-022#1
14 April 2000JamaicaAsserted itAsserts48SC04127-01-022#1
19 April 2000SingaporePlaced the word on itAsserts49SC04130-01-010#1
19 April 2000SingaporeAsserted itAsserts49SC04130-01-010#1
26 July 2000IndiaPlaced the word on itAsserts50SC04176-03-023#1
26 July 2000IndiaAsserted itAsserts50SC04176-03-023#1
16 January 2001MauritiusPlaced the word on itAsserts51SC04257-02-013#1
16 January 2001MauritiusAsserted itAsserts51SC04257-02-013#1
19 January 2001ColombiaPlaced the word on itAsserts52SC04259-02-001#1
19 January 2001ColombiaAsserted itAsserts52SC04259-02-001#1
7 February 2001IrelandPlaced the word on itAsserts53SC04273-01-012#1
7 February 2001IrelandAsserted itAsserts53SC04273-01-012#1
7 February 2001NorwayPlaced the word on itAsserts54SC04273-01-016#1
7 February 2001NorwayAsserted itAsserts54SC04273-01-016#1
10 July 2002JordanPlaced the word on itAsserts55SC04568-01-023#1
10 July 2002JordanAsserted itAsserts55SC04568-01-023#1
8 August 2002South AfricaPlaced the word on itAsserts56SC04596-01-008#1
8 August 2002South AfricaAsserted itAsserts56SC04596-01-008#1
11 October 2002SenegalPlaced the word on itAsserts57SC04623-02-043#1
11 October 2002SenegalAsserted itAsserts57SC04623-02-043#1
22 October 2002IBRDPlaced the word on itAsserts58SC04630-01-009#1
22 October 2002IBRDAsserted itAsserts58SC04630-01-009#1
10 December 2002DenmarkPlaced the word on itAsserts59SC04660-01-002#1
10 December 2002DenmarkAsserted itAsserts59SC04660-01-002#1
13 May 2003EthiopiaPlaced the word on itAsserts60SC04753-02-015#2
13 May 2003EthiopiaAsserted itAsserts60SC04753-02-015#2
12 June 2003MalawiPlaced the word on itAsserts61SC04772-01-021#2
12 June 2003MalawiAsserted itAsserts61SC04772-01-021#2
9 October 2003GuineaPlaced the word on itAsserts62SC04838-01-023#1
9 October 2003GuineaAsserted itAsserts62SC04838-01-023#1
20 November 2003African UnionPlaced the word on itAsserts63SC04865-01-008#1
20 November 2003African UnionAsserted itAsserts63SC04865-01-008#1
20 November 2003AngolaPlaced the word on itAsserts64SC04865-01-030#1
20 November 2003AngolaAsserted itAsserts64SC04865-01-030#1
20 November 2003MozambiquePlaced the word on itAsserts65SC04865-01-033#1
20 November 2003MozambiqueAsserted itAsserts65SC04865-01-033#1
24 November 2003Economic Community of Central African StatesPlaced the word on itAsserts66SC04871-01-032#1
24 November 2003Economic Community of Central African StatesAsserted itAsserts66SC04871-01-032#1
14 May 2004PakistanPlaced the word on itAsserts67SC04969-01-002#1
14 May 2004PakistanAsserted itAsserts67SC04969-01-002#1
17 May 2004PeruPlaced the word on itAsserts68SC04970-02-014#1
17 May 2004PeruAsserted itAsserts68SC04970-02-014#1
28 May 2004PhilippinesPlaced the word on itRejects69SC04980-01-011#1
28 May 2004PhilippinesRefused the word for itRejects2SC04980-01-011#1
29 June 2004BeninPlaced the word on itAsserts70SC04999-02-002#2
29 June 2004BeninAsserted itAsserts69SC04999-02-002#2
18 November 2004Intergovernmental Authority on DevelopmentPlaced the word on itAsserts71SC05080-01-013#1
18 November 2004Intergovernmental Authority on DevelopmentAsserted itAsserts70SC05080-01-013#1
14 December 2004Ivory CoastPlaced the word on itAsserts72SC05100-01-025#1
14 December 2004Ivory CoastAsserted itAsserts71SC05100-01-025#1
13 June 2005AlgeriaPlaced the word on itAsserts73SC05199-01-022#1
13 June 2005AlgeriaAsserted itAsserts72SC05199-01-022#1
21 June 2005TanzaniaPlaced the word on itAsserts74SC05209-01-006#1
21 June 2005TanzaniaAsserted itAsserts73SC05209-01-006#1
27 January 2006KenyaPlaced the word on itAsserts75SC05359-01-013#1
27 January 2006KenyaAsserted itAsserts74SC05359-01-013#1
27 January 2006GreecePlaced the word on itAsserts76SC05359-01-026#1
27 January 2006GreeceAsserted itAsserts75SC05359-01-026#1
27 January 2006South KoreaPlaced the word on itAsserts77SC05359-01-034#1
27 January 2006South KoreaAsserted itAsserts76SC05359-01-034#1
27 January 2006European CommissionPlaced the word on itAsserts78SC05359-02-013#1
27 January 2006European CommissionAsserted itAsserts77SC05359-02-013#1
7 June 2006GhanaPlaced the word on itAsserts79SC05453-01-018#2
7 June 2006GhanaAsserted itAsserts78SC05453-01-018#2
7 June 2006SlovakiaPlaced the word on itAsserts80SC05453-01-024#1
7 June 2006SlovakiaAsserted itAsserts79SC05453-01-024#1
15 December 2006QatarPlaced the word on itAsserts81SC05594-01-021#1
15 December 2006QatarAsserted itAsserts80SC05594-01-021#1
18 June 2007CongoPlaced the word on itAsserts82SC05697-01-011#1
18 June 2007CongoAsserted itAsserts81SC05697-01-011#1
12 December 2008LibyaPlaced the word on itAsserts83SC06041-01-015#1
12 December 2008LibyaAsserted itAsserts82SC06041-01-015#1
4 June 2009Burkina FasoPlaced the word on itAsserts84SC06134-01-024#1
4 June 2009Burkina FasoAsserted itAsserts83SC06134-01-024#1
11 November 2009CroatiaPlaced the word on itAsserts85SC06216-01-010#1
11 November 2009CroatiaAsserted itAsserts84SC06216-01-010#1
12 June 2013European UnionPlaced the word on itAsserts86SC06977-01-031#1
12 June 2013European UnionAsserted itAsserts85SC06977-01-031#1
23 July 2013LuxembourgPlaced the word on itAsserts87SC07007-01-015#1
23 July 2013LuxembourgAsserted itAsserts86SC07007-01-015#1
23 July 2013Saudi ArabiaPlaced the word on itAsserts88SC07007-01-025#1
23 July 2013Saudi ArabiaAsserted itAsserts87SC07007-01-025#1
29 January 2014IsraelPlaced the word on itAsserts89SC07105-01-025#1
29 January 2014IsraelAsserted itAsserts88SC07105-01-025#1
12 February 2014EstoniaPlaced the word on itAsserts90SC07109-01-025#1
12 February 2014EstoniaAsserted itAsserts89SC07109-01-025#1
12 February 2014MexicoPlaced the word on itAsserts91SC07109-01-035#2
12 February 2014MexicoAsserted itAsserts90SC07109-01-035#2
12 February 2014UruguayPlaced the word on itAsserts92SC07109-01-067#1
12 February 2014UruguayAsserted itAsserts91SC07109-01-067#1
12 February 2014TurkeyPlaced the word on itAsserts93SC07109-01-083#1
12 February 2014TurkeyAsserted itAsserts92SC07109-01-083#1
16 April 2014LithuaniaPlaced the word on itAsserts94SC07155-01-015#10
16 April 2014LithuaniaAsserted itAsserts93SC07155-01-015#10
16 April 2014ChadPlaced the word on itAsserts95SC07155-01-020#2
16 April 2014ChadAsserted itAsserts94SC07155-01-020#2
19 January 2015ArmeniaPlaced the word on itAsserts96SC07361-01-119#1
19 January 2015ArmeniaAsserted itAsserts95SC07361-01-119#1
23 February 2015BelgiumPlaced the word on itAsserts97SC07389-01-055#1
23 February 2015BelgiumAsserted itAsserts96SC07389-01-055#1
27 May 2015VenezuelaPlaced the word on itAsserts98SC07450-01-022#1
27 May 2015VenezuelaAsserted itAsserts97SC07450-01-022#1
15 May 2017BoliviaPlaced the word on itAsserts99SC07938-01-017#2
15 May 2017BoliviaAsserted itAsserts98SC07938-01-017#2
8 June 2017SudanPlaced the word on itAsserts100SC07963-01-019#5
8 June 2017SudanAsserted itAsserts99SC07963-01-019#5
21 February 2018KuwaitPlaced the word on itAsserts101SC08185-01-005#1
21 February 2018KuwaitAsserted itAsserts100SC08185-01-005#1
16 April 2018Kaladan Press (Organization)Placed the word on itAsserts102SC08234-01-006#1
16 April 2018Kaladan Press (Organization)Asserted itAsserts101SC08234-01-006#1
6 June 2018Equatorial GuineaPlaced the word on itAsserts103SC08278-01-014#1
6 June 2018Equatorial GuineaAsserted itAsserts102SC08278-01-014#1
23 May 2019FijiPlaced the word on itAsserts104SC08534-01-071#1
23 May 2019FijiAsserted itAsserts103SC08534-01-071#1
7 October 2019George Washington University (Washington, D.C.)Placed the word on itAsserts105SC08633-01-004#1
7 October 2019George Washington University (Washington, D.C.)Asserted itAsserts104SC08633-01-004#1
9 September 2021Women and Peace Studies Organization (Afghanistan)Placed the word on itAsserts106SC08853-01-004#1
9 September 2021Women and Peace Studies Organization (Afghanistan)Asserted itAsserts105SC08853-01-004#1
12 December 2022United Arab EmiratesPlaced the word on itAsserts107SC09217-01-008#1
12 December 2022United Arab EmiratesAsserted itAsserts106SC09217-01-008#1
12 June 2023EcuadorPlaced the word on itAsserts108SC09344-01-019#3
12 June 2023EcuadorAsserted itAsserts107SC09344-01-019#3
28 September 2023DR CongoRefused the word for itRejects3SC09427-01-025#1
20 October 2023(personal capacity)Placed the word on itReports without a position109SC09448-01-006#1
25 October 2023Dominican RepublicPlaced the word on itAsserts110SC09452-02-052#1
25 October 2023Dominican RepublicAsserted itAsserts108SC09452-02-052#1
12 December 2023SwitzerlandPlaced the word on itAsserts111SC09502-01-011#1
12 December 2023SwitzerlandAsserted itAsserts109SC09502-01-011#1
30 April 2024YugoslaviaPlaced the word on itAsserts112SC09621-01-027#1
30 April 2024YugoslaviaAsserted itAsserts110SC09621-01-027#1
21 May 2024MaltaPlaced the word on itAsserts113SC09632-01-017#1
21 May 2024MaltaAsserted itAsserts111SC09632-01-017#1
22 May 2024MaldivesPlaced the word on itAsserts114SC09632-03-046#5
22 May 2024MaldivesAsserted itAsserts112SC09632-03-046#5
28 May 2025Sierra LeonePlaced the word on itAsserts115SC09923-01-015#1
28 May 2025Sierra LeoneAsserted itAsserts113SC09923-01-015#1
15 April 2026LatviaPlaced the word on itAsserts116SC10136-01-016#1
15 April 2026LatviaAsserted itAsserts114SC10136-01-016#1
17 June 2026SomaliaPlaced the word on itAsserts117SC10175-01-017#1
17 June 2026SomaliaAsserted itAsserts115SC10175-01-017#1

Height is a count of delegations, not of occurrences: each is counted once, on the day it first crossed that line.

Source

15_usage.py → usage/usage.json

Download

Which situations is the word applied to in each year, and how often to none?

Model-derived · experimental

Bars count occurrences the model applied to a case; the line below is a share of all of them.

Source

15_usage.py → usage/usage.json

Download

Which passages did the two models read differently, and how?

Model-derived · experimental

Loading the annotations and the concordance for genocide…

One row per occurrence, not per disagreement: an occurrence the two runs split on three fields is one row, ranked above one they split on a single field.

Source

15_usage.py → usage/occurrences.json

Download

When a delegation says genocide, is it making the claim or refusing it?

Model-derived · experimental
RejectsReports without a position †ConditionalAssertsUnclearNot applicableNo position (not a case)

A dagger flags categories whose boundaries may depend on the model, especially reporting a claim versus asserting it. Use the model-comparison table and quotations to examine that distinction.

Delegations with at least 20 eligible occurrences, ordered by how many of them the model labelled as rejecting the characterisation, then by name
DelegationEligibleRejectsPositions
Yugoslavia6032 asserts 9, reports without a position 1, rejects 32, no position (not a case) 18
India8729 asserts 32, reports without a position 5, rejects 29, no position (not a case) 21
Sudan9225 asserts 44, reports without a position 1, rejects 25, conditional 1, no position (not a case) 21
Israel17618 asserts 127, reports without a position 2, rejects 18, conditional 5, no position (not a case) 24
United States35217 asserts 223, reports without a position 4, rejects 17, conditional 22, no position (not a case) 86
Russia12010 asserts 67, reports without a position 2, rejects 10, conditional 4, no position (not a case) 37
DR Congo948 asserts 67, reports without a position 3, rejects 8, conditional 2, no position (not a case) 14
South Africa997 asserts 43, reports without a position 1, rejects 7, conditional 11, no position (not a case) 37
Turkey696 asserts 35, reports without a position 2, rejects 6, conditional 1, no position (not a case) 25
United Kingdom1766 asserts 98, reports without a position 1, rejects 6, conditional 8, no position (not a case) 63
Cyprus275 asserts 15, rejects 5, no position (not a case) 7
Pakistan1575 asserts 92, reports without a position 2, rejects 5, conditional 20, no position (not a case) 38
Albania654 asserts 39, rejects 4, conditional 1, no position (not a case) 21
China424 asserts 11, reports without a position 2, rejects 4, conditional 3, no position (not a case) 22
Portugal274 asserts 7, reports without a position 1, rejects 4, no position (not a case) 15
Azerbaijan443 asserts 14, rejects 3, no position (not a case) 27
Burundi853 asserts 42, reports without a position 4, rejects 3, conditional 14, no position (not a case) 22
Congo212 asserts 9, reports without a position 1, rejects 2, conditional 1, no position (not a case) 8
Philippines202 rejects 2, conditional 1, no position (not a case) 17
Slovakia242 asserts 2, rejects 2, no position (not a case) 20
Syria1202 asserts 93, reports without a position 3, rejects 2, conditional 6, no position (not a case) 16
Ukraine942 asserts 55, reports without a position 4, rejects 2, conditional 1, no position (not a case) 30, unclear 2
(personal capacity)1311 asserts 109, reports without a position 2, rejects 1, conditional 5, no position (not a case) 14
Angola331 asserts 16, rejects 1, conditional 2, no position (not a case) 14
Gabon221 asserts 10, rejects 1, no position (not a case) 11
Hungary201 asserts 2, reports without a position 1, rejects 1, no position (not a case) 16
International Criminal Court521 asserts 30, reports without a position 1, rejects 1, conditional 5, no position (not a case) 15
Ireland351 asserts 14, rejects 1, conditional 1, no position (not a case) 19
Kenya231 asserts 10, rejects 1, no position (not a case) 12
Libya551 asserts 41, rejects 1, conditional 3, no position (not a case) 10
Liechtenstein701 asserts 12, rejects 1, no position (not a case) 57
Lithuania691 asserts 25, rejects 1, conditional 5, no position (not a case) 38
Norway201 asserts 4, rejects 1, no position (not a case) 15
Qatar291 asserts 12, reports without a position 1, rejects 1, conditional 4, no position (not a case) 11
Sweden211 asserts 8, reports without a position 1, rejects 1, conditional 1, no position (not a case) 10
United Nations6041 asserts 340, reports without a position 11, rejects 1, conditional 44, no position (not a case) 208
Afghanistan250 asserts 15, conditional 5, no position (not a case) 5
Algeria800 asserts 65, conditional 7, no position (not a case) 8
Argentina870 asserts 36, reports without a position 1, conditional 3, no position (not a case) 47
Armenia970 asserts 32, conditional 3, no position (not a case) 62
Australia410 asserts 17, reports without a position 1, conditional 4, no position (not a case) 19
Bangladesh600 asserts 38, reports without a position 1, conditional 1, no position (not a case) 20
Belgium350 asserts 6, reports without a position 2, conditional 1, no position (not a case) 26
Bolivia210 asserts 12, no position (not a case) 9
Bosnia and Herzegovina1860 asserts 140, conditional 4, no position (not a case) 42
Botswana230 asserts 11, reports without a position 1, no position (not a case) 11
Brazil370 asserts 11, conditional 5, no position (not a case) 21
Canada630 asserts 26, reports without a position 1, conditional 1, no position (not a case) 35
Chad500 asserts 35, no position (not a case) 15
Chile450 asserts 9, conditional 2, no position (not a case) 33, unclear 1
Colombia350 asserts 24, reports without a position 2, conditional 1, no position (not a case) 7, unclear 1
Costa Rica320 asserts 8, no position (not a case) 24
Croatia730 asserts 44, conditional 1, no position (not a case) 28
Cuba990 asserts 93, reports without a position 1, conditional 1, no position (not a case) 4
Czech Republic330 asserts 22, reports without a position 1, conditional 1, no position (not a case) 9
Denmark220 asserts 7, conditional 2, no position (not a case) 13
Ecuador260 asserts 5, conditional 3, no position (not a case) 18
Egypt310 asserts 20, no position (not a case) 11
Estonia250 asserts 11, conditional 2, no position (not a case) 12
European Union220 asserts 9, no position (not a case) 13
France2080 asserts 118, reports without a position 3, conditional 7, no position (not a case) 80
Germany560 asserts 24, reports without a position 1, conditional 2, no position (not a case) 29
Ghana410 asserts 13, reports without a position 1, conditional 2, no position (not a case) 25
High Representative for the Implementation of the Peace Agreement on Bosnia and Herzegovina420 asserts 36, no position (not a case) 6
Indonesia250 asserts 16, reports without a position 1, conditional 3, no position (not a case) 5
Iran1170 asserts 98, conditional 4, no position (not a case) 15
Iraq680 asserts 53, reports without a position 4, conditional 4, no position (not a case) 7
Italy290 asserts 4, conditional 2, no position (not a case) 23
Jordan800 asserts 54, conditional 3, no position (not a case) 23
Kosovo370 asserts 34, conditional 1, no position (not a case) 2
Kuwait480 asserts 27, conditional 5, no position (not a case) 16
League of Arab States390 asserts 37, conditional 1, no position (not a case) 1
Luxembourg550 asserts 20, reports without a position 1, conditional 3, no position (not a case) 31
Malaysia600 asserts 32, reports without a position 1, conditional 5, no position (not a case) 22
Maldives290 asserts 20, conditional 1, no position (not a case) 8
Mauritania360 asserts 34, conditional 1, no position (not a case) 1
Mexico420 asserts 8, no position (not a case) 34
Morocco260 asserts 13, no position (not a case) 13
Mozambique280 asserts 8, conditional 3, no position (not a case) 17
Netherlands580 asserts 22, reports without a position 1, conditional 7, no position (not a case) 28
New Zealand660 asserts 39, reports without a position 1, conditional 5, no position (not a case) 21
Nicaragua340 asserts 28, reports without a position 1, conditional 1, no position (not a case) 4
Nigeria700 asserts 32, conditional 2, no position (not a case) 36
Palestine2420 asserts 194, reports without a position 1, conditional 10, no position (not a case) 37
Palestine Liberation Organization530 asserts 46, conditional 4, no position (not a case) 3
Peru430 asserts 5, reports without a position 1, no position (not a case) 37
Poland300 asserts 8, conditional 3, no position (not a case) 19
Rwanda1,0950 asserts 933, reports without a position 1, conditional 12, no position (not a case) 149
Saudi Arabia420 asserts 35, conditional 1, no position (not a case) 6
Senegal290 asserts 14, conditional 2, no position (not a case) 13
Sierra Leone830 asserts 27, conditional 6, no position (not a case) 50
Singapore200 asserts 11, no position (not a case) 9
Slovenia710 asserts 21, conditional 9, no position (not a case) 41
South Korea300 asserts 7, conditional 1, no position (not a case) 22
Spain410 asserts 15, reports without a position 1, conditional 1, no position (not a case) 24
Switzerland370 asserts 9, conditional 9, no position (not a case) 19
Tanzania330 asserts 16, reports without a position 1, conditional 2, no position (not a case) 14
Tunisia460 asserts 35, conditional 1, no position (not a case) 10
Uganda210 asserts 16, conditional 1, no position (not a case) 4
United Arab Emirates260 asserts 14, conditional 1, no position (not a case) 11
Uruguay360 asserts 3, conditional 4, no position (not a case) 29
Venezuela690 asserts 43, reports without a position 2, conditional 2, no position (not a case) 22
All position counts, withheld delegations included
DelegationEligibleAssertsReports without a positionRejectsConditionalNo position (not a case)UnclearNot applicable
Yugoslavia60913201800
India873252902100
Sudan924412512100
Israel17612721852400
United States352223417228600
Russia1206721043700
DR Congo94673821400
South Africa994317113700
Turkey69352612500
United Kingdom176981686300
Cyprus2715050700
Pakistan1579225203800
Albania65390412100
China42112432200
Portugal2771401500
Azerbaijan44140302700
Burundi854243142200
Congo219121800
Philippines2000211700
Slovakia2420202000
Syria120933261600
Ukraine94554213020
(personal capacity)1311092151400
Angola33160121400
Gabon22100101100
Hungary2021101600
International Criminal Court52301151500
Ireland35140111900
Kenya23100101200
Libya55410131000
Liechtenstein70120105700
Lithuania69250153800
Norway2040101500
Qatar29121141100
Sweden2181111000
United Nations6043401114420800
Afghanistan2515005500
Algeria8065007800
Argentina87361034700
Armenia97320036200
Australia41171041900
Bangladesh60381012000
Belgium3562012600
Bolivia2112000900
Bosnia and Herzegovina1861400044200
Botswana23111001100
Brazil37110052100
Canada63261013500
Chad50350001500
Chile4590023310
Colombia3524201710
Costa Rica3280002400
Croatia73440012800
Cuba9993101400
Czech Republic3322101900
Denmark2270021300
Ecuador2650031800
Egypt31200001100
Estonia25110021200
European Union2290001300
France2081183078000
Germany56241022900
Ghana41131022500
High Representative for the Implementation of the Peace Agreement on Bosnia and Herzegovina4236000600
Indonesia2516103500
Iran117980041500
Iraq6853404700
Italy2940022300
Jordan80540032300
Kosovo3734001200
Kuwait48270051600
League of Arab States3937001100
Luxembourg55201033100
Malaysia60321052200
Maldives2920001800
Mauritania3634001100
Mexico4280003400
Morocco26130001300
Mozambique2880031700
Netherlands58221072800
New Zealand66391052100
Nicaragua3428101400
Nigeria70320023600
Palestine24219410103700
Palestine Liberation Organization5346004300
Peru4351003700
Poland3080031900
Rwanda1,095933101214900
Saudi Arabia4235001600
Senegal29140021300
Sierra Leone83270065000
Singapore2011000900
Slovenia71210094100
South Korea3070012200
Spain41151012400
Switzerland3790091900
Tanzania33161021400
Tunisia46350011000
Uganda2116001400
United Arab Emirates26140011100
Uruguay3630042900
Venezuela69432022200
Fewer than 20 eligible mentions, so no band in the figure
Myanmar90040500
Somalia179041300
South Sudan150042900
Greece197030900
Latvia132032600
Cambodia108010100
Dominican Republic65010000
Guinea1613010200
Honduras136013300
Madagascar20011000
Malta185013900
Romania110011900
African National Congress1110001000
African Union74000300
All-Africa Conference of Churches10000100
Andorra10000100
Austria1410001300
Bahrain136000700
Barbados11000000
Belarus11000000
Benin136001600
Brunei21000100
Bulgaria104002400
Burkina Faso73000400
Cameroon75000200
Central African Republic21001000
Centre for Strategic Communication and Information Security (Ukraine)33000000
Columbia University (New York)20000200
Comoros21001000
Confluence Advisory (Organization)31002000
Council of the European Union11000000
Darfur Advocacy Group (Sudan)52003000
Darfur Women Action Group74002100
Djibouti119000200
East Timor43000100
Economic Community of Central African States33000000
Equatorial Guinea95000400
Ethiopia167004500
European Commission11000000
Facilitator of the Burundi Peace Process10000010
Farida Global Organization1111000000
Fiji52000300
Finland90000900
Foundation for Human Rights (South Africa)11000000
FRELIMO11000000
Gambia11000000
George Washington University (Washington, D.C.)32000100
Georgia126000600
German Democratic Republic11000000
Grenada33000000
Guatemala1411021000
Guinea-Bissau11000000
Guyana1610003300
Haiti52001200
Holy See1620011300
IBRD11000000
Intergovernmental Authority on Development22000000
International Center for Transitional Justice10000100
International Committee of the Red Cross40000400
Iraqi Al-Amal Association11000000
Islamic Group at the United Nations11000000
Ivory Coast166003700
Jamaica147000700
Japan166003700
Kaladan Press (Organization)11000000
Kazakhstan40001300
Laos11000000
Lebanon108001100
Les Centres GHESKIO (Haiti)11000000
Lesotho11000000
Liberia42101000
Malawi21000100
Mali1913000600
Mauritius74000300
McGill University (Montréal, Canada)11000000
Medical Aid for Palestinians (Organization)21001000
Moldova10000100
Monaco70000700
Mongolia41000300
Montenegro10000100
MPLA88000000
Municipal Association of Women (Organization : Colombia)10000100
Namibia1711000600
Nepal10000100
New Development Bank10000100
NGO Working Group on Women, Peace and Security31100100
Niger103001600
Nigeria; United Nations11000000
North Korea11000000
Norwegian Refugee Council10000100
Oman1610002400
Organisation of Islamic Cooperation11000000
Organization of African Unity55000000
Organization of the Islamic Conference107001200
Oxfam International11000000
PAEMA55000000
Palau10000100
Pan Africanist Congress of Azania (South Africa)76000100
Panama1860001200
Patriotic Front of Zimbabwe22000000
POLISARIO Front44000000
Regional Center for Human Rights (Ukraine)10000100
Rossiya Segodnya (Russian Federation)11000000
Samoa10000100
Sao Tome and Principe10000100
Sauti Ya Mama Mukongomani (Organization : Democratic Republic of the Congo)11000000
secretariat of the OAU22000000
South West Africa People’s Organization11000000
Sri Lanka20000200
St. Vincent and the Grenadines83000500
State of Palestine11000000
Strategic Initiative for Women in the Horn of Africa (Organization)53001100
Sudan Family Planning Association20001100
SWAPO33000000
Swaziland11000000
Thailand20000200
Togo20001100
TRIAL International (Organization : Switzerland)33000000
Trinidad and Tobago20000200
Ukrainian Trade Union of Law Workers10000100
University of Oxford30000300
US/Middle East Project (Organization : New York)20000200
Vienna Center for Disarmament and Non-Proliferation10000100
Vietnam1310001200
VOICE Amplified (Organization)10001000
Women and Peace Studies Organization (Afghanistan)11000000
Women's Initiatives for Gender Justice10000100
Women's League of Burma22000000
Women's Peace Network (Myanmar)22000000
Yale University (New Haven, Conn.)86000200
Yemen1111000000
Yemen Arab Republic22000000
Yemen People's Republic33000000
Zambia1616000000
Zan Times (Firm : Afghanistan)10001000
Zimbabwe1410000400

Width is the share of that delegation's own eligible occurrences, not of the corpus.

Source

15_usage.py → usage/usage.json

Download

Which peoples do speakers name as the victims of a genocide, decade by decade?

Model-derived · experimental
Victim groups down the side, decades across; each cell counts the occurrences naming that group.
Group1940s1950s1960s1970s1980s1990s2000s2010s2020sAll
Palestinians4336186630471673
Tutsi99180116314
Yazidis574198
Bosniaks (Bosnian Muslims)304332895
People of Bosnia and Herzegovina5513160
Rwandans161618656
Jews18972331043
Kosovo Albanians86101741
Lebanese and Palestinians2244535
Ukrainians3434
Africans5154428
Armenians151227
Iraqis7610427
Fur, Masalit and Zaghawa (Darfur)4111126
Muslims in India1131722
Victims and survivors (unspecified)1532322292
Civilians and populations (unspecified)4772111132689
Women, children and men (unspecified)116314111046
Speaker's own people (unspecified)3951211940
Muslims (unspecified)1157

33 further peoples, named in 260 occurrences, are in the download.

Cells count occurrences whose victim group, as the AI model wrote it, falls in that group.

Source

15_usage.py → usage/usage.json

Download

How often does a speaker name a particular people as the victim, and how often no one?

Model-derived · experimental

Each bar divides a decade's eligible occurrences into four kinds; the shares sum to 100%.

Source

15_usage.py → usage/usage.json

Download

Which delegations name which peoples as the victims of a genocide?

Model-derived · experimental
Delegations down the side, peoples across; each cell counts the occurrences in which that delegation names that people.
DelegationPalestiniansTutsiYazidisBosniaks (Bosnian Muslims)People of Bosnia and HerzegovinaRwandansJewsKosovo AlbaniansLebanese and PalestiniansUkrainiansAll
Rwanda21023114234
Palestine117117
United Nations2321091667
Syria51265
Israel112957
Iran4335455
Cuba32351
(personal capacity)62212148
Pakistan131712145
United States691211144
Algeria371139
Iraq155135
Jordan2313335
Libya2934
Palestine Liberation Organization193834
Bosnia and Herzegovina1209131
Russia7133131
Mauritania2628
Tunisia261128
Armenia127

139 further delegations, with 734 occurrences, are in the download.

Cells count occurrences in which that delegation names that people, as grouped from the AI model's labels.

Source

15_usage.py → usage/usage.json

Download

Instructions and categories

These are the instructions (the prompt) and category definitions used for this model run. They document how the model was asked to classify passages. The version and content identifier let you distinguish this run from later revisions.

Show the prompt (sha256:3772c44d400f…)
# Model annotation prompt — `genocide`

version: 3
date: 2026-09-07
codebook: `annotations/lexicon/CODEBOOK.md`, human schema version 3; model schema 3.1

Version 3 removes model self-reported confidence. Human confidence is unchanged.
Situation/modality coding remains a separate reviewed schema-4 proposal; it is
not introduced by this revision. Prompt v2 is archived byte-for-byte in
`prompts/v2.md`, so the Bayreuth run prepared with it remains readable.

The raw bytes of this file are hashed with SHA-256 into every run manifest and into every
row of every `annotations.jsonl`. Editing so much as a comma produces a different hash, and
a row can then no longer be matched to the text that produced it — so an edit here is a new
prompt version. Bump `version:` above, **move the old text to `prompts/v<n>.md` unchanged**,
and give the run that uses it a new run id. The archive is what lets an edit here be made at
all: `15_usage.py` resolves a run against every prompt this repository holds, so revising
the instrument costs a new run id and not the runs already paid for. The repository stores
and checks out this file with LF line endings (`.gitattributes`), so the hash does not depend
on the machine that computed it.

Two sections below are read by `scripts/lib/llm.py`, and only those two: the fenced block
under `## System` is the system message, verbatim; the fenced block under `## User template`
is the per-speech user message, with its `{placeholders}` filled in at run time. Everything
outside the two fences — this paragraph included — is documentation for a human and never
reaches the model.

The system block carries one placeholder, `{referents_table}`, rendered from
`annotations/lexicon/referents.csv` at run time as `id — label (years) — description`,
grouped by kind. The controlled list grows through reviewed additions to that file; the
prompt does not restate it, so adding a referent does not silently become a prompt edit.
What *would* change the meaning of a label — the field definitions, the boundary examples,
the cascade — lives here and is versioned here.

## What v2 changes, and on what evidence

v2 is the review of 1 September 2026, §4.2, carried out. Six changes:

1. **`stance` becomes two fields.** `speaker_position` records what the speaker does with the
   characterisation; `concrete_case` records whether the word is applied to a determinate
   case at all. v1's `neutral_legal_reference` was doing both jobs and competing with the
   `institutional_title_or_mandate` function and with three meta referents for the same
   decision — the largest referent confusion cells were exactly those pairs. Now the decision
   is made once and the other fields follow from it.
2. **`reports_without_position` is defined positively**, and a settled legal status is an
   assertion. v1's `attributes_or_reports` scored F1 0.37 across the two instruments, with
   445 occurrences read as *attributes* by one and *asserts* by the other — the single
   largest disagreement cell in the corpus.
3. **Ten worked examples, every one of them from the corpus**, quoted verbatim with the line
   id it was taken from, so a reader can go and check it. v1's examples were all invented,
   and the corpus's own recurrent formulae were absent from it.
4. **New fields the study's question needs**: `accused_actor`, `victim_group`,
   `own_state_accused`, `salience`, `referent_source` and a one-sentence `rationale`.
5. **Two rules the year ranges and the compound cases needed**, taken from the codebook so
   that a coder and a model read the same sentence.
6. **A false positive still needs a real quote.** v1's cascade said "all four fields, with no
   exception", and one model answered the *evidence* field with the literal string
   `not_applicable` three times.

Three of the new rules are stated as what the two committed runs already do, rather than as
a correction: both read the Special Adviser's title as a non-case mention in 176 of 176
occurrences, both code 98% of commemorations as assertions, and neither was ever told to.
Writing those down does not move them; it makes them testable, and it stops the next
instrument having to rediscover them.

The `distancing` rule is the opposite case. The grammatical-frames codebook
(`scripts/lib/node_frames.py`) finds 78 occurrences in which the label is marked as somebody
else's — *so-called*, *supposed*, scare quotes, *what she called "genocide"* — and the two
instruments part on them: 33 of Luna's are `attributes_or_reports`, 27 of Gemini's are
`rejects_or_denies`. That is §4.1's report/assert disagreement localised in one construction,
so one rule and one example can settle it.

**A provenance note about the two artefacts this prompt was written from.** The frames
artefact's `matched` column and the referent list's absorption table are both *projections*
over the two committed runs' text — a regex over a window, and a clustering of
`proposed_referent` strings. Neither is a re-coding, nothing in either committed run was
rewritten to produce them, and both are counted here as evidence of where the instruments
disagree and not as evidence of what the right label is. Only a run settles that, and no run
has yet been made against this prompt.

## System

```text
You are annotating occurrences of the word "genocide" in the verbatim records of the United
Nations Security Council, 1946-2024.

TASK BOUNDARY. You classify discourse. You never decide whether an underlying event legally
constitutes genocide, and no label you assign is a finding about what happened. The question
is always what the pattern found and what the speaker is doing with the word in this passage:
whether the matched string is really an instance of the expression and, where it is, how the
expression is being used here.

Read the whole speech before labelling anything. Then label each numbered occurrence
separately, from the passage around it, using the rest of the speech only to resolve what
that passage leaves open. Abstention is a correct answer, and a wrong label is worse than an
honest one: prefer "uncertain", "unclear" to a guess.

FIELDS

verdict — exactly one:
  true_positive   The highlighted span is a real instance of the expression. Example: "acts
                  of genocide".
  false_positive  The string does not instantiate the expression. Example: damaged OCR joins
                  unrelated fragments into something a tolerant pattern matched.
  uncertain       The transcription does not permit a defensible decision. Example: OCR
                  damage leaves the passage illegible.

quotation — exactly one:
  not_quoted              The current speaker uses the expression in their own formulation.
                          "We consider these acts genocide."
  direct_quotation        The expression occurs inside words explicitly quoted verbatim.
                          "The witness called it 'genocide'."
  attributed_or_reported  The speaker reports or paraphrases another actor's terminology.
                          "The Commission described the acts as genocide."
  unclear                 Attribution cannot be resolved from the available passage.
  not_applicable          False positives only.

  Quotation does not determine position. Naming the source of a characterization says where
  it came from; it says nothing about whether this speaker adopts it. A speaker may quote a
  claim in order to endorse it, in order to reject it, or in order to leave it standing
  unjudged, and those are three different values of speaker_position with the same value
  here.

concrete_case — exactly one. Answer this before speaker_position:
  yes             The word is applied, in this passage, to a determinate situation, event,
                  people or place: a case that could be named. It does not have to be named
                  in the passage — see referent_source — only determinate.
  no              The word is not applied to any case here. It names the crime as a legal
                  category, appears inside the title or mandate of an office, a treaty or a
                  tribunal, or stands in a general formula about prevention or atrocity.
  unclear         The passage does not let you tell which of the two it is.
  not_applicable  False positives only.

  This is the decision v1 made three times over and made differently each time. Make it once,
  here. Everything below follows from it: when concrete_case is "no", speaker_position is
  "no_position" and the referent is one of the three non-case referents; when it is "yes",
  speaker_position is one of the four real positions and the referent is a case.

  What matters is what the MATCHED WORD is doing, not what the sentence is about. "This
  crime is forbidden by the 1948 Convention on the Prevention and Punishment of the Crime of
  Genocide" is a sentence about a concrete crime in which the matched word occurs only
  inside a treaty's title: concrete_case is "no".

speaker_position — exactly one:
  asserts                   The speaker advances or endorses the characterization, or treats
                            it as established. "This is genocide." Also: a legal status the
                            speaker treats as settled — "those convicted of genocide by the
                            ICTR", "known genocidaires", "the ICTY decision finding him
                            guilty of genocide" — because a speaker who reasons from a
                            finding has adopted it.
  rejects                   The speaker disputes, denies or refuses the characterization.
                            "There was no genocide." Also: the speaker marks the label as
                            false while reporting it — see the distancing rule below.
  conditional               The speaker presents it as a possibility, a risk, a threshold or
                            a warning rather than as an established characterization.
                            "Unless prevented, this could become genocide."
  reports_without_position  Use this ONLY when both of these hold: the passage names the
                            actor doing the characterizing, AND the speaker neither adopts
                            nor rejects it. Both halves are required. If the characterizing
                            actor is not named, the speaker is speaking in their own voice,
                            which is one of the three above. If the speaker endorses what
                            they report, that is "asserts"; if they disown it, "rejects".
                            The value means "somebody said this and I am not saying whether
                            they were right", and nothing else.
  no_position               concrete_case is "no", so there is no characterization of a case
                            to take a position on. Use this value exactly when concrete_case
                            is "no", and never otherwise.
  unclear                   The evidence supports no reliable choice.
  not_applicable            False positives only.

  THE DISTANCING RULE. Ask what the hedge is scoping over.
  - Over the LABEL — scare quotes around the word, "so-called", "supposed", "purported",
    "what X calls genocide", "allegations of genocide" — the speaker is holding the
    characterization at arm's length as somebody else's and declining it: "rejects". Not
    "reports_without_position": marking a label as false is a position.
  - Over a PERSON OR AN ACT, with the category itself untouched — "alleged genocide
    financier", "those alleged to have committed genocide", "accused of genocide" — that is
    an unproven charge against an individual, made inside a category the speaker treats as
    real: "asserts". This is the ordinary language of an indictment, and it is not
    distancing at all.

function — one or more, as a JSON array of strings:
  accusation_or_qualification     Names or legally characterizes conduct.
  warning_or_prevention           Warns of risk, or urges prevention or protection.
  commemoration                   Remembers victims, anniversaries or historical events.
  accountability                  Concerns investigation, prosecution, punishment or
                                  impunity. A proceeding, an arrest, a sentence, a fugitive,
                                  an amnesty. NOT merely because a court or tribunal is
                                  named: "genocide denial by ICTR defence lawyers" is an
                                  accusation against the deniers and concerns no proceeding.
  institutional_title_or_mandate  Occurs in the name or mandate of an office, treaty,
                                  tribunal, adviser or institutional mechanism. This is a
                                  restatement of concrete_case "no", not a separate finding:
                                  a word inside a title is not applied to a case.
  other                           A clear function not represented above.
  unclear                         The function cannot be determined.
  not_applicable                  False positives only.

  "Prosecute those responsible for genocide so that it never happens again" may be
  ["accountability", "warning_or_prevention"]. Never repeat a label. "unclear" and
  "not_applicable" cannot be combined with any other label, so an array containing either of
  them has exactly one element. Prefer the smallest defensible set; do not infer purposes
  that are absent from the text.

  COMMEMORATION IS ASSERTION. A commemorative formula — an anniversary, a remembrance, "we
  honour the victims of the genocide" — treats the characterization as established, so its
  speaker_position is "asserts" even though nothing is being alleged against anyone in the
  passage. Add "accusation_or_qualification" only when the passage also characterizes
  someone's conduct.

referent — exactly one identifier from the controlled list below.

referent_source — exactly one. Where in the request you found the referent:
  passage  The passage around the occurrence names or unambiguously identifies it.
  speech   Another part of the speech does, and this passage does not.
  header   Only the header lines above the speech do — the speaker's country or
           organisation, the agenda item, the meeting.
  not_applicable  False positives only; and the value for referent "unclear".

  This field is a measurement of the instrument and not of the passage, so record what you
  actually used. A referent read off the header alone is a weaker assignment than one read
  off the passage, and the study needs to know how many there are.

accused_actor — free text, a few words. Who the passage says did it, or is charged with it,
  as the passage names them: "ex-FAR and Interahamwe", "Radislav Krstic", "the Government of
  the Sudan". Empty when the passage accuses no one.

victim_group — free text, a few words. The group the word is applied on behalf of, as the
  passage names them: "the Tutsi in Rwanda", "the Bosnian Muslims of Srebrenica". Empty when
  the passage names none.

  Both fields are copied from the passage, never inferred from the case. If the passage says
  "genocide in Rwanda" and names neither perpetrator nor victims, both are empty. That is
  data, not a failure.

own_state_accused — exactly one:
  yes             The speaker's own State or organisation is the one the passage accuses, or
                  the one it would accuse if the characterization stood. A denial by the
                  accused State is "yes": the accusation is what is being denied.
  no              Somebody else is accused.
  not_applicable  The passage accuses no one, this is a non-case mention, or the verdict is
                  false_positive.

salience — exactly one:
  passing         The word appears in a list, a formula or an aside, and the passage does not
                  argue about it. The four-crime responsibility-to-protect formula and the
                  triad "genocide, war crimes and crimes against humanity" are passing.
  substantive     The characterization is what the passage is doing: it is argued for,
                  argued against, dwelt on, or the subject of the sentence.
  not_applicable  False positives only.

rationale — one sentence, free text, always present. Why the position and the case decision
  are what they are, in your own words. Not a restatement of the labels: say what in the
  passage decided them. Where you hesitated between two values, name the other one. This is
  read by humans adjudicating disagreements and costs almost nothing to write.


THE FALSE-POSITIVE CASCADE

If verdict is "false_positive", then quotation, concrete_case, speaker_position, referent,
referent_source, own_state_accused, salience and every element of function are
"not_applicable"; accused_actor, victim_group and proposed_referent are empty strings. If
verdict is anything else, "not_applicable" may not appear in quotation, concrete_case,
speaker_position, function or referent: there it is reserved for false positives and means
nothing else. referent_source, own_state_accused and salience are the exceptions, because
each of them has a genuine "there is nothing here to record" answer that is not about the
verdict at all.

The cascade stops at the evidence. A false positive still needs a real, verbatim
evidence_quote and a real rationale: the claim "this match is not the word being used" is a
claim about a passage, and it is unreadable without the passage. Do not write the string
"not_applicable" into evidence_quote. A false positive whose quote cannot be found in the
speech is refused when the row is written.

REFERENT

Choose the referent the occurrence is ABOUT: the case, event, people or institution the word
is applied to in this passage. It is not every situation the speech mentions, and it is not
the speaker's own country unless the passage applies the word to it.

{referents_table}

The years in parentheses are DOCUMENTATION, NOT A CONSTRAINT. They say when the case is
usually invoked; they do not say which speeches may carry it. Where a passage names a period
a referent's years do not cover, code the referent anyway. A 2009 speech about Gaza takes the
Israel/Palestine referent because that is what the speech is about, not because 2009 falls in
a range.

A passage naming two cases is coded as the first one named, and the pair goes into
"proposed_referent" as the speaker gave it. "Rwanda and Srebrenica" is the Rwanda referent
with proposed_referent "Rwanda and Srebrenica". Where a passage names a case and something
more specific inside it — Khojaly inside Nagorno-Karabakh, the Yazidis inside ISIL's crimes —
that is not a compound: code the more specific identifier, and leave proposed_referent empty.

When concrete_case is "no", choose among the three non-case referents in this order, first
match winning:

  institutional title or mandate   The word is inside the name or mandate of an office, a
                                   body or a tribunal. "Special Adviser on the Prevention of
                                   Genocide".
  Convention and legal definition  The passage is about the 1948 Convention, the legal
                                   definition of the crime, or jurisdiction over it. "the
                                   Convention on the Prevention and Punishment of the Crime
                                   of Genocide", "the definition in article II".
  genocide in general              Everything else abstract: prevention doctrine, the
                                   four-crime formula, a general statement about the crime.

Using the three reserved identifiers:

  other           The referent is identifiable but is not on the list. Use it, and put a
                  short name for what you identified in "proposed_referent" — a place, a
                  conflict, a people or an institution, a few words at most. Never invent an
                  identifier; the list grows only through human review.
  unclear         The speech does not let you tell which referent is meant. Choosing this is
                  correct whenever it is true. Never guess.
  not_applicable  False positives only.

"proposed_referent" is a short free-text note about the referent. It is required when the
referent is exactly "other", it carries the pair when a compound is coded as the first case
named, it is empty on a false positive, and it is otherwise empty.

EVIDENCE

"evidence_quote" is the shortest verbatim passage of the speech that both contains the
matched word of this occurrence and supports every label you assigned to it. Usually a
clause or one sentence; two sentences when the attribution or the condition begins in the
one before. Never the whole speech.

The minimum is the clause containing the modal or the attribution — whatever it is that
makes your position label what it is. "could amount to genocide" rather than "genocide",
because the modal is what makes the position conditional; "what she called 'genocide'"
rather than "genocide", because the attribution is the whole of the reason. A bare "have been
convicted of genocide" cannot support a position label, and a quote too short to support one
is treated as unlocated evidence.

Copy it character for character out of the speech text supplied with the request: the same
spelling, punctuation, capitalisation and spacing, including any OCR damage. The records are
scanned, and they break words across lines: "the Secretary- General's warning" is what the
record says, and it is what you copy. No ellipses, no corrections, no added quotation marks,
no markup, nothing joined across a gap. The quote is located back in the speech
automatically, and a quote that cannot be found there is recorded as invalid evidence.

WORKED EXAMPLES

Ten occurrences from this corpus, quoted verbatim, with the line id each was taken from. They
are the cases the previous version of this prompt got least agreement on. Fields not shown
take the values the definitions above give them.

1. Commemoration. UNSC_2014_SPV.7105_spch0017#3, Rwanda, 29 January 2014.
   evidence_quote: "As we commemorate the twentieth anniversary of the 1994 genocide against
   the Tutsi in Rwanda"
   concrete_case yes | speaker_position asserts | quotation not_quoted |
   function ["commemoration"] | referent: Rwanda | referent_source passage |
   accused_actor "" | victim_group "the Tutsi in Rwanda" | own_state_accused not_applicable |
   salience substantive
   rationale: "Commemorating an anniversary treats the characterization as established, so
   the speaker asserts it even though the passage accuses no one."

2. The atrocity triad. UNSC_2005_SPV.5264_spch0022#1, European Union, 20 September 2005.
   evidence_quote: "our responsibility to protect populations from genocide, war crimes,
   ethnic cleansing and crimes against humanity"
   concrete_case no | speaker_position no_position | quotation not_quoted |
   function ["warning_or_prevention"] | referent: genocide in general |
   referent_source passage | accused_actor "" | victim_group "" |
   own_state_accused not_applicable | salience passing
   rationale: "The word names one of the four crimes in the responsibility-to-protect
   formula and is applied to no case, so there is no characterization to take a position on."

3. Perpetrator noun. UNSC_1999_SPV.3987_spch0015#2, United States, 19 March 1999, on the
   Democratic Republic of the Congo.
   evidence_quote: "to collaborate militarily with ex-FAR and interahamwe, known genocidaires"
   concrete_case yes | speaker_position asserts | quotation not_quoted |
   function ["accusation_or_qualification"] | referent: Rwanda | referent_source passage |
   accused_actor "ex-FAR and interahamwe" | victim_group "" | own_state_accused no |
   salience passing
   rationale: "The perpetrator noun applies the word to the killings in Rwanda and not to the
   Congolese war the debate is about, and 'known' treats the characterization as settled."

4. Denial and ideology. UNSC_2011_SPV.6545_spch0010#9, Rwanda, 6 June 2011.
   evidence_quote: "the ongoing scourge of genocide denial by some in the academic and legal
   professions"
   concrete_case yes | speaker_position asserts | quotation not_quoted |
   function ["accusation_or_qualification"] | referent: Rwanda | referent_source speech |
   accused_actor "academics and lawyers who deny it" | victim_group "" |
   own_state_accused no | salience substantive
   rationale: "Condemning denial asserts the genocide that is denied; the accusation is
   against the deniers, and no proceeding is at issue, so accountability is not added."

5. Own-state denial. UNSC_2017_SPV.7963_spch0019#6, Sudan, 8 June 2017.
   evidence_quote: "the report of the International Commission of Inquiry on Darfur, which
   was submitted to the Council in early 2005, confirming that no genocide had taken place in
   Darfur"
   concrete_case yes | speaker_position rejects | quotation attributed_or_reported |
   function ["accusation_or_qualification"] | referent: Darfur | referent_source passage |
   accused_actor "the Government of the Sudan" | victim_group "" | own_state_accused yes |
   salience substantive
   rationale: "The speaker reports a finding and adopts it — 'confirming' endorses the
   conclusion — so this refuses the characterization rather than reporting it without a
   position."

6. A tribunal's finding. UNSC_2001_SPV.4379Resumption1_spch0003#1, Jamaica,
   21 September 2001.
   evidence_quote: "the recent ICTY decision finding Radislav Krstic guilty of genocide in
   the Srebrenica massacre"
   concrete_case yes | speaker_position asserts | quotation attributed_or_reported |
   function ["accountability", "accusation_or_qualification"] | referent: Bosnia and
   Srebrenica | referent_source passage | accused_actor "Radislav Krstic" |
   victim_group "" | own_state_accused no | salience passing
   rationale: "A conviction is a legal status the speaker treats as settled, and naming the
   court that reached it attributes the source rather than distancing the speaker from it; I
   considered reports_without_position and rejected it because nothing here suspends
   judgement."

7. The Convention. UNSC_1992_SPV.3095_spch0029#4, Pan Africanist Congress of Azania,
   15 July 1992.
   evidence_quote: "This crime is forbidden by the 1948 Convention on the Prevention and
   Punishment of the Crime of Genocide"
   concrete_case no | speaker_position no_position | quotation not_quoted |
   function ["institutional_title_or_mandate"] | referent: Genocide Convention and legal
   definition | referent_source passage | accused_actor "" | victim_group "" |
   own_state_accused not_applicable | salience passing
   rationale: "The matched word occurs only inside the treaty's title, so it is applied to no
   case even though the sentence around it is about one."

8. A warning. UNSC_1996_SPV.3692_spch0018#1, Japan, 28 August 1996.
   evidence_quote: "the Secretary- General's warning that if the worst-case scenario becomes
   a reality there could be a genocide in Burundi"
   concrete_case yes | speaker_position conditional | quotation attributed_or_reported |
   function ["warning_or_prevention"] | referent: Burundi | referent_source passage |
   accused_actor "" | victim_group "" | own_state_accused not_applicable |
   salience substantive
   rationale: "The modal makes the characterization prospective, and the case is Burundi: a
   warning is about a place, and a genocide that has not happened is not a referent."
   Note the hyphen and space inside "Secretary- General's": that is what the record says, so
   that is what the quote says.

9. Distancing. UNSC_2023_SPV.9524_spch0007#1, United Kingdom, 30 December 2023.
   evidence_quote: "President Putin claimed his invasion was to stop a supposed genocide in
   Donbas"
   concrete_case yes | speaker_position rejects | quotation attributed_or_reported |
   function ["accusation_or_qualification"] | referent: Ukraine | referent_source passage |
   accused_actor "Ukraine, in the claim being reported" | victim_group "" |
   own_state_accused no | salience passing
   rationale: "'Supposed' is the reporting speaker's own mark on the label and not part of
   what was claimed, so the speaker refuses the characterization rather than leaving it
   standing."

10. An office's name. UNSC_2004_SPV.4990_spch0011#1, France, 14 June 2004.
    evidence_quote: "the Secretary-General's initiative of establishing the post of Special
    Adviser on the Prevention of Genocide"
    concrete_case no | speaker_position no_position | quotation not_quoted |
    function ["institutional_title_or_mandate"] | referent: institutional title or mandate |
    referent_source passage | accused_actor "" | victim_group "" |
    own_state_accused not_applicable | salience passing
    rationale: "The word is part of an office's name, so it names a mandate and not a case."

OUTPUT

Return JSON only, matching the schema supplied with the request. No prose, no commentary, no
explanation outside the fields. Exactly one entry per numbered occurrence: the set of
"ordinal" values you return must equal the set of numbers in the occurrence list, each
appearing once.
```

## User template

```text
Speech: {filename}
Date: {date}
Speaker (country or organisation): {country_org}
Participant type: {participant_type}
Meeting: {meeting_symbol}
Agenda item: {agenda_item}

===== SPEECH TEXT BEGINS =====
{speech}
===== SPEECH TEXT ENDS =====

The speech text above is unmodified, and the character offsets below are zero-based indices
into it, with the end exclusive. Nothing in it is marked up, so an evidence quote copied out
of it is a quote of the record.

The header lines above are context, not evidence. A referent you take from them and from
nowhere else is recorded as referent_source "header", and the speaker's own country is not
the referent unless the passage applies the word to it.

Occurrences to annotate ({occurrence_count}), each with the sentence it falls in:

{occurrences}
```
Show the controlled referent list (version 2, 40 identifiers)
IdentifierDefinitionOccurrences
Rwanda
rwanda
Speeches invoking the mass killing of Tutsi in Rwanda and its aftermath as genocide or genocidal.1,978
Genocide in general
genocide_in_general
Abstract discussion of genocide, including prevention doctrine, with no single case in view.1,743
Gaza
gaza
Speeches invoking the Gaza war as genocide or genocidal, including the proceedings South Africa brought against Israel at the ICJ.835
Bosnia and Srebrenica
bosnia_srebrenica
Speeches invoking the war in Bosnia and Herzegovina, including the fall of Srebrenica and its commemorations, as genocide or genocidal.769
Genocide Convention and legal definition
genocide_convention_law
The 1948 Convention, the legal definition of the crime, and jurisdiction over it before the ICJ or the ICC.553
Institutional title or mandate
institutional_mandate
The name or mandate of an office or body, such as the Special Adviser on the Prevention of Genocide or a tribunal's title.483
Other known referent
other
A referent that is identifiable but does not yet have a controlled identifier.443
Israel and Palestine
israel_palestine
Speeches invoking Israeli conduct towards Palestinians, or the conduct of Palestinian and allied armed groups towards Israelis, as genocide or genocidal, up to the war that gaza covers.341
Darfur
darfur
Speeches invoking the conflict in Darfur as genocide or genocidal.220
Kosovo
kosovo
Speeches invoking the Kosovo conflict and the expulsion of its Albanian population as genocide or genocidal.128
ISIL and the Yazidis
isil_yazidi
Speeches invoking ISIL's campaign against the Yazidi population of northern Iraq as genocide or genocidal.126
Democratic Republic of the Congo
drc
Speeches invoking mass killing in the Democratic Republic of the Congo as genocide or genocidal, including the presence in its east of armed groups formed from the perpetrators of the Rwandan killings.105
Ukraine
ukraine
Speeches invoking genocide in connection with the war in Ukraine, in either direction: against Russia's conduct and in Russia's own claims about Donbas. Not the Soviet famine, which holodomor covers, and not the Crimean Tatars, who have their own identifier.102
South Africa under apartheid
apartheid_south_africa
Speeches invoking apartheid and the violence surrounding its end as genocide or genocidal.84
ISIL and other populations of Iraq and Syria
isil_iraq_syria
Speeches invoking ISIL's crimes against populations of Iraq and Syria other than the Yazidis as genocide or genocidal, such as Christians, Shia Muslims and Turkmen.77
Burundi
burundi
Speeches invoking the mass violence in Burundi as genocide or genocidal.70
Colonial-era mass violence
colonial_era
Speeches invoking mass violence under colonial rule as genocide or genocidal, such as the campaign against the Herero and Nama.63
The Holocaust
holocaust
Speeches invoking the Nazi extermination of European Jews and its commemoration as genocide or genocidal.60
Croatia and the Yugoslav wars
croatia_yugoslav_wars
Speeches invoking the Yugoslav wars outside Bosnia and Kosovo as genocide or genocidal, such as the fighting in Croatia and the Krajina, or the wars taken as a whole.53
Muslims in India
india_muslims
Speeches invoking the treatment of the Muslim population of India as genocide or genocidal.50
South Sudan
south_sudan
Speeches invoking the civil war in South Sudan as genocide or genocidal.33
Cambodia under the Khmer Rouge
cambodia_khmer_rouge
Speeches invoking the mass killing under the Khmer Rouge, including the work of the Extraordinary Chambers, as genocide or genocidal.31
Myanmar and the Rohingya
myanmar_rohingya
Speeches invoking the treatment of the Rohingya population of Myanmar as genocide or genocidal.31
Bangladesh and the war of separation
bangladesh
Speeches invoking the mass killing during the war that separated East Pakistan from Pakistan as genocide or genocidal.30
The Ottoman Armenians
armenian_genocide
Speeches invoking the deportation and mass killing of Ottoman Armenians as genocide or genocidal.26
Nagorno-Karabakh
nagorno_karabakh
Speeches invoking the Nagorno-Karabakh conflict and the departure of its Armenian population as genocide or genocidal.23
Syria
syria
Speeches invoking the conduct of the parties to the war in Syria as genocide or genocidal.18
Central African Republic
central_african_republic
Speeches invoking the sectarian violence in the Central African Republic as genocide or genocidal.17
Other sanctions regimes
economic_sanctions_other
Speeches describing a sanctions regime other than the Cuban embargo as genocidal, such as the measures against Iraq in the 1990s.17
Afghanistan and the Hazaras
afghanistan_hazara
Speeches invoking mass violence in Afghanistan as genocide or genocidal, in particular attacks on its Hazara population.13
Abkhazia and South Ossetia
abkhazia_south_ossetia
Speeches invoking the conflicts in Abkhazia and South Ossetia and the displacement of their populations as genocide or genocidal.11
East Timor
east_timor
Speeches invoking the violence around East Timor's independence ballot as genocide or genocidal.8
Crimean Tatars
crimean_tatars
Speeches invoking the wartime deportation of the Crimean Tatars, or their treatment under the occupation of Crimea, as genocide or genocidal.6
Iraqi Kurds and Anfal
iraq_kurds_anfal
Speeches invoking the Anfal campaign and the Halabja attack against Iraq's Kurdish population as genocide or genocidal.5
Khojaly
khojaly
Speeches invoking the killings at Khojaly as genocide or genocidal, which Azerbaijan raises as an event of its own.5
The Soviet famine in Ukraine
holodomor
Speeches invoking the Soviet famine in Ukraine as genocide or genocidal, which speakers raise as a case of its own rather than as part of the war that ukraine covers.4
United States embargo against Cuba
cuba_us_sanctions
Speeches describing the United States embargo against Cuba as genocidal.3
Xinjiang and the Uyghurs
xinjiang_uyghurs
Speeches invoking China's treatment of the Uyghur population of Xinjiang as genocide or genocidal.1
Not applicable
not_applicable
Reserved for false-positive candidate matches.0
Unclear referent
unclear
The available context does not support a reliable referent assignment.0

Comparison with human coding

No human-reference comparisons are available yet. The sample contains 469 entries covering 458 distinct occurrences. Two readers are to code each independently. Accuracy scores require agreed or adjudicated reference labels; the absence of a score means accuracy has not been measured.

Top