A new dataset aims to compile and make available for analysis every national security and defence policy document ever published anywhere in the world. The dataset’s creator, Andrew W. Neal, explains how the collection reveals trends in public security discourse and threat perceptions, and how the dataset can be accessed and analysed by scholars.
Over the past few years, our team at the University of Edinburgh has assembled what may be the largest database of national security and defence documents ever created. With 575 documents from 113 countries, the patterns emerging from this data offer new insights into how states articulate their security priorities.
The story begins in 1987, when the US published the first public national security strategy. Before then, security policy was largely a ‘black box’. In the UK, MPs and Peers weren’t even allowed to mention ‘national security’ in Parliament. The intelligence services didn’t officially exist. The progressive opening of security policy since then reveals significant changes in how states communicate about security. This phenomenon of document publication reflects a broader liberalization of security policy, where previously secretive domains have become subject to public scrutiny and debate.
Building this database required systematic work. My student assistants and I searched in multiple languages – even Georgian, which doesn’t use the Latin alphabet. We began with the ISO standard country list and worked methodically through it. We had some help from existing lists and generous scholars, but much was detective work. We found documents through official government websites, academic repositories, and elsewhere. While we’re confident we’ve found most publicly available documents, there could be more to discover. If you come across something we’ve missed, I’d welcome hearing about it.
Trends and typology in security policy
The data reveals a clear shift away from traditional military threats over time. Around 70% of identified threats are now ‘non-traditional’ – for example, cyber-attacks, climate change, organized crime. This raises important questions about how security is understood in different contexts. Using Stephen Walt’s 1991 definition of security studies as concerning “the threat, use and control of military force”, we can see how far the field has evolved. Back then, the topics were mainly military. Now the picture is far more complex, with threats ranging from environmental degradation to economic instability, from pandemics to food security.
One of the most significant patterns concerns the difference between rich and poor countries. The UK and US identify the greatest number of threats, and most Global North countries identify significantly more threats than those in the Global South. This could reflect civil service capacity, international engagement, or perhaps that poorer countries focus on fewer, more immediate concerns. When we correlate threat identification with economic data, we find striking patterns. Upper-middle-income countries like China tend to identify more threats than least developed countries, but still fewer than advanced economies. These patterns warrant deeper investigation.
The unexpected findings are particularly revealing. Despite being one of the most researched topics in critical security studies, and a major political issue across the world, migration barely appears in these documents. This tells us something important about the relationship between academic research priorities and official security discourse. It raises questions about how our political and ethical concerns as researchers shape research agendas. Similarly, corruption appears more frequently than migration, despite receiving far less scholarly attention in security studies.
Technical considerations
The database costs have been modest – less than £10,000 in total. It’s permanently available through the University of Edinburgh, with cloud computing funding secured for the next four years. We provide both original language versions and machine translations, along with around 50 different variables for documents and countries. This allows for various forms of statistical analysis alongside traditional qualitative research. The variables include everything from basic country data to alliance membership, regime type, and economic indicators.
The technical architecture is accessible with a bit of technical know-how or curiosity. We use Google semantic search technology that has some underlying similarities with Google Translate, finding similar meanings rather than just exact words. When we search for “climate change is a threat”, we might find “Samoa is highly vulnerable to natural disasters and climate change” – different words, same concept. By adjusting similarity thresholds, we can capture nuanced variations in how threats are articulated. This semantic approach allows us to identify threat framings even when they don’t use explicit threat language.
We’re now moving everything to a cloud computing platform to facilitate collaboration. While knowledge of Python (a programming language) can be helpful, you don’t need to be a coding expert to use this data. The raw information is available in spreadsheets, and AI tools can help analyse patterns. For example, you could examine how specific threats evolve over time, or investigate regional patterns in threat identification.
Security policy learning
What makes this particularly valuable is what it tells us about public security discourse. While we can’t know what state security apparatuses really think without opening the black box – many countries have classified versions of these documents – how they articulate their security concerns publicly matters for policy and international relations. The documents shape national agendas and influence policy learning between states. We can trace how new security concerns emerge and diffuse across different countries and regions.
Another interesting finding concerns the language of threat itself. While securitization theory emphasizes existential threats, this terminology rarely appears in the documents. Instead, we find qualifications like ‘serious’, ‘major’, or ‘fundamental’. The meaning of these gradations isn’t always clear, but they reveal something about how diplomatic language works both externally and internally within governments. We’ve also found interesting patterns in how threats are characterized as ’emerging’, ‘growing’, or ‘immediate’, suggesting different temporal frameworks for understanding security challenges.
The database shows interesting regional patterns as well. NATO members tend to identify similar threats, suggesting policy coordination or shared threat perceptions. Small states often emphasize different threats than their larger neighbours. Some countries, like Costa Rica, which has no military, don’t identify security threats at all. These variations offer rich opportunities for investigating how state characteristics influence security discourse.
Using the dataset
I’m interested to see how other researchers might use this resource. Whether you’re examining how threats evolve over time, how different regions approach security, or how economic status affects security priorities, there’s substantial data here to explore. You might investigate patterns in cooperative security language, analyse how referent objects change over time, or examine how different types of states frame particular threats.
If you’re interested in using the database for your own research, consider getting in touch. I can run simple queries in the dataset or discuss potential collaborations. While we all manage competing demands, the collaborative projects that have emerged from this work have been intellectually rewarding.
Understanding how states communicate about security isn’t just an academic exercise – it shapes how we collectively think about and respond to contemporary challenges.
Blog based on an online seminar with Andrew Neal hosted by the BISA Security Policy and Practice Working Group on 12 November 2024.
Professor Andrew W. Neal is Chair of International Security at the University of Edinburgh.
Citation
Neal, Andrew W; Gardner, Roy B. (2024). National Security and Defence Documents Dataset (1987-2024) [dataset]. University of Edinburgh. School of Social and Political Science. Politics and International Relations. https://doi.org/10.7488/ds/7797
Funders:
The Scottish Council on Global Affairs (SCGA) Project, ‘A comprehensive national security corpus and dataset’, funded as an Insight Award.
BA/Leverhulme Small Research Grants, ‘A history of threats: mapping changing security issues and their conceptualisations in national security documents globally’, SRG23\231686.
Foreign Commonwealth and Development Office project: ‘Non-Western dynamics of peace and transition management’ FCDO Project Number: 300708-144, work package on ‘National Security Strategies, emergent powers and ‘Sustaining Peace”.
School of Social and Political Science Research Adaptation Fund, financed by the Scottish Funding Council.
The views and opinions expressed in posts on the Rethinking Security blog are those of the authors and do not necessarily reflect the position of the network and its broader membership.
Image Credit: National Security and Defence Documents Dataset (1987-2024) logo superimposed on covers of the US National Security Strategy, 1987 and UK Integrated Review Refresh, 2023.
