Secret Files from World War to Cold War: British Government and Secret Intelligence and Foreign Policy files

Secret intelligence, to borrow the often used cliché by Sir Alexander Cadogan, has been regarded as the ‘missing dimension’ of Britain’s diplomatic and political history. This phrase certainly describes the near absence of the subject from academia even into the 1990s when the first batches of intelligence-related material made it into the public domain. The picture is entirely different today with the ‘missing dimension’ getting ever smaller, as evidenced by this digital collection of papers in Secret Files from World War to Cold War: British Government and Secret Intelligence and Foreign Policy files, sourced from the UK National Archives (TNA) and published by Routledge. The collection covers the work of Britain’s intelligence and security agencies – the Secret Intelligence Service (SIS), the Security Service (MI5), Government Communications Headquarters (GCHQ) and their predecessors – during the First and Second World Wars, and into the early Cold War, with 4,500 downloadable files. The online collection also includes the papers of the Joint Intelligence Committee (JIC), since 1936 providing intelligence assessments to the armed forces and Britain’s policymakers, that range widely over the themes of defense, diplomacy, security and, of course, the development of Britain’s ‘secret state’ over the first half of the 20th century.

At the heart of the collection are the files of the Permanent Under-Secretary’s Department (PUSD), Cabinet Secretary’s Miscellaneous Files (CAB 301) and Churchill’s famous ‘golden eggs’ of intelligence passed to him from Britain’s codebreakers at Bletchley Park and Berkeley Street (HW1), as well as the voluminous papers of the JIC. These provide a rich tapestry of stories and detail on the inner workings of Britain’s spies up to 1953. While many would be drawn to the well publicised bi-annual releases of MI5 papers to TNA, the files in the digital collection provide an insight to the intelligence community as a whole. Importantly, the Cabinet Office and Foreign Office files show the interaction between policymakers and intelligence officials, providing a rare glimpse into the impact of Britain’s spy agencies at the top.

Perhaps the more important files here are those of PUSD, the link between the Foreign Office and intelligence agencies, the bulk of which appeared to researchers at TNA in March 2005 and May 2013. While SIS’s records remain off limits to all but a select few, PUSD’s files – and those in the digital collection – allow students and researchers to look at use of the ‘Secret Vote’ – an annual fund used to pay for intelligence – from 1870s to the formation of Britain’s Secret Service Bureau and the origins of MI5 and SIS. The collection holds a number of self-contained juicy stories of interest to dissertation students and others; examples include, papers dealing with Hitler’s deputy, Rudolf Hess, following his flight to Britain in May 1941 and subsequent correspondence, accounts of Churchill’s boozy wartime meeting with Stalin in August 1942, and rumors of the Duke of Windsor’s alleged pro-Nazi sympathies while Governor of the Bahamas (1940–2). Of greater significance are the files on the operational intelligence provided by SIS’s wartime human intelligence (HUMINT) networks and the Government Code & Cipher School (GC&CS). Files on GC&CS’s diplomatic codebreaking and the decrypts – known in Whitehall as the ‘Blue Jackets’ or ‘BJs’ because of the distinctive files they were circulated in – highlight the importance of codebreaking to British foreign policy. There are also many gems here from the Cold War period. The stark contrast between the wartime deluge of information collected on Nazi Germany and dearth of intelligence on Stalin’s Russia is made clear with internal discussions on SIS’s post-war activities. Also covered are intelligence’s associated activities of propaganda and covert action or special operations. Interestingly, the FO 1093 series contains several examples: Operation Embarrass, an SIS operation to stop illegal Jewish immigration to Palestine by attacking empty refugee ships in 1947, and Operation Valuable, a joint SIS-CIA operation to destabilise the Communist regime of Enver Hoxha, both operations that underline the downsides of James Bond-type activity.

Also important are the Cabinet Secretary’s Miscellaneous Files (CAB 301) from the Cabinet Secretary’s private office dealing with the administration, funding and role of the intelligence agencies, first appearing at TNA in 2013. Since its creation during the Great War, the Cabinet Office and the most senior Civil Servant in Whitehall, the Cabinet Secretary, has been an important individual in coordinating British intelligence. Once again, there are interesting self-contained stories here: the bugging of King Edward VIII’s telephone calls during the abdication crisis remains one of the most sensitive intelligence operations authorized by government. But the most significant papers here deal with the funding of intelligence and the central role of the Treasury, a department often overlooked in histories of Britain’s spies. The series also includes important reviews on the state of intelligence: Sir Maurice Hankey’s 1940 review led to the removal of MI5’s first Director General, Sir Vernon Kell, and provides an important snapshot of the intelligence community in the early stages of the Second World War, while Bland’s 1944 inquiry into SIS helped set the post-war priories of the service. Also apparent here are the post-war difficulties faced by British intelligence in changing intelligence targets from Nazi Germany and the Axis to the Soviet Union. Sir Norman Brook’s report, completed in March 1951, offers a frank assessment of the underlying problems in post-war British intelligence and the dangers of operating against the Soviet Union with a ‘closed society’ and ‘extremely high standard of security’.

The JIC’s assessments and committee minutes – CAB 56, CAB 81, CAB 158, CAB 159, CAB 163 and CAB 176 – also provide a detailed insight into British intelligence from the formation of the committee in 1936, throughout the Second World War to the early Cold War. In many ways, the files highlight the maturing of not just the JIC but the intelligence community as a whole, with the committee playing an increasingly central role in Britain’s wartime and post-war intelligence effort. From tactical analyses to long-term forecasting, operational intelligence to important policy decisions, the JIC’s assessments were pivotal to wartime decision-making, underlined by the committee’s continuing and growing influence in Whitehall post-1945. The papers here give a detailed insight for anyone interested in British defence, diplomacy, intelligence and security throughout the period.

Like all archival sources dealing with intelligence and security, there are limitations here with the digital collection; understandably, as with the original file material at TNA, there are retained or missing files in the collection – helpfully listed in the guide. The FO 1093s, especially when dealing with SIS’s activities, contain redactions. Like the actual files at TNA, retained documents or pages have been marked with placeholders to ‘provide a full representation of the files in their current form’. Understandably, this is more to do with the declassification and release policies of the UK government, than this online collection. Despite the release of its authorized history of SIS by the late Professor Keith Jeffery in 2010, the service maintains a strict policy and does not, as the preface to the history makes clear, ‘disclose the names of agents or of living members of staff and only in exceptional circumstances agrees to waive the anonymity of deceased staff’. There are also restrictions on the work of Britain’s codebreakers, GCHQ and their predecessor the Government Code & Cipher School. It’s unclear whether future releases to the series – or files released as a result of future requests under the Freedom of Information Act (2000) to the Cabinet Office or FCO – will be included here, so as to provide an up to date record of the collection. As part of their centenary commemorations in 2019, GCHQ’s in-house historian has already promised to release ‘as many source documents … to the National Archives’ leading other government departments to reconsider redactions. It is to be hoped that this collection is updated in turn to remain up-to-date.

It’s also worth pointing out that some of the material – 24 files in the PUSD papers, as well as several CAB 301’s – are already available for digital download from the TNA’s website, albeit at a small cost. Post-1953 papers from the Cabinet Secretary’s Miscellaneous Files – documents that are available for free download from TNA’s own website – looking into the fallout of the disappearance of Commander ‘Buster’ Crabb and special operations during the Suez Crisis, both in 1956, are also missing, thanks to the editorial decision to use 1953 – ‘the armistice in the Korean War and the death of Joseph Stalin’ – as the effective cut-off point when the first stage of the Cold War ‘effectively came to a close’. Admittedly, there needs to be a cut off here, but documents such as Sir Edward Bridges’ report into the disappearance of Commander Crabb are important Cold War documents, resulting in a change in the relationship between Downing Street and the agencies. Another intriguing omission is Sir Horace Seymour’s June 1952 report into the management and functions of SIS (CAB 301/133) which, despite an ongoing process of reform that had started with Sir Neville Bland’s 1944 report, highlighted significant administrative and intelligence flaws in the agency. It can only be presumed that the editors cut this out as some of the file material reaches July 1956, as with other CAB 301’s now available to read in TNA’s reading rooms.

One danger of an edited collection like this is that users may only consult the files here rather than tread what one prominent UK intelligence academic calls the ‘well worn path to Kew’ or private collections. If past experience is anything to go by, earlier releases of MI5’s papers to TNA, while giving insight into the organisation’s structure, organisational culture and surveillance operations, have made some researchers a little too reliant on Security Service archives without consulting the records of other government departments that would give better insight into their impact. For example, while the FO 1093s give good insight into Britain’s covert action strategy during the early Cold War, researchers would still be well advised to consult other Foreign Office or Cabinet Office papers from the period. Churchill’s private feed of intelligence found in the HW 1 series provides a similarly restricted view of wider British wartime codebreaking when compared to the vast HW 12 series at Kew. The papers of the JIC, too, provide what Richard Aldrich describes as little more than ‘a Jane Austen novel … a world of corridors, conversations and circumlocution’ and that, while the FO 1093s provide operational detail, the TNA and other archives provide the more ‘racy’ details. Researchers will also need to visit TNA to consult some of the omitted CAB 301 files here. In short, it needs to be remembered that document collections like this are no easy fix to the traditional focus on academic rigour and historians still need to cast their net as wide as possible.

Admittedly, all this is a minor gripe. The huge majority of files here are only physically available to see at TNA and perhaps one of the biggest ‘value added’ factors with the collection is that the digital archive importantly saves time and money – both resources often not open to university students and even most researchers in today’s busy higher education environment. Yes, researchers will still need to see the collection as part of a wider research strategy, but there are significant benefits. Anyone wanting to read the HW 1 series in Kew’s reading rooms, for example, to understand Churchill’s wartime access to intelligence would face a long, laborious and often backbreaking process, taking weeks even months to get through over 3,500 files. There entire collection – minus HW 1/1 which, worryingly, is ‘missing’ from TNA’s archive – is available here and available to search and download. The same with the capacious papers of the JIC. The early Cold War files of the committee in CAB 158 and 159 run to thousands of pages and, from experience, take up days of arduous photography to get digital copies. Add to the list the wartime assessments of the JIC and the reader faces a mountain of documents to wade through. The committee’s former Chair, Sir Percy Cradock, himself pointed to the ‘mass of material for the historian’ admitting there was a significant problem of ‘how to reduce it to manageable scope’. The online collection, available to search and download from the classroom, office or at home, makes this more manageable. Anyone wanting to quickly get access to the JIC’s assessment of Soviet interests and intentions can quickly access them, with ease of access – documents can be downloaded from almost anywhere – making this an important resource for teachers and FE lecturers or anyone interested in the collection that may not normally see this material.

The collection is certainly easy to navigate and the content summarized by the editorial team. The wider context to some of the themes here are explored in the introductory subject essays by the editorial team, including historians Anthony Glees, Dennis Smyth, Matthew Jones and Michael Goodman, the JIC’s official historian, as well as the former Chief Historian of the Foreign and Commonwealth Office, Gill Bennett. Also helpful are the nine descriptions covering each of the file series – CAB 56, CAB 81, CAB 158, CAB 159, CAB 163, CAB 176, CAB 301, FO 1093 and HW 1 – from Stephen Twigge, Head of Modern Collections at The National Archives, giving an overview of each of the nine series. Perhaps missing here is a broader understanding of the role of British wartime cryptography, often overshadowed by the work of Bletchley Park (or Station X) in the literature. Smyth’s essay on the use of ULTRA righty observes the importance of Britain’s codebreakers to military strategy and operations, but the HW 1 (or DIR/C) collection also holds important diplomatic decrypts from GC&CS’s codebreakers at Berkeley Street, London, that helped shape Churchill’s and the Foreign Office’s wartime diplomacy, also highlighted by files in the FO 1093 series.

From a user’s perspective, the collection is easy to grasp and has a number of nice features. A quick search option allows readers to look for key terms and you can even see other topics that users have been looking at. Completely new to the subject? There are options to search by theme – ‘British domestic security’, ‘Propaganda, censorship and psychological warfare’ and others. Clicking on the theme ‘Intelligence funding and finances’ there were specific references to the JIC’s files as well as others in FO 1093s and CAB 301s. As with other search engines, results can also be sorted by date, title, document type or series, with other filters restricting searches on geographical grounds, by organisation or document type. Unlike ‘Discovery’, the TNA’s online catalogue, searches go into the detail of the file material itself, highlighting files of interest that individuals or specific organisations appear in, showing references that may have gone unnoticed before. Another handy feature is the ability to search by key people, selected by the editorial board. Search options for Hitler, Churchill, Chamberlain, Eden and others provide a quick an easy way of finding documents on the central politicians and statesmen of the period. Sounds the same as any other search engine, perhaps? Here differences with TNA’s catalogue become clearer. For instance, a search for Sir Dick White – the only man to lead both MI5 and SIS – on ‘Discovery’ only highlights files where his name can be found in the actual title. The same search in the online collection provides references to files that he appears in, having actually searched for ‘Sir Dick White’ in the file material itself. This is another important example of ‘value added’.

From my own perspective, a search for files on Clement Attlee – the subject of a major study in 2016 – found wartime JIC papers that he had access to, a search that would have required me to look through several volumes of JIC papers, taking days even weeks. The key people search function also allows students, teachers and researchers to study some of the lesser known personalities involved in espionage from the 1870s through to the end of the Korean War. The Chairs of the JIC, heads of the intelligence agencies, diplomats and spies can all be found here. The files related to Alan Nunn May, Klaus Fuchs and SIS’s H. A. R. ‘Kim’ Philby provide information on the significant low points for British intelligence, while searches for SIS’s Chief (‘C’) Major General Sir Stewart Menzies – a figure who dominates the agency throughout the collection – offers readers the ability to figures little understood outside the world of academia. Once again, the function finds individuals in the documents themselves, not just in the title of the file, allowing readers to find references they’d often overlook.

In all, the collection is certainly welcome. Students, teachers, researchers and academics will find this a rich resource for teaching and writing on intelligence with the so-called ‘missing dimension’ getting smaller and smaller for each new generation interested in British intelligence and security. Certainly, those with access to the collection have no reason to neglect the input of Britain’s intelligence and security agencies from their work. While there are some caveats, as this review has made clear, these are relatively minor when compared to the real positives of the collection. The sheer size and usability of the digital records makes this a worthwhile aid for anyone interested in early 20th-century international, political, military and intelligence history. It’s only hoped that a new online collection incorporates future releases to the UK National Archives.