The Spy and the State: The History of American Intelligence

Book: The Spy and the State: The History of American Intelligence
Jeffrey P. Rogg
Oxford, Oxford University Press, 2025, ISBN: 9780197678732, 632pp, Price: £30.99
Reviewer: Cameron Sparkhall
Citation: Cameron Sparkhall, review of The Spy and the State: The History of American Intelligence, (review no. 2495)
10.14296/RiH/2026/2495
Date accessed: 20 October 2026

Book:

The Spy and the State: The History of American Intelligence
Jeffrey P. Rogg
Oxford, Oxford University Press, 2025, ISBN: 9780197678732, 632pp, Price: £30.99

Reviewer:

Cameron Sparkhall

Citation:

Cameron Sparkhall, review of The Spy and the State: The History of American Intelligence, (review no. 2495)
10.14296/RiH/2026/2495
Date accessed: 20 October 2026

After the Bay of Pigs fiasco in 1961, the public and the press began to question the Central Intelligence Agency’s (CIA) role within American society. The New York Times reporter Ben Bagdikian declared the Agency ‘represents the dilemma modern America faces in a world where it proclaims the Democratic ethic but where the consequences of nuclear miscalculation and surprise are intolerable.’[1] Bagdikian’s comments demonstrate a broader issue than just the CIA’s survival after the Bay of Pigs. He offers an insight into a recurring conflict that has plagued the history of American intelligence: how can a nation, built on the constitutional principles of liberty, democracy, and transparency, indulge in subversive, undemocratic, and secretive methods of espionage abroad and at home?

This unique American dilemma serves as the focal point of Jeffrey Rogg’s excellent book, The Spy and the State: The History of American Intelligence, which skilfully provides a comprehensive exploration of US intelligence from the Revolutionary War to today. Adapted from Samuel Huntington’s conceptual framework of civil-military relations in The Soldier and the State, Rogg uses ‘civil-intelligence relations’ as an analytical lens to trace the development of US intelligence by exploring its contentious relationship with the government and the American people. Rogg emphasises that ‘Americans, both individually and collectively, have shaped intelligence in the past and will continue to do so in the future’ (p.2). His argument that intelligence has been moulded by its public perception shows how it was not just a conversation held within the highest echelons of the executive, but an issue represented within, and impacted by, the broader public debate.

Rogg’s concept of civil-intelligence relations provides a refreshing perspective on the history of US intelligence because it demonstrates that it was not merely a “top-down” approach, implemented within the realms of secret, unaccountable presidential and executive control.[2] It has had to adapt to the processes of American democracy. For example, it must respond to Congressional and public oversight whilst maintaining the secrecy required to conduct covert operations. Unlike other overviews that provide institutional histories of US intelligence agencies that focus on the field’s operations and functions, Rogg reframes intelligence history as a political and constitutional struggle over secrecy and openness.[3] The Spy and the State recognises that intelligence is not a separate entity that is distinct from the outside world. It has, and continues to have, a close relationship to culture, society, and politics. Rogg’s work therefore speaks to and complements a recent wave of scholarship that link the world of intelligence to the public sphere.[4]

For intelligence agencies to effectively operate, they require secrecy. This secrecy creates a ‘state of dynamic tension with US civil society’ because intelligence is a ‘fundamental opposition’ that runs against ‘an informed citizenry, which is a pillar of the American constitutional system’ (p.4). Intelligence secrecy is treated with public suspicion because it has led to abuses of power. Notably Rogg highlights throughout his book that the persistent use of domestic surveillance has undermined trust between citizens and the intelligence community. A notable example would be Operation MHCHAOS, a massive CIA surveillance program created in 1967 with the aim to determine whether anti-Vietnam War protestors received foreign funding. The program lasted until the early 1970s where the CIA collected thousands of files on American citizens. After its exposure by the investigative reporter Seymour Hersh in 1974, public opinion soured on the Agency and led to ‘the most searching congressional investigation in American intelligence history.’ (p.312). Congress took extraordinary steps to establish strict measures of oversight that are still in use today. As The Spy and the State rightfully notes, the imbued relationship between intelligence and the American people has resulted in a similar pattern in where the public tolerates intelligence operations, ‘when faced with danger,’ but protest against it when it impacts their civil liberties (p.5-6). A notable example of this inconsistency would be the unearthing of the Glomar Explorer project in the 1970s. The CIA operation to raise a Soviet nuclear submarine was met with remarkable positive press coverage, praising the Agency’s ingenuity and technical brilliance, but the revelation of MHCHAOS received public backlash because it was perceived as intelligence overreach due to its targeting of American citizens. In response to such ambivalence in public response, there have been periodic moments in US history where intelligence has appeared and reappeared within the public debate due to ‘crisis and controversy’ (p.6).

The book’s structure covers four eras, tracing the development of the US intelligence system and the tensions surrounding civil-intelligence relations. The first section encompasses the Revolutionary War to the Civil War. Intelligence played a vital role in the Nation’s founding, from decrypting a coded letter that helped unmask Dr. Benjamin Church, a spy who worked for the British, to Thomas Paine’s pamphleteering propaganda efforts in his celebrated Common Sense. The expansion of presidential power, granted by the Constitution’s vague framing, gave President George Washington complete control over the Continental Army’s intelligence activities and had ‘established a precedent for intelligence in the future United States’ (p.32). Despite its increased importance under Washington, intelligence was sporadic, amateurish, uncoordinated, and viewed with scepticism. The Civil War shifted intelligence towards other executive government departments such as the State, War and Navy. But intelligence was still not given the status it needed to operate effectively. President Lincoln relied on amateur spy and private detective Allan Pinkerton to provide information about the number of Confederate troops. Pinkerton deliberately overestimated the size of the Confederate Army in an effort to appeal to General George B. McClellan, who believed that the Rebel soldiers outnumbered the Union. As Rogg notes, this demonstrated the risk of intelligence politicisation, and it had ‘damaged civil-intelligence relations’ (pp.86-87). Intelligence was employed haphazardly and was used as a political tool by Lincoln to weed out his critics and undermine civil liberties at home (pp.94-95).

The second section recalls the creation of the early intelligence community. It begins with the establishment of military-dominated intelligence agencies such as the Office of Naval Information (ONI) in 1882 and the Military Information Division (MID) in 1885. Both suffered from intense bureaucratic turf wars that undermined their work and the perception of intelligence as a legitimate instrument used for statecraft. Rogg argues that constant bureaucratic infighting over the ONI and MID, coupled with Lincoln’s last order to establish the US Secret Service in 1865, and the quick ascension of the Bureau of Investigation (BOI, now FBI), blurred the lines over the intended roles of each, and led to a serious crisis of identity for the development of US intelligence. Congress struggled to keep up with the new institutions and failed to provide clear legislative charters for each, impacting civil-intelligence relations. The result was an uncertainty surrounding which organisation held control of domestic intelligence and law enforcement. The two became indistinct and intertwined, leading to a confusing implementation of intelligence by institutions that were deliberately subordinated within the departments of the federal government. This emerging intelligence community, Rogg states, ‘would prove to be hardly a community at all’ (p.130).

The period between the World Wars complicated matters further. The State Department wanted its own agency that would conduct both foreign and domestic intelligence whilst the newly established communications intelligence organisation, the Cipher Bureau, could not fulfil its potential due to bureaucratic tensions. President Franklin D. Roosevelt failed to establish clear guidelines for the implementation of domestic intelligence, granting J. Edgar Hoover’s FBI an unprecedented escalation of power and influence. Rogg argues that the public did not harbour suspicions of the FBI because it was a civilian institution (p.177). Contrasting with military intelligence agencies, the FBI had to be more transparent with the public because of this civilian status. Hoover recognised this as he adopted numerous public relations campaigns to promote his FBI’s public profile.[5] I would have liked to have seen this point expanded further because it really cuts into the difference between public perceptions of military and civilian intelligence agencies, and, more broadly, the paradigm of civil-intelligence relations itself.

The book’s third section traces the aftermath of the Second World War to the end of the Cold War. It involves the creation of an entrenched national security state with the establishment of the nation’s first permanent, peacetime intelligence organisations. The 1947 National Security Act created the CIA and brought with it intense public scrutiny and legislative debate. However, as Rogg expertly elucidates in this chapter, there were key issues surrounding the drafting of the 1947 Act that led to tremendous consequences for the scope, role, influence, and implementation of intelligence activities. A vague provision within the 1947 Act directed the CIA to ‘perform such other functions and duties related to intelligence.’[6] This was used by the Agency ‘to argue it was given a legislative mandate to perform covert action, a mission that would come to haunt the CIA, the intelligence community, and American civil-intelligence relations’ (p.250). Covert action had dominated the Agency’s path during its 1950s Golden Age, undermining ‘its original statutory missions of intelligence coordination and analysis’ (p.272). This culture of clandestinity shifted towards a more transparent phase during the 1960s and 1970s.[7] Rogg is careful to highlight how the Bay of Pigs and Operation MHCHAOS fuelled the tensions within civil-intelligence relations. These exposed the Agency to public and congressional criticism but, more importantly, as Rogg points out, these events demonstrated that the overlapping of foreign and domestic intelligence called into question the CIA’s compatibility with American democracy.

The final section explores the end of the Cold War to the present. After the dissolution of the Soviet Union, the public began to question the CIA’s relevance. Calls for transparency and reform dominated the public debate which led to budget reductions within the intelligence community whilst some even argued if the US still needed the CIA.[8] But it was the ramifications of 9/11 that drove tensions within civil-intelligence relations. The rapid reform of the intelligence community instigated during the Global War on Terrorism blurred the lines between domestic intelligence and law enforcement. Just as the FBI was granted opaque authority for its domestic operations during the 1930s, undermining civil liberties, the war on terrorism gave the organisation sweeping powers to obtain information. It expanded its use of National Security Letters, an administrative subpoena to gather information on any individual without a warrant (p.434). The CIA, having survived its brief period of existential threat, was given extraordinary powers to capture, interrogate, and torture suspected terrorists, whilst the National Security Agency (NSA) conducted a massive domestic surveillance program. Despite these egregious acts, Rogg explains that post-9/11 patriotic fervour resulted in an increase in intelligence missions and domestic surveillance methods because public opinion drove this expansion. In times of threat to national security, the American people tolerated the abuse of civil liberties if it promised safety and protection (p.427).

Rogg argues in his conclusion that technological innovations and developments since the First World War has been tantamount to an Intelligence Revolution. In its struggle to become a respected, standalone enterprise after the Second World War, intelligence extended beyond the state to American civil society due to the rise of US state and private group networks during the Cold War. In today’s world, the Intelligence Revolution is at its peak. Whilst the public expects transparency, particularly after Edward Snowden revealed in 2013 that the NSA collected data on citizens, private companies continue to acquire more information. These companies now hold tremendous surveillance powers that conflict with the privacy rights of the American people (p.442). The US government is just as complicit as the corporate world. The war on terrorism demonstrated how the state harnessed a massive surveillance program that undermined civil liberties. As Rogg has shown throughout, the US has been slow to establish adequate oversight measures to respond to rapid developments in intelligence capabilities and practices. Recent innovations in the field of open-source intelligence and artificial intelligence likewise run the risk of broader abuses of power due to a lack of clear legislative guidelines.  

 The Spy and the State convincingly argues that the overarching theme of civil-intelligence relations is a useful window into a comprehensive history of US intelligence because it gets to the heart of why public perceptions of intelligence matter. But there are two other themes in the book that need addressing because they demonstrate why US intelligence has had a long and complex history in relation to the American public. The first is the theme of intelligence professionalisation, which took hold from the Revolutionary War until the creation of the CIA. Intelligence was not taken seriously as a career until the Agency’s establishment in 1947. Before this, it was constantly at the mercy of presidential power, military domination, and petty bureaucratic squabbles. It was relegated to a temporary role, employed haphazardly, and only serving to further the careers of the few individuals who used it. As Rogg rightfully highlights, the US had consistently failed to professionalise intelligence during key moments of the field’s development prior to 1947. This lack of professionalisation, compounded by an absence of civilian leadership, placed the use of intelligence as a tool for statecraft at a disadvantage. The US military took advantage of technological developments after the Civil War and employed an ‘educated’ and ‘trained’ staff to professionalise at the expense of an independent intelligence service (p.106), whilst departments continued to subordinate intelligence during the First World War (p.140). Not even the Office of Strategic Services (OSS) in World War Two solved the issue, for it ‘was an amateur organization aspiring to professionalism, not a professional intelligence service’ (p.217).

Linking to The Spy and the State’s central argument, the development of US intelligence aligns with the perceptions of the American public. The head of the OSS, William Donovan, recognised this unique relationship as he mounted a public relations campaign to promote the successes of OSS wartime operations, aiming to establish a permanent intelligence service.[9] The creation of a professional intelligence system was successful in part because key press columnists provided their unwavering support for it after the war. For example, prior to the CIA, the weak Central Intelligence Group (CIG) was criticised by the Washington local paper, the Evening Star. Their foreign affairs correspondent, Constantine Brown, criticised the CIG because it became embroiled in departmental turf wars over who should control intelligence. He argued that only a professionalised intelligence organisation would move beyond the bureaucratic constraints that needlessly burdened the CIG. A CIA was essential, he declared, and it needed an immediate legislative charter, to prevent US intelligence from lagging dangerously behind intelligence agencies of the other world powers.[10]

The second key theme that runs throughout the book is the struggle with legislative oversight. The notion that Congress cannot keep a secret has been used by the leaders of various intelligence agencies to shield their valued sources and methods. Rogg provides useful historical links to effectively argue that oversight procedures have regularly been slow to keep up with developments in US intelligence. Vague legislative charters have failed to clarify guidelines over the implementation of intelligence activities. The hasty passage of the Contingent Fund in 1790 and the 1949 CIA Act provided a ‘black budget’ granting intelligence organisations carte blanche to conduct operations with minimal legislative interference (pp.257-258). Congress has also been reluctant to provide adequate oversight. The Founding Fathers Benjamin Franklin and John Jay were against the sharing of intelligence between the Committee of Secret Correspondence and the Continental Congress (p.23). Later, at the height of the Cold War, senior senators such as Richard Russell and Leverett Saltonstall opposed any measure to increase legislative oversight of the CIA (p.284). Even during the 1975 ‘Year of Intelligence,’ congressmen criticised the CIA director, William Colby, for being too candid about his Agency’s past activities.[11]

The longstanding absence of a professionalised intelligence service and the haphazard oversight of these services are two themes Rogg traces brilliantly in his work. They provide a useful approach towards understanding the culture of US intelligence and the inherent conflicts associated with it. Another theme that would have complemented this well might have been the inclusion of secrecy as a phenomenon that shapes what the public know. Secrecy has led to various myths and misconceptions over intelligence. The lexicon of national security has been used to justify withholding information, contributing to gaps in the public’s knowledge regarding the processes and results of intelligence. Members of the public have filled these gaps with conspiracy theories and fanciful representations in popular culture. Directors of intelligence have used secrecy to further their services’ own needs. For example, the CIA leader Allen Dulles often divulged intelligence secrets in interviews but also argued that the press publishes too much sensitive information. In other words, Dulles used secrecy to control the narrative. He dictated what could be published whilst stating that the press knows too much about intelligence. Secrecy is a key factor in understanding intelligence culture and intelligence history and with all the important themes Rogg discusses, maybe it could have been given a seat alongside them?

This does not, however, distract from this excellent, thought-provoking text, which I have thoroughly enjoyed reading and reviewing. The book’s writing is fluent, whilst the structure and subsections greatly aid the flow of such a large study. Rogg’s expert research makes good use of a rich variety of primary sources, including government documents, congressional records, and personal correspondence, which serve as examples of how valuable archival research is to historians. The use of public polls and newspapers help us trace the broader public debate on intelligence. It is telling that a history on US intelligence can be aided by public sources. It again speaks to the unique American tension between openness and the supposedly hidden executive world of intelligence that such details can be read within the contents of newspapers. The Spy and the State is more than just an administrative overview of US intelligence, it tells the much-needed story of America’s own fraught relationship with intelligence, which continues to this day. It contributes to our understanding of intelligence within the US domestic setting by highlighting the importance of public discussion and emphasising the vital role citizens hold in the shaping of America’s intelligence services.

 

[1] Ben H. Bagdikian, ‘Unsecretive Report on the C.I.A.,’ The New York Times, October 27, 1963, p. 208.

[2] See Christopher Andrew, For the President’s Eyes Only: Secret Intelligence and the American Presidency from Washington to Bush (New York: HarperCollins Publishers, 1995) and William J. Daugherty, Executive Secrets: Covert Action and the Presidency (Lexington: University Press of Kentucky, 2004).

[3] See Mark M. Lowenthal, Intelligence: From Secrets to Policy, 10th ed. (Thousand Oaks, California: CQ Press, 2026) and Jeffrey Richelson, The US Intelligence Community, 7th ed. (Boulder: Westview Press, 2016).

[4] See for example, Timothy Melley, The Covert Sphere: Secrecy, Fiction, and the National Security State (Ithaca: Cornell University Press, 2012); David Shamus McCarthy, Selling the CIA: Public Relations and the Culture of Secrecy (Lawrence: University Press of Kansas, 2018); David P. Hadley, The Rising Clamor: The American Press, the Central Intelligence Agency, and the Cold War (Lexington: University Press of Kentucky, 2019); Simon Willmetts and Constant Hijzen, ed., Spies, Culture and Society: Coming in from the Cold (Georgetown University Press: Washington, D.C., 2026).

[5] See Beverly Gage, G-Man: J. Edgar Hoover and the Making of the American Century (London: Simon & Schuster, 2022).

[6] National Security Act of 1947, Pub. L. 253, 80th Cong., 1st sess., July 26, 1947.

[7] For turning points when the CIA experienced increased public scrutiny, see Hugh Wilford, The Mighty Wurlitzer: How the CIA Played America (Cambridge, MA: Harvard University Press, 2008); Tity de Vries, ‘The 1967 Central Intelligence Agency Scandal: Catalyst in a Transforming Relationship between State and People,’ The Journal of American History, 98, no. 4 (2012), 1086-1087; Richard H. Immerman, The Hidden Hand: A Brief History of the CIA (Hoboken: Wiley, 2014); and Simon Willmetts, ‘The Burgeoning Fissures of Dissent: Allen Dulles and the Selling of the CIA in the Aftermath of the Bay of Pigs,’ History, 100, no. 340 (2015), 167-188.

[8] See Daniel P. Moynihan, “Do We Still Need the C.I.A.?” NYT, May 19, 1991, p. E17.

[9] For examples of Donovan’s PR success in promoting the OSS’s wartime efforts, see William H. Harrison, ‘New Agency Needed as OSS Replacement,’ Evening Star, September 16, 1945, p. C2, and ‘OSS Active in End of Nazis in Italy,’ The New York Times, September 18, 1945, p. 8.

[10] For examples of Constantine Brown’s substantial commentary on the state of US intelligence see Constantine Brown, ‘New Central Intelligence Badly in Need of Funds,’ Evening Star, June 16, 1946, p. C1, and Constantine Brown, ‘CIG Called Failure as Spy Bureau,’ Evening Star, November 10, 1946, p. C1.

[11] For literature on CIA Congressional oversight see David M. Barrett, The CIA and Congress: The Untold Story from Truman to Kennedy (Lawrence: University Press of Kansas, 2005), and Dafydd Townley, The Year of Intelligence in the United States: Public Opinion, National Security, and the 1975 Church Committee (Palgrave Macmillan, 2021).

August 2026