The last chapter of Britain’s most infamous spy scandal can finally be told as the release of security service files reveals the long hunt for Kim Philby, the notorious KGB mole inside MI6, who spied for Russia for three decades.

Philby was recruited by the Soviet intelligence service in 1934, soon after leaving Cambridge University. He joined MI6 (also known as the Secret Intelligence Service, SIS) during the war and rose swiftly through the ranks, passing on to Moscow every secret he could gather, with murderous results. In 1963 he was finally exposed while working as a journalist in Beirut, and fled to Moscow, where he died in lonely alcoholic exile 25 years later.

Every stage of that extraordinary espionage journey is depicted in the 21 MI5 files declassified by the National Archives: from Philby’s recruitment as a young communist to the moment he was confronted in Beirut by Nicholas Elliott, his oldest friend in MI6. That confrontation was recently depicted by Damian Lewis and Guy Pearce in the ITV drama A Spy Among Friends.

Damian Lewis as Nicholas Elliott in A Spy Among Friends.

Philby’s confession to Elliott is the most remarkable document in the cache, a confection of truth, half-truth and lies that gave the spy just enough time to alert the KGB and arrange his escape. Elliott secretly recorded their conversation, and the transcripts have been released for the first time, giving an extraordinary insight into the mind of Britain’s greatest traitor and a testament to a friendship brutally betrayed.

Webs of Deceit

In one exchange, Philby owned up to exposing Constantin Volkov, a KGB officer who tried to defect to Britain in 1945. Volkov and his wife were consequently captured by undercover KGB hitmen, taken to Russia and liquidated.

The Volkov incident was a moment of extreme peril for Philby who, by 1945, led Soviet counterespionage at MI6. In September of that year, Volkov walked into the British consulate in Istanbul and offered to hand over an astonishing trove of secrets in exchange for £50,000 and resettlement in the UK.

Volkov’s most significant revelation was the existence of nine Soviet agents inside the British establishment, seven in the Foreign Office and two in MI6. One of these, Volkov revealed, was head of a counterintelligence branch: that was Philby.

Realizing he was about to be unmasked, Philby sent an urgent warning to his KGB handler, took over the case personally and flew to Turkey. By the time he arrived in Istanbul, Volkov and his wife had already been kidnapped, drugged, placed on stretchers wrapped in bandages and spirited out of the country on a plane to Bulgaria by a doctor and two KGB officers. They were never seen again.

Volkov had offered MI6 a vast array of material: not just the identities of KGB spies, but also copies of all the material they had provided, a full list of every KGB officer in Moscow, safe houses, codes and even the keys to filing cabinets inside KGB headquarters.

Had this intelligence reached the West, it would have changed the course of the Cold War. Philby’s intervention, and Volkov’s death, was his greatest single act of espionage on behalf of the Soviet Union.

In his report to MI6 from Istanbul, Philby diverted attention from the real reason for Volkov’s disappearance, writing:

“The probable explanation is that Volkov betrayed himself. Either he, or his wife, or both, made some fatal mistake.”

Philby was insouciant when asked about the case by Elliott in their final conversation, 18 years later.

“Presumably what you passed to them [the KGB] was mainly directed towards information of direct interest to them, like the Volkov business?” said Elliott. “Indeed,” said Philby, and changed the subject.

Had [Volkov’s] intelligence reached the West, it would have changed the course of the Cold War.

In 1951 Philby faced another crisis after the defection of his fellow KGB spies Guy Burgess and Donald Maclean. By then Philby was MI6 station chief in Washington, one of the most senior roles in British intelligence.

Burgess was working for the Foreign Office and staying in Philby’s home when Philby discovered that Maclean, also in the Foreign Office, was about to be unmasked back in the UK. He sent Burgess to warn him.

From Washington, he sent a letter to Burgess containing a coded message, disguised as a complaint about Burgess’s abandoned car, telling him to hurry up and get Maclean out of the country as the MI5 mole-hunters were closing in on all on them: “If I am dragged into elaborate measures in connection with the car, I propose to charge you heavily for it …”

Philby speaks at a press conference at his parents’ house in London after being accused of espionage, 1955.

When Philby discovered that Burgess had defected too, he knew he was in deep trouble, telling Elliott:

“The moment was one of the worst of my life.”

Summoned back to London, Philby was subjected to a ferocious grilling from Helenus “Buster” Milmo, a barrister working for MI6. The transcript of the interrogation is among the newly released files.

Philby was cross-examined repeatedly about his links to Maclean and Burgess, his communist first wife, his political views and the fate of Volkov.

“Did you despise Volkov?” Milmo demanded.

“He was just a defector,” said Philby, flatly denying any role in his death.

Philby confessed to nothing, and although Milmo and MI5 were convinced of his guilt, they concluded he had made “no admission that could be used in a prosecution”.

Philby was forced to leave MI6, but with the help of his former colleagues, notably Elliott, he obtained work as a journalist. In 1956 he moved to Beirut as correspondent for The Economist and The Observer.

Double Act

In 1962 MI5 finally found the proof it needed, when Philby’s friend Flora Solomon revealed that he had attempted to recruit her as a Soviet agent back in 1934.

Her MI5 interrogation is included in the Philby files, including a telling moment in 1951 when she bumped into him at a cocktail party and he “looked at her appealingly—she was in no doubt that he was asking her to say nothing of what she knew”.

Elliott was dispatched to confront him and extract a confession. Having bugged an apartment, Elliott invited Philby to a meeting and directly accused him of treachery.

At first Philby denied everything, but then agreed to confess, framing this as an act of friendship.

“I certainly would not have spoken to anyone else,” he said, oozing flattery. “When you told me that you yourself believed the evidence against me, that did it. I have had this particular moment in mind for 28 years … Here’s the scoop.”

Elliott in the 1940s.

Philby said he had believed his only choice was between “suicide and prosecution,” but Elliott offered him a way out, by promising that if he gave a full confession, the evidence would not be used to prosecute him. Parts of the file are still redacted, and it is not clear what legal backing, if any, Elliott had to offer immunity. It may simply have been a ruse to get him to talk.

Over the next few days, in conversations and in writing, Philby offered his half-confession, after giving a “personal assurance that he was telling the truth, the whole truth and nothing but the truth”. Some of it was indeed true but almost valueless (“chickenfeed” in spy parlance), parts were deliberately vague, much of it was false and all of it was laced with Philby’s charm and repeated allusions to their friendship. It was, he said, a “tremendous relief” to “spill the whole beans”.

He admitted tipping off Maclean— “As I recruited him in the first place the least thing I could do was get him off the hook”—but claimed to have had little contact with him since 1934, which was a lie. He insisted that:

“Anthony Blunt ‘did not, and would not, have worked for the Russians.”

This was another lie. He made no mention of the Treasury official John Cairncross, another of the Cambridge ring.

Above all, Philby lied that he had stopped working for the KGB in 1946 and had not been in contact with the Russians since. In fact, he had continued spying for the Soviets throughout his time in America, and was still doing so in Beirut.

Asked why he had broken ties with the KGB in 1946, he said: “I saw after the war the things I had wanted in the ’30s coming to pass … The health service and so on.”

Asked about the many people he had betrayed to the Soviets, he fudged: “I can remember nothing specifically.” In reality scores, probably hundreds, died as a direct result of his espionage.

Throughout his exchanges with Elliott, Philby displayed what he called “controlled schizophrenia”, the double-think of the double agent. He said:

“I really did feel tremendous loyalty to SIS and I made some really marvelous friends there, but the overruling inspiration was the other side.”

What Graham Greene called “the splinter of ice” in Philby’s soul frequently emerges in the transcript.

Philby’s K.G.B. identification card.

At one point, he describes one of his KGB handlers as “a very, very nice fellow indeed … but obviously prepared to torture and slaughter anybody”.

Elliott concluded his meetings with Philby by saying “he had always taken it that Kim had always regarded loyalty to friends as a more important factor than loyalty to country. It was clear that in light of what Kim had said that in practice, idealism could bitch [overcome] the friends. Kim agreed and said that, thinking it over, if he had his whole life to lead again, he would probably have behaved in the same way.”

“You needn’t worry that I will do a Burgess,” Philby said. “If I intended to do so, I would have done so years ago.”

They parted amicably, after Philby agreed to make himself available to continue being debriefed. “Any subsequent problems and questions would be followed up” by Peter Lunn, the MI6 station chief in Beirut.

“I prefer to carry on as usual,” Philby said.

“Yes. I think this is the thing,” Elliott replied. The MI6 officer then flew on to Africa.

“I really did feel tremendous loyalty to SIS and I made some really marvelous friends there, but the overruling inspiration was the other side.”

A few days later, on January 23, 1963, Philby failed to turn up for a dinner party, climbed on to a Russian freighter moored in Beirut harbor and headed for Moscow. The note he left behind for his wife, Eleanor, who knew nothing of his espionage, was a last lie:

“My beloved, I have been called away at short notice. I am sorry I cannot be more explicit at the moment, but plans are somewhat vague. Don’t worry about anything. Tell everyone that I am doing a tour of the area. All love to the kiddies, and tons to yourself. Your, Kim.”

The declassified files answer many of the remaining questions surrounding Philby, the most successful double agent in history. But one mystery remains. Did Elliott intend that Philby should abscond, and deliberately leave the door open to Moscow? Was Elliott fooled one last time, blinded by friendship, or was he playing a far more subtle game?

Even Philby was never sure of the answer. From Moscow, he wrote to his old friend: “I cannot help thinking that perhaps you wanted me to do a fade”—intelligence jargon for a defection.

Elliott deliberately muddied the waters, put nothing down on paper and in 1994 he took that last secret to the grave.

Ben Macintyre is a writer at large at The Times of London and the best-selling author of The Spy and the Traitor, A Spy Among Friends, Double Cross, Operation Mincemeat, Agent Zigzag, and Rogue Heroes, among other books