In 2014 I was approached by a publisher with an offer I could not refuse. Would I agree to write the authorized wartime history of the SAS with unprecedented and unrestricted access to the regiment’s secret archives? Had this offer been made by hand, I would have bitten it off.

The Special Air Service retains an extraordinary grip on the British imagination. The very name summons up images of muscular men performing astonishing feats of endurance, and killing their enemies, secretly, with brutal professional efficiency and high-tech weaponry. No fighting unit in the world has been the subject of greater speculation, admiration and fantasy.

The SAS is embedded in British national mythology. Everyone has heard of it yet the operations of this special forces regiment are usually veiled in mystery. In theory, SAS veterans are supposed never to reveal what they do, and those that break this oath of discretion are ostracized. The regiment never comments on its operations, neither confirming nor denying.

What historian would turn down the opportunity to pierce that shroud of secrecy?

But then I began to have second thoughts. A quick trawl of Amazon revealed numerous books on the SAS, many of them terrible: accounts of derring-do written in thumping macho prose. Did I really want to join their number?

The term “authorized” can mean very different things: would the SAS leave me a free hand, or would the regiment try to control the content of the book? And what, exactly, was in this archive? “Archive” is a grand word for what turned out to be a nondescript room in an army barracks, filled with bulging filing cabinets, guarded by a charming secretary armed only with endless biscuits.

One look inside those files convinced me that here was historical gold dust: not just official reports, but private letters, postcards, diaries, memoirs written by the wartime soldiers of the SAS, and hundreds of photographs. There were hours of uncut video footage in which “the Originals”, as they styled themselves, spoke about the formation of the regiment, its origins in the north African desert, followed by operations in the Mediterranean, Greece, Italy, behind the lines in Nazi-occupied France and finally in Germany.

SAS veterans are supposed never to reveal what they do, and those that break this oath of discretion are ostracized.

The most important single artifact was the regiment’s 500-page war diary, a compilation of photographs and eyewitness accounts, gathered immediately after the Second World War and bound inside a swastika-stamped leather folder “liberated” from the retreating German forces.

Many former members of the SAS, it seems, had simply bundled up their souvenirs and memories at the end of the war and sent them to the regimental association for safekeeping. With the help of the regimental archivist, I began combing through this trove.

The result was SAS Rogue Heroes, the narrative non-fiction story of the regiment from its inception in 1941 until 1945, when it was temporarily disbanded. That book has now been made into a six-part series for the BBC by Steven Knight, the creator of Peaky Blinders, a superb adaptation with a rocking modern soundtrack, gripping action and some superb individual performances.

With perfectly reasonable license, the series has introduced a beautiful woman and a love story into a tale that was almost exclusively male and unromantic, but the tongue-in-cheek disclaimer at the outset is correct: “The events that seem most unbelievable, are mostly true.”

The adaptation also carries a dark undertone that is true to history, for the reality of the SAS was more complicated, darker and far more interesting than the one-dimensional popular myth.

The bravery of the early SAS was astounding, the tales of individual survival and resilience inspiring, but the archives also revealed a unit that was highly eccentric, deeply diverse, peopled by oddballs who would never have fitted into a conventional army, amateurs, pirates, the motley foot soldiers of what was, in many ways, a ragtag private army.

Some of the early SAS recruits were clever and tough, but others were sensitive, of no more than average strength and fitness, and yet courageous in other, quite unexpected ways. Some were homosexual. Many were brutally hardened by the experience of behind-the-lines warfare, and became more so as the war entered its bloody final phase.

All of this is reflected in Knight’s script, depicting a group of courageous warriors who teeter on the edge of madness, reveling in warfare, and deeply, irrevocably damaged by it.

Far from being a paragon of efficiency, the unit nearly perished at birth after a disastrous parachute operation in the Libyan desert that should never have taken place. The main threat to the survival of the SAS came not on the battlefield, but from old-fashioned military top brass who deeply disapproved of its covert methods.

The SAS was dreamed up by David Stirling (played by Connor Swindells), a young army officer, while lying in a Cairo hospital bed in the summer of 1941. A lieutenant in the Commandos, Stirling had injured his spine during an experimental and unauthorized parachute jump. While convalescing, he hit on the idea of dropping parachutists at night into the north African desert who could then creep up on the enemy airfields strung out along the coast, blow up as many aircraft as possible under the cover of darkness, and then slip back into the desert.

“The events that seem most unbelievable, are mostly true.”

Stirling began recruiting soldiers to join “L Detachment Special Air Service Brigade”. Even the name was an in-joke, chosen as part of a deception plan dreamt up by a cross-dressing intelligence officer named Dudley Wrangel Clarke (Dominic West) to give the false impression that a large brigade of parachutists was being assembled. In fact, Stirling’s initial troop amounted to fewer than 60 men.

These included Northern Irishman Paddy Mayne (Jack O’Connell), an Ireland and British Lions rugby player with an explosive temper, an alcohol problem and a tendency toward extreme violence. Mayne would end up destroying more planes than any fighter pilot on either side of the war. Stirling’s second in command was Jock Lewes (Alfie Allen), an Oxford-educated lieutenant with matinee idol looks, who invented a new sort of bomb that could be stuck on to parked planes.

Stirling was an unlikely figure to lead what would become a famously fit combat unit: he was tall, stooped, upper-class, vague in manner, a sufferer from crippling migraines and desert sores, who drank and smoked heavily but was entirely immune to discomfort or fear. Field Marshal Montgomery said of him: “The boy Stirling is mad, quite mad. However, in war there is often a place for mad people.”

The unit trained in close-quarters combat, explosives and parachuting. They marched for days through the desert with minimal food and water, and prepared for parachute landings by leaping out of the back of a speeding truck. When it became clear that parachuting into the desert was impractical (and frequently lethal), Stirling teamed up with the Long Range Desert Group (LRDG), the army’s reconnaissance, raiding and intelligence unit that could transport the SAS to and from the Axis airfields across hundreds of miles of empty desert.

By the end of 1942 the SAS had developed survival techniques to the point where they could live for weeks on end in the desert, launching hit-and-run raids and then vanishing back to their camouflaged encampments.

Life expectancy was short in the fledgling SAS. If they managed to survive a raid, they still faced being spotted in the desert by German reconnaissance planes and attacked from the air. In addition they had to endure sandstorms and crippling heat. Much of SAS Rogue Heroes was shot on location in Morocco during lockdown; if the cast appear to be choking on sand, sweat-soaked and heatstruck, that is because they were.

SAS attacks inflicted significant damage on Axis air power, sapped German and Italian morale and tied up thousands of enemy soldiers defending the airfields who might otherwise have been deployed on the front line. But theirs was a theatrical and psychological role, as well as a military one.

The men of the SAS were mindful of their own drama, for they looked, dressed and to some extent played the part of swashbuckling desert fighters. Regular irregulars, they carried a variety of weaponry. Some adopted Arab headdresses or bandanas; few wore regulation uniform; almost all, including Stirling, sported bushy beards. These were desert warriors in the tradition of Lawrence of Arabia, but mounted on Jeeps rather than camels.

For all his peculiarities, Stirling was a fine judge of character and the criteria on which he selected his men still apply to SAS recruits today. “I didn’t want psychopaths,” Stirling observed, before listing the qualities he did want: “Courage, fitness and determination in the highest degree but also, just as important, discipline, skill, intelligence and training.” He chose men who were adaptable, unconventional and, when necessary, merciless.

My concern that the SAS would try to control the book proved groundless. The only red line was an understanding that my account would not extend beyond the war to later SAS activities in Malaya, Oman and Northern Ireland.

In the course of researching the book I met many men who had served in the SAS, but only one who could tell me what it had been like from the start. Major Mike Sadler was a young navigator with the LRDG when he took part in the first successful SAS raid, when a six-man team led by Mayne destroyed 24 stationary aircraft and a fuel dump on Tamet airfield. Sadler (Tom Glynn-Carney in the series) then guided them back to the desert base camp as the dawn rose.

The only surviving member of the original SAS, Major Sadler is now 102 and blind, but his memory remains as sharp as ever. Three years ago I took Knight to meet him (he has since given his blessing to the BBC series). The two men sat on a sofa while the old soldier described another raid, on July 26, 1942, during which he navigated a convoy of 18 Jeeps armed with twin Vickers K machineguns across 70 miles of desert, to attack Sidi Haneish airfield. They burst onto the airstrip and opened fire as they drove between the aircraft, destroying at least 37 of them.

In contrast to some more recent SAS veterans, Major Sadler is a master of understatement. Asked to describe what it was like being hunted and bombed from the air by German Stukas in open desert, the most he will concede is: “Well, I suppose it was quite alarming.”

Old soldiers never die, they simply fade away. Soon there will be no soldiers left from the original SAS; but far from fading away, some 80 years after the regiment’s birth, the story of the SAS grows ever more vivid, in books, on screen and in legend.

SAS: Rogue Heroes is streaming now on BBC One in the U.K. and from November 13 on Epix in the U.S.

Ben Macintyre is a writer at large for The Times of London and the best-selling author of The Spy and the Traitor, A Spy Among Friends, Double Cross, Operation Mincemeat, Agent Zigzag, and Rogue Heroes, among other books