“It required an immense amount of stamina,” Emma Thompson says. “After each one, you said, ‘Never again.’” The actress is talking about her experience of being in Merchant Ivory films. She made her name in Howards End, winning the Oscar for best actress for her performance as the arch Margaret Schlegel, and went on to star in The Remains of the Day opposite Anthony Hopkins and a young Hugh Grant.

The films are typical of those made by Ismail Merchant and James Ivory; they look like elegant period dramas, but beneath the surface repressed emotions simmer, with an often brutal humanity. Look at well-behaved Lucy Honeychurch and the rebel George Emerson’s awkward and passionate relationship in A Room with a View. “Period movies had been clunky and all about the frocks until then, but this was different,” Thompson says, in a new documentary, Merchant Ivory.

Quivering upper lips: Anthony Hopkins and Emma Thompson in The Remains of the Day.

Behind the scenes, it was often fraught. Hugh Grant says “the sets crackled with subliminal lust”; Vanessa Redgrave and Thompson remember screaming rows with Merchant, and all the actors involved agree that money was “short”. The budget wasn’t in place by the time they started filming Heat and Dust, starring Julie Christie, and Merchant would wake up early and steal telegrams from the actors’ hotels so they wouldn’t get wind that their agents were telling them to leave.

“I could never have done it without Ismail,” James Ivory says. The pair, who were also in a relationship, had one of the longest partnerships in film, making more than 40 films over 50 years.

Ivory is in his apartment on Manhattan’s Upper East Side (he also has a 19th-century mansion in the Hudson Valley). Behind him are the intricate drawings that were used in the credits for A Room with a View. Ivory is perky company, wearing a pink shirt he bought on a trip to Rome. He’s 96 — and at 89 became the oldest Oscar-winner, getting the best adapted screenplay award for Call Me by Your Name. Merchant, who died in 2005 aged 68 while being treated for stomach ulcers, used to live with him in Manhattan, and Ruth Prawer Jhabvala, who wrote many of their films, was in the flat downstairs until she died in 2013 aged 85.

Two of a kind: James Ivory and Ismail Merchant at the Venice Film Festival in 2003.

Merchant produced the films, managing the stars and the money, which allowed Ivory to concentrate on directing. “Ismail could do everything, practically,” Ivory says fondly. “I’d never take a nap and let him direct a film, but he could do everything else.” He mostly screened out the rows that Merchant had with the actors. “I watched one of his shouting matches with Vanessa Redgrave on The Bostonians from the window and just thought, well, they will have to decide eventually.”

Merchant and Ivory met in 1961 at the Indian consulate in New York, at a screening of a documentary Ivory had made, The Sword and the Flute. “I liked him instantly,” Ivory says. “He came up to me afterwards and was very sincere and enthusiastic.”

At first they were discreet about their personal relationship because Merchant was from a conservative Muslim family. Their 1987 film adaptation of EM Forster’s novel Maurice, starring Grant, explored gay relationships in a repressed culture. It’s the film that got Stephen Soucy, who made the Merchant Ivory documentary, hooked. “They did such an amazing job to communicate that struggle of grappling with sexuality. When I saw it as a closeted gay young man in upstate New York it resonated.”

Picturesque repression: James Wilby and Hugh Grant in Maurice.

In his 2021 memoir Solid Ivory the director is more explicit, talking about their love triangle with Richard Robbins, who composed music for their films. “I thought Jim was a Victorian,” Helena Bonham Carter says of the first time they met. “How wrong I was.”

Merchant made the sets fun places to be, which made up for the lack of funds. “No one got rich doing a Merchant Ivory film,” says Grant, whose brother told him he couldn’t turn down an invitation to audition for Merchant Ivory because they were “classy”. “I did it for the curry nights Ismail put on, which made it feel like a family. He also stole my mother’s car for a week once.”

In the documentary Grant tells a story about how there wasn’t enough of a budget to give everyone a room when they were promoting Maurice, so Ivory took a nap in Grant’s. It was fine until the director went to brush his teeth and realized he’d used Grant’s hemorrhoid cream instead of toothpaste. When I mention it to Ivory, he chuckles at the memory.

“No one got rich doing a Merchant Ivory film.”

They weren’t always strapped for cash — for their 2000 film The Golden Bowl they worked with Harvey Weinstein. It was a disaster. Ivory described Weinstein as “unlovely in manner and speech, possessing no artistic talent of any kind”, trying and failing to take his name off the credits.

“We paid him back his money, which was in a way the most stupid thing we ever did at Merchant Ivory,” Ivory says. “I think it was four million dollars. Harvey was, well … he was not my favorite distributor. I think he was a bully to everybody, but don’t get me talking about that kind of thing because I will be sued in two seconds.”

What does he make of one of Weinstein’s convictions being overturned? Bizarrely, he says: “I wasn’t sure why he was convicted anyway, but I don’t want to waste precious time talking about Harvey Weinstein, I can tell you.” Despite not being aware of the details of Weinstein’s conviction (for rape and sexual assault), Ivory is clearly not a fan. Later, when talking about aging, he describes him as his “enemy” and says: “You ought to feel exhilarated that you’re incredibly old and all your enemies like Harvey Weinstein have died — except that he is still alive.”

Ivory is relatively upbeat about aging. “The only problem is I can’t figure out whether my 96th birthday picnic should be on the Friday, which is my birthday, or a Saturday or Sunday,” he said when we spoke in June.

The corset-bound heart: Helena Bonham Carter in A Room with a View.

“One is amazed that one is so old,” he continues. “One of the things is that almost everybody you know is gone. I have only two remaining friends from high school and college days. People I remember as young are now in their seventies. But here I am, I feel good, there’s nothing wrong with me, I just go on.”

Death doesn’t frighten him. “If I became terribly sick and every day was terrible to get through I would have other feelings.” What’s his secret to such a long life? “I have had incredibly good health my entire life, which I put down to two things. I sleep well and I’ve always had a good appetite and I’ve been happy. I like a drink also.”

He’s still working. “I acted in a friend’s film recently,” he says, proudly. “I’m pretty good — it’s a French film called Her Song and I play an old man.” He laughs. “It seems like every day I have to do something or the other. And I write. I had an idea recently that I wanted to write about living in Palm Springs when I was 13.”

His family spent a year there because his mother had sinus trouble and his father, who worked in the sawmill industry, thought the desert would be better than another winter at home in Oregon. “My previous life had been being taught by nuns at Catholic school. Palm Springs meant I was plunged into this worldly junior high,” he says.

Ivory was adopted as a baby. He later discovered that his birth parents were from London and Dublin — but he rejects the idea that his films have a British sensibility. “I’ve made how many films? Around five are English, I’ve made a lot of American films, a whole slew of Indian films, as many French films, ones in Argentina and China. That’s a thing I really don’t like is that people don’t think about our other films. I wish they did.”

He thinks he first went to the cinema aged five in Oregon and remembers seeing Gone with the Wind when it came out. “The nuns in my Catholic school said when Rhett Butler uses the word damn at the end we should cover our ears.”

His mother died aged 63. “She didn’t live long enough to see my films, but my father was very pleased; he liked them very much and always encouraged me. He bankrolled a lot of stuff that wouldn’t have happened if it hadn’t been for him.”

What does he watch now? “It’s hard to see wonderful foreign-language films any more — all the little movie theaters that used to show them are closed. I can’t believe that’s happened in New York. I saw Oppenheimer. I didn’t bother with Barbie.” He defines period drama broadly — he’s not interested in Bridgerton making the genre more representative, but instead wants to talk about The Zone of Interest, about the family of an Auschwitz commander living just outside the camp. “You’d have to call that a period drama. It was a terrific film.”

And how does he think his own films have dated — would he have done anything differently? “I have a number of friends in England who are upper middle class, even aristocratic, and when they speak, they close their eyes. We should have done that with the James Fox character in The Remains of the Day. It’s very rare though that I see mistakes.”

His advice for film-makers is simple: “It has to be the principal thing in your life, keep pushing.”

Susannah Butter is the executive editor of Culture at The Sunday Times of London