In a shop selling lobster pâté on the Île d’Yeu—a 12-square-mile patch of beach and whitewashed houses off the French west coast—you can buy a can of local air. That’s how much its devotees love the place. But it remains somewhat of a secret. Even in France, few could locate Île d’Yeu on a map. (It is near the more well-known Île de Ré, off the Vendée.)

Port-Joinville, the island’s quaint port town.

Yet it attracts quite a set. The Belgian king and his family have spent summers here for more than two decades, recently buying a modest, tiled house amid pines. (By law, every house must be white.) King Philippe, Queen Mathilde, and their four children picnic and kite surf. Wall Street financiers and French actors—The Count of Monte Cristo star Pierre Niney, comedian Gad Elmaleh—mingle at the market in the hilltop village of Saint-Sauveur. Michel Barnier, briefly the French prime minister last year, is among the top politicians and journalists who go. However, “there’s no social hierarchy here,” a former mayor once said. “Nobody cares.”