At the end of the "Combray" section, the narrator says that his mind returned
by an association of memories, to what, many years after leaving that little town, I had learned, about a love affair Swann had had before I was born, with that precision of detail which is sometimes easier to obtain for the lives of people who died centuries ago than for the lives of our best friends.And so Proust excuses the fictional device of recounting "memories belonging to another person from whom I had learned them." Yes, it's a cheat, but all fiction is a cheat.
"Swann in Love" begins as a comedy of manners, with the introduction of the Verdurins and their "little set" or "little circle" or "little clan," which consists largely of "a person almost of the demimonde, Mme. de Crécy, whom Mme. Verdurin called by her first name, Odette" and a "former concierge" who is the aunt of a pianist under the Verdurins' patronage. The Verdurins' "little set" is just that: a microcosmic society mimicking the larger social set from which they were excluded.
So Swann, who moves in the highest circles, is something of a catch for the Verdurins. He is a dilettante who "had wasted his intellectual gifts in frivolous pleasures and allowed his erudition in matters of art to be used to advise society ladies what pictures to buy and how to decorate their houses." His Achilles heel is his susceptibility to women. "And though Swann was unaffected and casual with a duchess, he trembled at being scorned by a chambermaid, and posed in front of her." So even though Odette de Crécy was not his type, he falls for her.
Her profile was too pronounced for his taste, her skin too delicate, her cheekbones too prominent, her features too pinched. Her eyes were lovely, but so large they bent under their own mass, exhausted the rest of her face, and always gave her a look of being in ill health or ill humor.
Moreover, she knows nothing about the art he so admires, including his "unfinished" (i.e., abandoned) work on Vermeer, that he keeps using as an excuse for not visiting her: "'You're going to make fun of me, but that painter who keeps you from seeing me--' (she meant Vermeer) 'I've never heard of him; is he still alive?'" The way Swann is drawn into the trap reminds me of Lydgate being snared by Rosamond Vincy in Middlemarch.