The altitude is the whole explanation. There is less atmosphere overhead than anywhere else you will photograph in Spain, and almost no humidity to scatter what gets through, so the sun arrives as a point source: shadows with edges you could cut yourself on, and a gap between the lit and the unlit side of a face that no sensor handles gracefully. In July, between noon and seven, this is simply not a city you can photograph people in outdoors, and pretending otherwise is how a session gets sold and then quietly ruined.
The same physics is a gift for five months of the year. From late October the sun never climbs high enough to do that damage, and Madrid gets something the coast never does — a whole short day of low, clean, raking light, from about nine in the morning until half past five, with no haze in it. A December session here has no bad hour. The other structural fact is that the centre is walkable and the good things are close together: Retiro to Cibeles to Gran Vía is thirty minutes on foot through three completely different cities.