Yankees ticket prices through the box office are extremely expensive. Always best to just purchase on StubHub or Vivid Seats. Also, most tickets behind home plate or the dugout are probably corporate owned I imagine.
As well, I'm sure a lot of New Yorkers in the summer months are away at the Hamptons or on vacation. Being in NYC during the summer is dreadful. Would not want to sit outside during an afternoon game anywhere on the East coast lol.
Yankee Stadium is still a bit of a commute from downtown. Not much around the stadium as well, since its in the Bronx. It's not like Fenway or Wrigley that is surrounded by a bunch of bars/restaurants etc. As well, I don't consider Yankee Stadium one of the nicest ballparks in baseball. It's still decent, but nothing special IMO. Citi Field is a lot better.
Another reason could be because Yankees fans are pretty much spread throughout the US and Canada. Not all live in NYC. Thats why when the Yankees visit another ballpark, they're swarmed with Yankees fans.