First - I think you should exclude extra innings from the discussion. This is about the average 9 inning baseball game.
Secondly - From what I've read, it's more about the "pace", not the total length of the game. I LOVE BASEBALL, but personally, I'd love to see an increase in the pace of the game. I've always loved watching guys like Buehrle get the ball and throw it...keep things moving. Nobody here can tell me they enjoy pitchers who slow the game down with runners on base, or batters stepping out to adjust their batting gloves again, or trips to the mound by the catcher or pitching coach. IMO, removing/restricting that crap from the game would increase pace and have little to no impact on the game itself.
Personally, I do notice and hate how long playoff games take. They all start at 8 or 8:30 these days and most run past midnight. I obviously sacrifice sleep to watch all Jays playoff games, but if Texas is beating Houston 10-6 in the 7th and it's 11:00 - I'm going to bed because I have to work the next day. I would watch (or finish watching) more playoff games if they could end at 11 or 11:30 (max).
I don't need/want games to be 2 hours long, but I do support efforts to get the average game below the 3 hours. Speed up pitching changes, restrict trips to the mound, speed up video review, etc. Increasing the pace of the game speaks to everyone IMO - not just to the casual fans.