I dunno about that. The casuals abandoned it, yes, but not the die-hards. The casuals were fed up of watching an AAA team getting crushed by the rest of the NL East every year. Can't really blame them-- most fans that show up to ballgames (in non baseball towns) are casuals, and we've noticed that only winning and other free stuff brings casuals back to the park.
I think a new stadium downtown with a competitive payroll would work-- it would be better than in Tampa, I can assure you that.