I never saw the lure of a used baseball. I have an old bag of old baseballs in my basement... they are just baseballs.
I usually tell whoever is sitting next to me that they are in charge of any ball that comes because I'm not interested. I'm there to watch, not shag foul balls.