Well personally it didn't offend me either. It's baseball and he was talking about baseball players, none of whom are women. I didn't think it was sexist at all. Your work situation is slightly different as you work with women but I still wouldn't be offended. Frankly, I think you're a little girl for thinking like that.
Now I get why you don't think Stoeten is an SJW. He might not be but he definitely buys in to that s*** at least to some degree. And so do you.