I don't think it's silly at all to make the assumption that players who played baseball, hockey, football, or basketball... when the sports were in their infancy, wouldn't be able to hold a candle to the athletes of today.
If you took Babe Ruth out of his era and put him in today's era with pitchers throwing 95 mph with insane breaking balls, changeups, and different pitchers not just every game but multiple different pitchers in the same game... it just wouldn't compare. He'd probably strike out like 70% of the time.
The overall skill level of sports back then was just completely inferior to what it is today. Pro athletes back then didn't have to be on a pro teams radar when they were 12 years old, compete nationally and internationally multiple times before they were even thought about being signed.