Not necessarily bad. Some nodes (I think) don't respond to the pings. If there was something really problematic in terms of delay, you'd likely still capture it if I understand it right.
If that's working okay and similarly on both, then you can move on to looking at other things. It could be the site itself, but you shouldn't get the variation you are if it is. Maybe some ad is killing things. If you open the developer tools (chrome or firefox), you can reload the page and see what's taking the time to load.