It's almost certainly a combination of random variance, mental variance, and other factors. Negative stretches could be affected by things like minor injury, fatigue, or soreness. The eye level of the batter combined with arm height and angle of the pitcher could place the ball coming out of a different location of the batting eye. Some batters react to weather differently (I assume), etc. It's very difficult to determine what is random variance, and what is caused by variables you haven't considered in your model (or perhaps variables you simply don't have access to).