1. What metrics do you use and rely upon consistently?
Looking into pitching performance can be a bit trickier (or at least seem a bit trickier) because of all the different ERA estimators out there. You have ERA, FIP, xFIP, SIERA, DRA, TIPS, and many others.
None of these are straight up better than the others. Some of them are better in certain circumstances, though. The key is to understand what they do and when you might want to look at one thing rather than another.
Over a very large sample size (many years) all you really care about is ERA. Yes, ERA. Run prevention is the whole point. The problem with ERA is that it can be influenced heavily by things outside of a pitcher's control, so there are better things to look at as you shrink the sample size. For a season, or maybe two seasons, it's probably better to look at FIP. If you think a pitcher had a bizarre (lucky or unlucky) HR rate against them, use xFIP instead. When comparing ERA to FIP or xFIP, you'll want to glance at BABIP and LOB% as well, so you can get the whole luck picture.
In VERY small sample sizes OR when you are trying to figure out if there has been some sort of skill change, you need to get more granular. Velocity, velocity trends, pitch mix, spin rates, horizontal/vertical movement, whiff rates, called strike rates, and even video of pitching mechanics can all be worth exploring.