Yeah if the Jays can't add a true front of the rotation starter like Cease or deGrom, probably it's best to double down in the bullpen.
Hoby Milner would be a nice lefty and has posted a 1.6 WAR thus far with a 2.17 ERA/2.12 FIP and 8.15 K/9.
One of Bednar/Duran/Clase/Jax/Bird would be nice.
If the Jays really want to add an arm who could start, would target someone who can flip back and forth between the rotation and pen. Mike Soroka could be a nice fit who could provide multiple innings out of the pen and start if needed in an emergency if someone goes down on the IL.
Mentioned Harrison Bader's name before, and also just read Max suggest him. Would be a nice fit here and upgrade over Straw and strengthen the Jays outfield defense even more. Plus as some have mentioned Varsho's name in trades, at least adding an OF like Bader would lessen the blow of losing Varsho if the Jays were able to move him for a top of the rotation arm like Cease.