So rotation or bullpen? Issue 1: sure Johnson started in the minors, but that was over 4 years ago. And, he never threw more than 156 innings in the minors. He'd be on a strict pitch count to say the least. After throwing 91 innings last year, it's conceivable that he could go somewhere between 120 and 150 this season depending on health.
What about performance? Well he struck out only 5.74 batters per 9 innings last year, so he would likely be a contact pitcher. Also, he would have to rely on his secondary stuff rather than the fastball heavy repertoire he's leaned on lately. Yes, he has 3 good pitches. Yes they could play in the rotation. But they would all likely take a hit, with his fastball velocity likely dropping from 95 to 92 (where it was when he was starting).
At the end of the day, Johnson holds more value to the Orioles and other teams as a reliever. Potentially a very good closer. While I don't blame the Orioles for considering it, I doubt he makes it out of spring training in the rotation.