If you're layering a shirt under a sweater, a slender blazer or a fitted lady jacket, you'll want it to be more fitted. Or even in a stretch fabric (I think stretch fabrics can look frumpy if they're the one layer).
LTKStyleTip LTKWorkwear LTKFindsUnder100