Depends on what you put below and in what situation you use it.
Personally, I also use a lot of wool for everyday use (the classic buisiness casual merino sweater and such, and those tend to be rather delicate) and I always combine them with a cotton T-shirt or dress shirt. The latter also works really well (looks wise) with some ice breaker 260 wheight /out/ sweaters.
But you're right, for /out/, cotton isn't the smartest idea, but a lot of the more robust woolens do survive the washing machine, so I also have a few woolen T-shirts to put under the mid layer. They're also fast drying (since they are really thin) so that's an added benefit, so I can wash them when I'm out for longer times. Or I just use some of my remaining poly t-shirts (though those are getting fewer) as a base layer.