Spent grains used for bread baking--what's been your experience?

I finally took the plunge and had my first homebrew experience last week. (Technically, it was more like an apprenticeship, as I honestly didn't know what I was doing most of the time. But my buddy was a great teacher and I had a fantastic time and now I can't wait to do it again.) I bake bread, make my own pizza dough and thought that it would be cool to incorporate some of the spent grains into the mix.

We used twenty-something pounds of malt, which I understand is typical. That's far more than I had any idea what to do with. (Choppy phrasing, but you get the point.) Even with the couple of quart bags I took back we composted most of the grain.

While the grains cook and soften there is still the problem of the husks. If you bake with your spent grains how do you ensure the final product isn't so... woodsy or fibrous? Personally I don't mind but I have had it made known to me that the bread I've baked is ...um, noticeably "rustic" to the point of being a bit too much. Everyone's a critic these days!

Even if I dialed back the percentage of spent grains in a particular recipe I'm not sure what to do about the husks and whatnot that make it in there.

Love to hear from the bakers and brewers who bake out there! Thanks.