This isn't a bad idea. I've done the same thing with my OWB holster, except I use modified socks. They don't interfere with drawing or re-holstering and I can change em to match whatever I happen to be wearing. The holster sock can have a lot of options for color, texture and "fluff"... A thick or fuzzy sock can breakup the outline and soften printing by a good amount. Socks are already made to match clothing, pick what you like, snip em down and sew em up, make sure there's no snagging (good elastic is all it needs). Now your holster no longer stands out if it peeks out.
I do not love the bright sword for its sharpness, nor the arrow for its swiftness, nor the warrior for his glory. I love only that which they defend. -- J.R.R. Tolkien