I agree, it's frustrating when just removing the soundcard/audio streaming element from VC would help immensely (I say 'just', perhaps it's not that simple). Making VC a control-only VST would fix so many issues.
After recently installing Win 10 I thought I'd try plugging the USB in and seeing how it fared, but after half an hour VC decided to go haywire, spitting MIDI notes out at random. Nope -- back to MIDI and SPDIF for me.