I'm in the other camp. The modulation sources and utilities are the "sauce" that makes the sound great (doing best Tony The Tiger impression).

If you want to swap out different oscillators/sound generators to get different timbres... cool. But it's the other bits that do the heavy lifting in great sounds.