I think it matters. The amount of error in a hand-tweaked knob will vary over time while a decent LFO will be relatively stable over time. I think we can sense that error (if not hear it), and that error is what I think contributes to interesting sound/music. They say guitar playing is all in the fingers. Keith Richards' feel will never be as precise as Steve Vai, but I know who I'd rather listen to. Of course no one else cares, but that's what makes something interesting to me.
Barring random and chaos modules, which can be unmusical in some uses, If we all have the same precise utilities, the same LFOs, Maths, Kinks, etc., there are only so many original combinations to make an interesting sound. What ultimately differentiates any of this music then? I would say the human error is what differentiates it all. Separating into utilities-are-more-important-than-oscillators (or vice versa) "camps" sort of takes away from the OP's fun thought exercise.

-- farkas

well I do agree with you - but you can always mix in a little random or chaos with other modulation

& I'd definitely rather listen to Keith play guitar than Steve Vai or any one like that

no one type of module is more important than another - but in most cases you will get more out of fewer sound sources and a decent selection of other modules, than out of more sound sources and fewer other modules

I also think it massively depends on how big your rack is - the smaller it is the easier it is to play it manually - the bigger it is the more the need for modulation - and as I said I mainly use modulation, but I also tweak things over time - but not to the extent I have seen a lot of performers - some of whom appear to be just touching knobs as opposed to actually wiggling them

a lot of the time I have the modular running whilst I do other things - so it's difficult to tweak constantly

"some of the best base-level info to remember can be found in Jim's sigfile" @Lugia

Utility modules are the dull polish that makes the shiny modules actually shine!!!

sound sources < sound modifiers < modulation sources < utilities