Oh, yeah...you CAN have too many oscillators! When it gets to a point that trying to keep the rig in tune with itself is becoming a joyless and impossible task, you've got too many oscillators. But also keep in mind that that bar WILL change as you gain experience over years, and as you build up a system that fits your own expanding musical needs.
From my experience, I'd say that seven VCOs is a good stop-point. And this is from my own experiences of using my Digisound 80 (four oscillators) in tandem with my ARP 2600 (three oscillators). A good example of what results from this can be found in one of my works, "Beneath Puget" (https://daccrowell.bandcamp.com/album/beneath-puget); on that work, the only other "voices" consist of an MS-20 doing something very static, plus the "intrusions" from the theremin. It's also worth noting that this recording was "live in studio"...one pass for the whole thing.