It's always a good idea to understand that a "modular fx rig" is a modular synthesizer minus it's in rack sound source
I would look at more utility modules - the plumbing - the (relatively inexpensive) dull polish that lets the shiny modules shine brighter - I personally don't count Maths as a utility module - although it can be used as such - because it is so powerful
I always recommend Mutable Instruments Links, Kinks and Veils as a good starter set for this in a relatively small space
Expert Sleepers Disting mk4 would also be a very useful module - as would Ornament and Crime (both are swiss army knife modules)
A matrix mixer and sequential switch may also be useful
"some of the best base-level info to remember can be found in Jim's sigfile" @Lugia
Utility modules are the dull polish that makes the shiny modules actually shine!!!
sound sources < sound modifiers < modulation sources < utilities