Normally, I wouldn't suggest doing a build in 84 hp like this, but that would be for full-on systems. This fits better in the "mission specific" build category, though. Let's see here...

(time passes, then...)
ModularGrid Rack

OK...this is a fairly radical reworking of your original. Note that I didn't go with the mic pres on either end; you don't build these systems for looks, but functionality...and having these in the same place (with the new supporting modules) makes far more sense. So they're on the left end, and each one is paired with a buffered mult and an envelope follower. That last addition is super-important, as each one extracts gate, trigger, and amplitude information from the incoming audio, and these can be used in a number of points in the build for modulation, start/stop signals, etc.

As for the buffered mults...yeah, normally I would say to not put mults in a build that's this small, but I envisioned this as a parallel processing rig, and those buffered mults are critical to equally splitting the incoming signal. Once split, the signals (and this is a wholly stereo device) are sent to:

Twist: This is a clone of Mutable's Warps, handles lots of different timbral transformations; see here for what it does: https://www.modulargrid.net/e/mutable-instruments-warps

Beads: Mutable's updated (and soon to vanish!) version of Clouds, a granular processor.

Lubadh: Your stereo loop/tape delay.

FX Aid XL: More standard effects such as delay, reverb, and the like.

So, instead of being forced into a series arrangement (although, this being modular, you can patch it however you wish), you now have the capability of all four processors receiving and transforming the SAME signal. So, right after the FX Aid XL, you'll note that there's a 4-in stereo mixer. This lets you control your composite mix made up from the processors. And given that you can easily tinker with the wet/dry on the FX Aid XL, you also get a thruput from the preamps. Then at the end, since this is going to see field use, use with electric instruments, etc, I put in one of Happy Nerding's Isolator 2022 modules for a balanced output and your headphone preamp.

I removed several things...the Sisters VCF (the Twist does much the same thing...and FAR more, such as a 20-band vocoder), the Strymon interface (this will be dense enough...no stompboxes needed!), the quantizer (not really necessary, as you'd noted) and the Pam's (you can clock this...if needed...with the clock gens in your other devices) in order to implement the other modules, but from my experience with audio processing, this PLUS the parallel architecture will result in far better and more controllable results.