Looking at this, I see a lot of the typical "sexy" modules and scant few utility modules...attenuators, mixers, VCAs, logic (ESPECIALLY with sequencer-based systems!), modulation sources, and so forth. This really sells the sizable sampler section short, as you have sampler hardware here that could really use modulation. I would suggest tossing out the Poly and the F8R for starters, because these seem to not have the sort of analog modulation that would be useful. Could be wrong, though...but at the same time, the Poly looks a bit like a "Max in a box" device. Knowing how complex Max can get, I do wonder if something like that can handle a REALLY complex bunch of objects.
But yeah, these huge modules are really cramping what the system could do. If the idea is to create a multichannel sampling environment, it may make more sense (and be a lot cheaper) to hunt down a complex sampler from the 1990s/2000s and implement that instead. This is why I've got an Akai S6000 here...when all I need is straight samples, it's perfect, and if I want to modulate that, I can potentially send analog modulation to the sampler, although telling it what to do via MIDI can be just about the same level of flexibility.
Something like this: https://reverb.com/item/45970853-akai-s6000-midi-sampler That's pretty much identical to the one I have...18 outs, dual MIDI implementation, XLRs on the inputs, detachable control panel. Then once you have something of this sort in hand, get a floppy replacement from Gotoh or Nalbantov, and you can jam a MUCH bigger "drive" into the unit via USB drives or SD cards. Once the sampler's all set up, then use a module that can digitize analog signals for MIDI transmission to send all of the voice trigger/gates and modulation to the sampler. With that, you can chuck out ALL of the samplers in the build and focus on modules to drive the sampler, the CV-MIDI conversion, timing, some extra VCFs, modulation, etc. You could even use something like the FH-2 to send/return analog signals to your DAW, and then rechannel them to MIDI in there. Lots of possibilities...