To "fix" your system there aren't any small adjustments to be made. That's the point we're trying to express. To make your proposed system worthwhile in the limited amount of space really isn't workable. We have to add functionality and there's just no space to do that. A larger case would make it possible. Until there is more space, you're not going to get anywhere.
For what you have here... you'd be better off buying a semi-modular synth and a simple desktop sampler. It will be less expensive and complete.
If you're going to do this all in the confines of Eurorack, you're going to need a bigger budget and a bigger case. Sorry to be that blunt about it. But there's no making this build work with just a couple of tweaks.