The use of samples or digitally recorded audio in modular is more of an adjunct than a method for sample playback. You'd have less expense and more success using something like an Akai MPC Live MkII. It will allow for you to load libraries of sounds as well as interface with the modular synthesis world.

That looks like an awesome machine, but seems to be more beat oriented? Doesn’t look like it has the exact tools I’m looking for, although would probably be great for sequencing modular etc.
-- ethanlawrence

It's used by a lot of beat makers but it's not limited in that way. When you speak of loading libraries of sounds, especially orchestral, some simple Eurorack sample playback module isn't going to cut it. I'm still chuckling. There would be so much manual set-up to get anything decent. The Eurorack sample playback/recorder hardware is usually very barebones. They aren't going to playback anything close to standardized multi-sampled kits like Kontakt etc. The polyphony is going to lack as well as any tools for management.

Using a standalone hardware sampler that lends itself to modular is going to work a lot better than sampler purpose built for modular.