Last links and build then I need to get back to work. Found this thread detailing each aspect so make of it what you will but basically you want a high speed single core due to the bottle neck of DMA direct memory access between the GPU and CPU. You also want at least as much RAM on the MB as your GPU’s have. So a 1080ti has 12 you want 16 on the mother board. So in this instance you want your MB to scale with GPU’s if you add any the MB selected here can go up to 128GB. HDD’s pose a bottle neck also so NVMe ssd’s are in order and this MB can hold 3 NVMe HDD’s total.
This is beginning to make more sense on how our AI servers are built at work. We have large farms of regular SSD’s but all the drives in the actual computer servers are NVMe drives. So basically the entire work load is transferred from 3.5" disk drives or SSD’s to the NVMe drives when the compute is done it transfers it back and starts the next batch. File used often are on the SSDs and the files not used that often on slow disk drives. May want to make a similar setup with this machine on a scale that fits your budget.
If I was to build one for myself here is what it would look like.
I changed it from a 1080ti as you can get a Titan X for about the same price currently and get better AI performance as it was intended more for this use case than gaming. Titan V would be better but twice the price.