I looked around before posting - the App Nap trick is a good start, I was dealing with speeds from 100 - 300 KBps to as high as 1 MBps if I was lucky. After disabling App Nap I am able to get 12.68 - 13.68 Mbps.
However, the speed has fallen to around 3 Mbps, then jumped back up to around 7 - 10 Mbps, only to fall back to 5 Mbps, even as low as 2 Mbps.
With all other games I get a solid 13 - 15 Mbps download, never dropping below around 12 - 13 Mbps. I use wired cat6 cable and have 181 mbps download speed. I understand this translates to 22.6 Mbps, and that I can't expect 22.6 Mbps sustained... but lows of 1 Mbps are pretty frustrating.
This seems like something the devs should fix, even the App Nap trick shouldn't be necessary, I've never had to do that for any other games.