My son was having the same problem but then I totally powered down his xbone by unplugging it for 30 seconds, then plugged it back in, and then he got into game with no wait.
@Rokal you don't have to power cycle. Just exit out of the game. Hit the nexus (home button), press start then quit. Restart the game and that has always worked for me.
Lol oh well, I guess. 9/10 times I would say try reinstalling the game to see if that issue gets fixed but if you have a digital copy, that might not be a viable option.