Maybe put it in bug report section, they read bug report thread titles a least, but whether they'll fix it is another matter.
On PS4, it's the same button for back out menu and decline as well, but I no longer press that button, I click start button instead so it bring me right out of all menus, even quicker.
Try all other buttons too if you are on Xbox, you might just found a better shortcut for back out all menus.