And I decided to sit out such a loading screen.
The result:
After the inactivity time out ran out the loading screen of the battle ground was replaced with a Craglorn loading screen and I was kicked to the PVE area immediately. Well, that worked flawlessly...

In Craglorn I found out that I had the deserter's penalty and at the same time the game was bugging me, that the battleground was ready. So I tried to enter the battleground and was kicked to the log on screen immediately.

Has anyone experienced an infinite load screen while loading into a BG, but has been able to hear the audio of the ongoing BG and sound effects from your own character, while still just seeing the load screen?

I've had this happen a handful of times on Xbox. Most recently when it happened, I was able to get voice chat going with a player in the BG and he confirmed that my character was moving around and jumping and activating abilities as I pressed buttons. I went as far as to get his assistance in walking my character off the ledge of the spawn area and into combat to have my character killed to see if that would somehow fix the load screen. It did not, my character died and I could hear the sound effects of the death recap, but then I couldn't activate the respawn. It was all pretty messed up, the only Silver Lining being the fact that, with all of the activity, my character was in the BG till it ended and I didn't get the penalty.