Get the dwemer mount; then you're not holding the compass. Problem solved!
ALL HAIL SNUGGLORR THE MAGNIFICENT, KING OF THE RNG AND NIRN'S ONE TRUE GOD! Also, become a Scrub-scriber! SJ Scrubs: Playing games badly to make you feel better about yourself.
My only question would be for why you have the mount in the first place if you don't like the compass. I'm not against the idea of being able to disable the effect if one so desires, but I don't find it strictly necessary either. I personally really like the compass effect, but to each their own.
Alternatively, if you want a mount without the compass, just use one of the normal mounts instead of an Apex variation. Problem instantly solved.