If you activate Tab Targeting by pressing Tab, you can keep pressing Tab to cycle through the available targets until you get the one you want, but I don't know of any way to turn it off again or disable it from ever happening in the first place if you'd just accidentally pressed Tab.

As far as automatically aiming at targets without Tab-Targeting them, I think the game tries to target the enemy closest to wherever you're looking, within reason-- that is, if the enemy is too far out of alignment then you won't target them-- but if there are multiple enemies clustered together then you can end up attacking an enemy other than the one you were trying to aim for. Unfortunately, I think all of that is out of your control and there are no settings relevant to it, although I may be wrong.