The cap is as much as you can stack. A player can have up to 32k effective resistance and even go above that to counter your penetration.

Edit: You also will not know how much penetration the player target has since many players do not play tanks so they will likey not have 32k resistance.

You can "over penetrate", and that's about it. Resist cap is 33k something, which is 50%. Then, you can always decrease their resistances, by Minor/Major Fracture and Breach and Crusher enchants. In theory, you can even use Alkosh too. Think you can get like 12k just from debuffs/enchants(?), then you have 20% resistances "ignored" if using a 2H maul, and if you use a Sharpened one you have another like 2750 or whatever.

In addition, you can also wear sets like Spriggan, New Moon Acolyte, Titanborn, Kra'gh and (soon) Balorgh. Finally, you can put CP into "piercing". All in all, you can pretty much get a 33k "resist capped" enemy down to almost 0% I guess, never done any hard research on it. Anyone who says "there's no reason to go above resist cap in PVP", have no idea at all what they are talking about. I even use (spell/physical) resist potions on my PVP tank (with well over 33k resist), and I can easily say it makes a big difference. These gives an unnamed resist buff, which stacks with major/minor resistance buffs.

So: There's probably a penetration cap of sorts, as there's a limit to how much you can ultimately stack, but generally you can only "over penetrate", which btw. is kind of unlikely for a normal build - but there's a breaking point, where the returns from further pen is minuscule, and better invested in weapon damage etc.
As long as you don't lose out on too much else, there's no such thing as "too much resistance" in PVP, the effective mitigation will be a lot lower than you think, especially when fighting multiple enemies, and in particular if you can't cleanse negative effects/debuffs.