Which video settings directly impact the framerate in ESO? I'm getting around 17 fps in the tutorial starter area when the lady helps you kill enemies.
But seriously, without further details on your setup and settings it's hard to give solid advice. If you're using the default video settings (i.e. those you get when clicking the default button in the video settings window), you should be good.
On the hardware side, make sure your video card runs cool and doesn't go into thermal throttling.
On the software side, texture quality, subsampling quality, shadow quality, and water reflections sure do have a major impact. Lowering them to the next lower setting should help a ton. View distance can have an impact, too, depending on your amount of video memory. Disabling features like anti-aliasing, ambient occlusion, and bloom should help with both performance and memory preservation.
But first and foremost, disable VSync (vertical sync), if it's on. It'll half your framerate if your card can't drive the display at full multiples of your refresh rate (60 Hz for LCD, usually).