I just started playing ESO few days ago and once I got to Mistral in Khenarthi's Roost I realized that there are some serious performance issues in this game in dense areas, so I decided to investigate as I always do in games that are not running properly and leave some results as small performance guide here.
First and very easy thing to do is to determine what is causing performance issues. For strictly performance issues not related to servers or your connection, they are two main problems that can happen: not enough system resources, when your components, mainly CPU and GPU, are not able to drive the game at satisfying performance on settings you choose, or various optimization issues, like impaired/old engine that is unable to utilize your system resources properly and basically cannot even hold itself.
To check what is going on, we only need to do two things:
First thing to do is to test the game with underclocked GPU to determine if lowering your graphical power will affect performance. If yes then GPU is limiting factor and you need more powerful GPU to get desired performance at given graphics settings. This is more optimistic scenario because at least you have some control over performance and you can improve it by upgrading your setup.
However, if lowering your GPU power by underclocking is not affecting performance, then it means limitation lies in either your CPU or game engine that is unable to utilize your CPU properly which makes it unable to deliver enough information to GPU to for it to be fully utilized.
To determine whether its engine or CPU related issue, assuming that we excluded possibility of GPU limitations before, we need to disable some of CPU cores and test the game with different core count to see how many cores it can actually use.
If the game is using all your cores properly and you can see big decrease in performance from each cores, then it means that your CPU is not powerful enough and it is causing performance issues. From this point you can either overclock your CPU or upgrade it to improve performance.
The least optimistic scenario is when the game is not utilizing your CPU properly, and for example there is no difference between 2 and 4 cores. This means that the limitation is in game engine that is not optimized and designed properly and cannot utilize your CPU properly. This means that you are very limited in terms of max performance you can achieve, regardless of how good your CPU is, because it just won't be utilized, and usually if CPU is not utilized properly, neither is GPU, so you are loosing twice.
GPU Limitation Test
To check if there are any GPU limitations I have clocked and tested my R9 Fury at 4 different performance levels, shown by Fire Strike benchmark graphics scores:

1100MHz is slightly overclocked profile that should give peformance similar to overclocked GTX 980 and GTX 1060 6 GB.
800 MHz profile is achieving performance similar to GTX 970 and R9 390.
650 MHz profile stacks up with R9 380X in terms of performance.
525 MHz, lowest possible on this GPU, is matching GTX 960 and R9 380 performance.
Now once we have very different GPU power levels available, we can try them out in game and see how lowering the power of GPU affects performance.

As you can see, it is obvious that are no GPU limitations on my system. Significant decrease of GPU performance from downclocking the card from 1100 to 800 MHz gave no difference in performance. Only decreasing performance by almost half with downclocking to 650 MHz gave some performance decrease.
In conclusion, there are obviously no GPU limitations and we can move on to CPU/engine tests.
CPU Limitation Test
In this test we are going to disable CPU cores and see how the game scales with one, two, three and full four cores.

As you can see there are obvious engine limitations here and game is just not utilizing my CPU properly. Single core gives horrible performance, enabling second core gives boost to 53,4 FPS with 44 minimum and enabling third and fourth core gives only slight improvement versus dual core and there is no difference between triple and quad core setting.
In conclusion, the game engine is a limitation here, making the game unable to utilize quad core CPU properly, making use only of two cores and slightly third one.
Fixing The Performance
As you saw above, we encountered CPU limitations caused by unoptimized game engine that cannot be helped by any hardware upgrade if you already own good one, so there are only two things left to do.
First one is to take the most out of the part of hardware that is actually utilized by game. In this case it is done by overclocking the CPU:

As you can see, even if your CPU is not utilized properly there are some frames to gain from overclocking. By default 4690K works at 3,9 GHz in Turbo Mode, so overclocking to 4,5 GHz gave very healthy performance increasement, from 47 frames that original 3900 MHz clock would output to 54 frames of 4500 MHz overclocked setting, gaining almost 15% of performance.
Now it is time to look at some graphics settings in game. I have tested Draw Distance, Water Reflections Quality and Shadow Quality settings.
Note: I wasn't able to catch any differences in Maximum Particle Systems and their Distance while benchmarking, but I remember that increasing Maximum Particle Systems to over 1400 brought me down to as low at 25 FPS after some time of wandering around Mistral, so better keep this setting at default or lower, especially if there doesn't seem to be any difference visible from increasing this setting.

As you can see Draw Distance setting has very significant impact on performance, this is by far the most demanding settings in game. There is nearly 40 FPS difference between the highest and the lowest settings. It is also essential for visual quality, but some adjustments and compromises can be made to gain performance as there is some nice flexibility and 100 settings draws things really far, it is very hard to notice any popping in, so downgrading this setting a bit is recommended as you simply cannot achieve good performance with 100 settings in dense areas with any existing hardware because of engine limitations.

Water reflections are also very demanding, assuming that you are near water, taking almost 15 FPS. What is interesting is that there is no difference in performance with Off vs Low and Medium vs High settings. While this is understandable for Medium and High settings, as I cannot see any difference in reflections quality between those settings, enabling reflections from Off to Low with literally zero performance cost seem quite strange. There is some substantial difference between Low and Medium/High, but considering performance cost, leaving it at Low is a good advice.
Shadows performance is affected by number of players on screen, and benchmarking all available settings introduced some inconsistencies as number of players on screen is changing dynamically as ESO seem to be very well populated, so I was able to benchamark only 3 settings properly.
Shadows are mostly GPU bound and it seems to be like that here too, as downgrading this setting from Ultra to Medium didn't give any substantial difference in performance. Only turning them completely off gave some significant results, unfortunately at even more significant visual quality cost.
Conclusion
In conclusion, unfortunately ESO continues the trend of MMO games being released on impaired and outdated engines that cannot utilize even mid-end hardware properly. However in this particular game you are given enough tools to improve your performance at reasonable visual quality cost. There is also no significant stuttering as far as I experienced the game so far, framerate is dropping quite heavily in dense areas but those are "smooth FPS drops", if that makes any sense. This is certainly not one of those games that are running completely horrible regardless everything, hardware or even settings.
In general, optimization is reasonably bad.