I have a GTX 980 running in Ultra. My GeForce Experience software keeps telling me my game settings are not optimal and gives me a one click option to have it set the game settings for me. What it wants to do is put the game in full screen mode so that it can put the resolution in 3840-2160 and then down sample to 1920-1080, it says this provides a better quality image. I normally run in windowed (fullscreen) at the monitor resolution of 1920-1080. When I allow it to optimize the settings, I don't see any improvement in the perceived resolution, in fact some surfaces seem fuzzy, but the biggest draw back is the overall image clearly and excessively jitters as I pan around. The card is struggling to keep up with the down sampling when the scene is changing rapidly. Because of this I can't run it in this supposedly optimal setting.

This is an example of taking the best setting for the graphics card's features is not necessarily what's best for the game experience. You may want to try lowering the various video settings to see if one of them is just a bit too much for the system and might provide a smoother image.

There is also some jitter introduced by the inability to slide a mouse smoothly across a mouse pad. The camera setting Smoothing helps with this but it needs to be a slider where you can increase the effect because it is barely noticeable. You can turn down the camera rotation speed below 10 and get a smoother pan but you'll need a large size mouse pad.

I use a wireless Razor gaming mouse and like Sir Andy said, there are settings in the driver software for the mouse movement sample resolution. The effect is similar to the smoothing setting in ESO, it causes you to have to move the mouse farther to get the same amount of camera movement, which helps eliminate the little start stop effects of the friction as the mouse moves across the pad.