How often do you switch off your PC? Ideally it should be every day, but of course plenty will take a different view. In your circumstances, rebooting the machine before playing ESO would seem to be a solution. However, an upgrade may also be worthwhile, what are the specs? What else are you running at the same time as ESO? Are you launching it direct or through Steam?
There are the common userSettings.ini performance tweaks you could go through, but if your issues are mainly around game start up, it's more likely you'll be better off looking at system tweaks.
ESO is heavily CPU bound, so theoretically speaking, anything that lightens the load on your CPU will benefit the game. One thing a lot of people will recommend for gaming in general is to end any non-essential windows services from running in the background, a nifty tool for doing that is MZ Game Accelerator (other similar tools are available in both paid and free versions). I also run a batch file to set the priority of the executable, you can do this manually from the details tab in task manager, but I much prefer running a command line (fewer clicks). The command to save is:
wmic process where name="eso64.exe" CALL setpriority 128
Run the batch as administrator (after eso has been launched)
Finally, I know people say it often, add-ons, yes, add-ons will affect your performance. Selectively choose add-ons to use on a per character, per activity basis is the best way round that.
Edit to add:
additionally, you mentioned this happens after you've been playing a game recently before launching ESO. Could be you have some old tasks or memory held up from whatever that game was. A simple shortcut can resolve that by purging and resolving any idle tasks:
%windir%\system32\rundll32.exe advapi32.dll,ProcessIdleTasks
Just create a shortcut to that or add it to your set priority batch.