What on Earth is going on with incremental patching?

LaucianNailor

@chris.dillman @ZOS_GinaBruno

I wonder if either of you could explain why fixing what appears to be some issue (1 or 2 key problems) appears to introduce multiple new ones. Let me explain:

My machine ran perfectly on 2.3 on the PTS (late 2013 27"iMac, 24GB RAM & 4GB Nvidia GTX780M Graphics Card).
Prior to the TG on PTS I was unable to run in Wrothgar due to pink textures and very regular (20 minute) crashing.

On actual Thieves Guild launch I was crashing every 4 - 10 minutes making the game unplayable. Chris PM'd me with a suggestion regarding extended App file limits and I have had an almost flawless experience for nearly a week (not one crash and no loss of textures).

Incremental hits live overnight in the EU (supposedly to resolve this file limit issue without having to launch from the terminal) and suddenly:

1. Neon pink items once again appear when this was already solved (for me). 2 characters in Coldharbour, 1 sees a horse at the stable normal, the other sees it a neon pink mess with a cross. Clearly the texture is on my machine and available, but the client seemingly ignores it with one character (others not tested in this location as yet)
2. Multiple times on logging in a character, no add-ons are recognised as being installed (all 12 were being seen prior to this). Logging out and in or using /reloadui is erratic. Sometimes they show up in the add-ons menu and can be used in game, sometimes the add-on list is blank and sometimes they show up in the list but cannot be used in the game - they appear ignored by the client.
3. I log into one of my character (Thando) and there is no world....start a falling animation that loops and despite logging out and back in again, this is still the same. After 2 further log-out/in attempts, he 'lands' near a way shrine and his add-ons don't work.I can't bring myself to log him out and in again incase he start falling again.

Why is this constant battle to fix 1 or 2 issues creating more issues and a general growing of disdain for the game. Please stop providing ammunition for the detractors due to a seemingly 1 step forward, 2 back repeating scenario. I'd prefer a reliable playing experience (and will continue to give you my subscription money whilst I do) rather than regular buggy, unreliable content that rather than extends my playing pleasure, simply drives me to frustration and potentially away.

KhajitFurTrader

@LaucianNailor, could you please try and move your entire Elder Scrolls Online folder in your profile's Documents elsewhere, so that the game can start fresh? Two of the issues you're seeing might be related to addons. I've read of numerous Windows users crashing right after character select today, turns out it was Master Merchant causing the issue. Maybe the Mac client handles these new kinds of addon issues in 2.3.6 more gracefully, but still incorrectly.

As for the pink textures, please try a repair from the launcher if you're still seeing them after a new Elder Scrolls Online folder has been created in the step above.

LaucianNailor

Hi @KhajitFurTrader

OK, so I've moved my ESO folder and re-launched the game to allow the configuration folder to be rebuilt. I've then run a repair (No indication that any files were re-downloaded during this process, so my assumption was that this was fine; this is also based on the mention above that exactly the same horse had a missing texture when on one character and was perfect on another loaded character).

I've only loaded the one character since the repair and are not seeing pink textures, which is a good start, but this was also the case immediately following patching on my first character I loaded up this morning.

Regarding the add-ons, I don't use and have never downloaded Master Merchant (I saw the post about this issue also) so am struggling to accept that the ones I use are the issue, especially given as they have all been updated (not out of date) and were all used flawlessly for about a week with no texture issues or crashing, between the work around to launch from the terminal and last nights overnight patch (including multi-hour play sessions).

I'll now drop back in all the add-ons I use, re-set them all up again and see if the problems I had immediately post patching reoccur.

Thanks for your suggestions

Pomaikai

What the HELL did y'all just do with this hot fix? I could play without crashing, but just got a notice to quit, update, and restart the game. Did so, and now I can no longer play ESO! I can get to character select, but the game crashes to desk as it tries to load into the game itself.


STOP BREAKING THE DAMNED GAME WITH UNTESTED HOT FIXES!!!!!!!!!!!!!!!!!!!!

Turelus

Pomaikai wrote: »

STOP BREAKING THE DAMNED GAME WITH UNTESTED HOT FIXES!!!!!!!!!!!!!!!!!!!!

Damned it they do, damned if they don't...

Pomaikai

Turelus wrote: »

Pomaikai wrote: »

STOP BREAKING THE DAMNED GAME WITH UNTESTED HOT FIXES!!!!!!!!!!!!!!!!!!!!

Damned it they do, damned if they don't...

No, damned if they DON'T TEST! We're basically beta testing these patches because they aren't tested on the PTS, or through a smaller but broad enough beta testing group to identify issues BEFORE they push it live!

LaucianNailor

OK, so an update:

I played on a character for just over an hour with no problems. however then.....

Dropped some gear in the Bank for one of my alts. Logged out and then logged the second character in (who was left in the Bank in Vulkel Guard). Loaded up with no land or building assets at all, so he started the repeated 'falling' until I logged out. This by the way, is a different character than the one who had the falling issue prior to the repair carried out above.

I really am losing the will with this!

KhajitFurTrader

Pomaikai wrote: »

No, damned if they DON'T TEST! We're basically beta testing these patches because they aren't tested on the PTS, or through a smaller but broad enough beta testing group to identify issues BEFORE they push it live!

Small, incremental patches like the most recent one have never been tested on the PTS beforehand, and I doubt that will ever become practice. Generally speaking, PTS is for testing new content like DLCs, not for verifying if bugs have been addressed in a particular patch that's going to be pushed on live. I'm glad they have such a high update frequency -- putting everything on PTS before releasing would be detrimental to it.

Also, since the version number has changed, all addons that had been updated to work with 2.3.5 are theoretically now outdated again since 2.3.6 hit live.


And good luck, @LaucianNailor!

Pomaikai

KhajitFurTrader wrote: »

Pomaikai wrote: »

No, damned if they DON'T TEST! We're basically beta testing these patches because they aren't tested on the PTS, or through a smaller but broad enough beta testing group to identify issues BEFORE they push it live!

Small, incremental patches like the most recent one have never been tested on the PTS beforehand, and I doubt that will ever become practice. Generally speaking, PTS is for testing new content like DLCs, not for verifying if bugs have been addressed in a particular patch that's going to be pushed on live. I'm glad they have such a high update frequency -- putting everything on PTS before releasing would be detrimental to it.

Also, since the version number has changed, all addons that had been updated to work with 2.3.5 are theoretically now outdated again since 2.3.6 hit live.


And good luck, @LaucianNailor!

I don't PAY to beta test for companies too lazy to do so on their own.

LaucianNailor

KhajitFurTrader wrote: »

Also, since the version number has changed, all addons that had been updated to work with 2.3.5 are theoretically now outdated again since 2.3.6 hit live.

Are you sure this is correct? My understanding is that ZoS push an approved API version (currently 100014), which providing add-ons adhere to, remains valid (up to date) until a changed API version is required. These updated API sets usually relate to significant patches (full point i.e. .X) and not incremental (part point i.e. .(n).X)

Therefore all add-ons written to run with 2.3 should not be flagged as out of date until 2.4 as no documented API changes have been made or communicated shouldn't they?

KhajitFurTrader

LaucianNailor wrote: »

Are you sure this is correct? My understanding is that ZoS push an approved API version (currently 100014), which providing add-ons adhere to, remains valid (up to date) until a changed API version is required. These updated API sets usually relate to significant patches (full point i.e. .X) and not incremental (part point i.e. .(n).X)

Therefore all add-ons written to run with 2.3 should not be flagged as out of date until 2.4 as no documented API changes have been made or communicated shouldn't they?

Well, honestly I don't know, and you may be right. But after repeatedly reading reports today that removing Master Merchant fixed crash issues after character load for some people, I guesstimated that sometimes, new game versions and existing addons don't mix well. It might be caused by side effects form bug fixes, or (undocumented) changes to the API. Me, I'd wait with enabling/using addons until their authors explicitly released them for the latest game version.

LaucianNailor

Interesting update and I'm not sure if this will help Chris.

I've had my only crashes since the workaround last week and had 2 back to back which happened as I pressed f13 to take a screenshot (of the returned missing textures).

Additionally, on 1st run, all appears to be fine for that character. Missing textures and add-on's being recognised, then vanishing only occurs following logging out a character and logging in a different one. If I close the client and load up the character I had the problems with, they are cleared. Not sure if this relates to some problem with unloading assets at character log-out or some memory problems when swapping characters?

KhajitFurTrader

LaucianNailor wrote: »

Additionally, on 1st run, all appears to be fine for that character. Missing textures and add-on's being recognised, then vanishing only occurs following logging out a character and logging in a different one. If I close the client and load up the character I had the problems with, they are cleared. Not sure if this relates to some problem with unloading assets at character log-out or some memory problems when swapping characters?

Good idea. Alas, I tried to reproduce it by logging through my characters, but no luck: I still have yet so see a single pink texture. If anything, the client runs stable and very, very smooth -- I'm getting 30+ FPS on ultra-high settings (Shadows tuned down to high, and HIGH_RESOLUTION_SHADOWS turned off) at 2560x1440 in very crowded urban areas. Same machine as yours (the amount of RAM above 8 GB doesn't really matter). This is a weird bug, I hope they can find the cause and squash it.

LaucianNailor

Further test this morning running through all my characters for stable training and writs @chris.dillman

1. Load 1st character into Coldharbour from client launch - all seems as it should be; no missing textures, all add-on's working perfectly
2. Log out 1st character and log-in 2nd in Coldharbour - missing stable horse textures (seen with no problem by 1st character), all add-on's functioning perfectly
3. Log out 2nd character and log-in 3rd in Coldharbour - same missing horse texture as above, all add-on's functioning perfectly
4. Log out 3rd character and log-in 4th in Rawl'Kha - some missing pet textures (guar), all add-on's functioning perfectly
5. Log out 4th character and log-in 5th in Eastmarch - all chicken textures missing, add on's functional, however icon to open Craftstore (has been fine up to now) has become invisible, although it can still be moused over and used. Transit to Shadowfen to hand in a writ - Missing collectable horse texture noticed. Transit back to Eastmarch and attempt to locate a survey location on the main map (using Lost Treasure add on) - indicator has not become visible on the main map, so carry out a /reloadui to see if this clears the missing craftstore icon and survey mark. All add-on's vanish from screen and no longer appear in the add-on menu for the character.
6. Quit and restart the client again with the 5th character - all once again perfect, missing chicken textures back, all add-on's available, CraftStore icon showing and survey location showing on main map.

In short, this degrading client behaviour has been introduced with the incremental pushed to EU a couple of days ago. the ZoS crash handler appears to be kicking in again, so you should be getting some reports rather than via Apple. As for memory, my peak use was just over 7000 (24GB ram in my system), so no OOM crash, but I'm convinced there is some issue with management of assets/add-on's that get borked through the character unload/load process.

EDIT: On my next play through, following the above, I swapped between a couple of characters and received an ESO alert window I've never seen before:

Pending data load error (399176 - c00000000003e2c)

It gave me options including 'debug' and 'crash' - I clicked debug, but the client crashed anyway.

KhajitFurTrader

Could you please take a look into your UserSettings.txt file? It's possible that the LUA memory limit has been reset to its default value of 64 MB. Look for the line

SET LuaMemoryLimitMB "xx"

where xx is the current value. Theoretically, this value could be boosted up to a few gigabytes now on a machine with ample memory, but 640K 512 MB "ought to be enough for anybody".

LaucianNailor

KhajitFurTrader wrote: »

Could you please take a look into your UserSettings.txt file? It's possible that the LUA memory limit has been reset to its default value of 64 MB. Look for the line

SET LuaMemoryLimitMB "xx"

where xx is the current value. Theoretically, this value could be boosted up to a few gigabytes now on a machine with ample memory, but 640K 512 MB "ought to be enough for anybody".

Good suggestion and spot...yes it has been (re)set to 64mb, so I've changed it up to 512. Not sure how this impacts lost textures, but to be honest I'm willing to give anything a try at the moment. Will report back after another session later

UPDATE: Sadly, further testing has shown this has had no impact at all I logged into and played 1 character including completing a delve, using the wayshrine and crafting. After banking some items I logged out and into a 2nd character. Furniture items in Mournhold were now neon pink (fine when i logged him out) and add-on's not visible on screen (though still listed and activated in the menu). A /reloadui left textures still missing and also the add-on's now missing from the menu list despite being there seconds earlier!!

Delpi

For me not only add ons had disappeared, some icons of the inventory and action bars are gone too

LaucianNailor

Further update:

Took the unprecedented step of rebooting my Mac, you know, just in case!! note: it can be ran for weeks between forced update reboots with no issues. I also ran a further repair on the client and there was no indication that any elements required downloading.

Just loaded up one character and played for a couple of hours. Zoning in to the Riften Outlaws refuge I lost the background image on my main map. The new icons showing the location of the fence etc where there, just no background image. As @Delpi mentioned I also lost a number of item images in the sell window (also missing when looking in the inventory).

It appears that multiple transitions on a single character not just swapping characters can also have a negative effect. Oh, and after closing and restarting the client; running in Wrothgar for a little while, I tried to log into my Banker character and once again received an alert widow with the option of debug or crash (different codes this time).

Pending data load error (307053 - c00000000008fe4)

@ZOS_GinaBruno @ZOS_JessicaFolsom Multiple issues encountered submitted to customer support and have pointed them to this thread as full background
Ref:160316-001076

DjSolJAH

LaucianNailor

Hey @KhajitFurTrader

Your post above re the 'set LUA memory' had me thinking about some of the errors I've been getting with vanishing add-on's etc. I set this to 512 and verified it had changed after I closed the file.

I logged out a couple of hours ago, but decided to have another look at the UserSettings.txt file in /Users/*****/Documents/Elder Scrolls Online/liveeu.

Interestingly, the value has been re-set to 0:
SET FOVChangesAllowed "1"
SET AgreedToPEGI "1"
SET LuaMemoryLimitMB "0"
SET GamepadSensitivityFirstPerson.2 "0.72000003"
SET GamepadSensitivityThirdPerson.2 "0.85000002"

This shows as being modified at the time I logged out. I wouldn't have thought this was normal behaviour and wonder if this is a cause?

KhajitFurTrader

Now that's interesting. Edit: And no, I wouldn't think of it as normal behavior, either. @chris.dillman, any thoughts?

Re-set its value to 512. Save the file, then make it read-only: in the Finder, right-click it, select "Get info" from the menu. In the new window, scroll down and expand Sharing & Permissions. Select "Read Only" for every principal (user or group) in the list. You might have to unlock the lock in the lower right corner by clicking on it first, before you may make changes (needs a user w/ administrative privileges).

capricorn152245ub17_ESO

LaucianNailor wrote: »

Hey @KhajitFurTrader

Your post above re the 'set LUA memory' had me thinking about some of the errors I've been getting with vanishing add-on's etc. I set this to 512 and verified it had changed after I closed the file.

I logged out a couple of hours ago, but decided to have another look at the UserSettings.txt file in /Users/*****/Documents/Elder Scrolls Online/liveeu.

Interestingly, the value has been re-set to 0:
SET FOVChangesAllowed "1"
SET AgreedToPEGI "1"
SET LuaMemoryLimitMB "0"
SET GamepadSensitivityFirstPerson.2 "0.72000003"
SET GamepadSensitivityThirdPerson.2 "0.85000002"

This shows as being modified at the time I logged out. I wouldn't have thought this was normal behaviour and wonder if this is a cause?

Did you change the file while the game was running? I ask because my understanding has been that the game client writes back to these files on game close, and so any changes you make if the game is running are re-overwritten (why, I don't know, it was something I had to deal with back in 1.5 or 1.6 or something).

capricorn152245ub17_ESO

Delpi wrote: »

For me not only add ons had disappeared, some icons of the inventory and action bars are gone too

I've been having this problem too. Additionally, when trying to resurrect someone, the bar just appears to be full the whole time rather than progressing up to completion.

LaucianNailor

@capricorn152245ub17_ESO

To answer your question, I wiped the settings folder to allow the game to build a new default settings file. For LUA memory, this is 64mb afaik. I amended this to 512mb prior to launching the game and so for it to reset to 0 on exit is puzzling.

My concern with this was that some in game event, say triggering /reloadui is changing this value and shutting down add-ons or at least causing some game anomalies and crashes.

capricorn152245ub17_ESO

LaucianNailor wrote: »

@capricorn152245ub17_ESO

To answer your question, I wiped the settings folder to allow the game to build a new default settings file. For LUA memory, this is 64mb afaik. I amended this to 512mb prior to launching the game and so for it to reset to 0 on exit is puzzling.

My concern with this was that some in game event, say triggering /reloadui is changing this value and shutting down add-ons or at least causing some game anomalies and crashes.

This is an again, I don't know why it works this way, but here's why it happened: you made the changes in a program like textedit or something, right? I get the same thing when I do it that way.

The only way I've been able to make the change stick is to do it through the command line, using a text editor like vim or the like, as follows:

sudo vim "filename"
press i to begin inserting text
go to the line you want to change and change it.
type ctrl+c
type :wq!
should save your changes then

Please let us know if this doesn't help.

KhajitFurTrader

Never had any problems with editing UserSettings.txt with TextEdit or TextWrangler, the changes always stuck. Don't know what happens here, but the game seems to overwrite the value on exiting.

LaucianNailor

@KhajitFurTrader

Hi, as per your suggestion, I changed the SET LUA in the settings file back to 512 and restricted access to read only across all users (I am the admin user profile, btw).

The game launched OK and I experienced the same as previous posts. 1st character was fine; ran writs, stable training and harvested a cloth survey from the writ turn in. opened 2nd character (Banker) so only carried out deposits/withdrawals without leaving the building. 3rd character then started to see pink creatures in Rawl'Kha, 4th character had all add-on's missing from play and also gone from the add-on's list (whether loaded or not) as soon as they appeared in the world.

I quit at that time and checked the settings file was still showing LUA at 512 (which it was). I expected to get a client crash or at least an error if the game tried to update the 'locked' file, so it doesn't appear as if it tried. I'll load back in and try a /reloadui to see if that tries to do anything untoward with the settings file.

Update: /reloadui doesn't appear to have attempted to change the settings (no crash or error), so I'll leave it locked for now and see what happens with ongoing play. Still, this seems as if the degrading performance/texture loss/add-on loss is still there....just related to something else

KhajitFurTrader

Hmm, I'm beginning to wonder whether this has anything to do with the addon settings now being available account-wide...

Delpi

I can't load any add-on anymore. I select them, reload ui and nothing happens. By the way, when I load another character the game crash with a message (the first time I saw this error with a 'crash button' made me laugh lol) I wonder if It's just a problem of mine or you have it too...?

LaucianNailor

Delpi wrote: »

I can't load any add-on anymore. I select them, reload ui and nothing happens. By the way, when I load another character the game crash with a message (the first time I saw this error with a 'crash button' made me laugh lol) I wonder if It's just a problem of mine or you have it too...?

I spotted this for the first time following the incremental pushed to EU server this week. I've quoted a couple of different error codes I received in posts above, but yes it was ironic that two option buttons I had included Debug and Crash.....debug ended up crashing the client in any event

capricorn152245ub17_ESO

Delpi wrote: »

I can't load any add-on anymore. I select them, reload ui and nothing happens. By the way, when I load another character the game crash with a message (the first time I saw this error with a 'crash button' made me laugh lol) I wonder if It's just a problem of mine or you have it too...?

I've received the same crash notice with the switching characters crash. I am finding that changing the file handling as in KhajiitFurTrader's post in part 4 is still working for me to prevent this.

I find when my add-ons won't load as you've described, quitting the game and restarting generally fixes the problem. (reloadui and logout and back into the character seem to have no effect). Upon logging back in after a full quit, the add-ons are all back and happy.