How do bugs make it to live ? ✔

WeylandLabs

How do bugs make it to live if there fixed or adjusted from the previous patch or dlc ?

I'm asking this becuase of disease damage status effect bug. As it was showing major defile but not actually giving the debuff ? It makes me think how long has that been a bug.

And how does new changed or adjusted abilities - passives - status effects- etc -etc get bugged if there working properly on the PTS.

Just a few questions - if any one has the knowledge to answer that would be most helpful.

I ask this becuase I seen the dev notes and they are looking into the crown poisons doing more damage than the 2x dot poisons. I'm thinking how is this possible - you'd have to purposely change it to make it stronger right. Becuase it wouldn't change by itself - or am I wrong ?

Thanks ... 👍

mairwen85

In cocoons.

Katheriah

Easy: no testing or bad testing, having other priorities. :-)

starkerealm

mairwen85 wrote: »

In cocoons.

MajBludd

Not listening to people on pts reporting them and the need to push things out for quarterly profits.

essi2

There is a concept in Software development called 'Known Shippable', basically the bug is known to QA/Dev leads, but it is classified as 'not worth' investing time in over more pressing matters.


ZOS' ability to prioritize is sometimes abit dodgy as they for whatever reason consistently underestimate the impact gameplay bugs and sometimes even ability/ability class buffs will have on the Live server.


And sometimes they just miss things or randomly ignore specific and testable feedback given on PTS builds.

starkerealm

Shokasegambit1 wrote: »

...or am I wrong ?

A little.

Before we get started, I want you to look at this:



So, a couple things.

Large chunks of the code base are being rewritten. The stuff like memory management is going to require some pretty extensive rewrites of how those systems work. That's back end stuff. When you have legacy code you're replacing, that interfaces with more legacy code, you're going to get bugs. Someone wrote that code. It made sense to them. No one on the planet understands what the hell they were thinking.



When you're the only person writing code on a project, it's relatively easy. You understand your own logic, so your greatest enemy is typos (and off by one errors.)

When you're working with a large team, and you're all writing code, good luck. Good documentation, and coherent project design will help. But, stick enough programmers in the room, and inevitably one of them won't be strange enough. They'll implement something that makes sense to them, and no one else.

Come back six months later after they've moved on to a new project on the dark side of the moon, or Kansas, and good luck figuring out what they wrote. It looks like a coherent procedure call, but you're pretty sure it actually summons the Ghost of Christmas Past when you execute it in the wrong sub.

You wanted to know about the poisons. What could affect that? Dunno.

It should be a short list. The poisons should be connected to the item database, and that should be it. They're scaling, but that's an item database trait. They're not the only ones. It should apply a penalty to them, but that might have been removed. Other scaling items, like the Double Bloody Mara, have been altered. It's possible one of those changes affected the poisons.

But, if you ask the Ghost of Christmas Past, maybe they were given hard coded values, flagged for scaling, and now kinda do whatever they want. At that point, those values could be hooked into other things you wouldn't expect. So, when you change a value from an ability, it could turn out that the poisons are scaling off of that. They shouldn't be, but it's possible. I've seen some weird ass connections in games before. "This var returns the data I want, let's just call that and move on with our day."

So, how does that happen? ESO is a very big, very complicated, machine. No one person fully understands everything. Hundreds of people (if not more) have worked on the code base. How do bugs happen? Someone solved a problem without warning everyone else what they did.

Morgul667

They simply dont care enough

wrath_of_erana

zos doesnt give a f thats how

ChunkyCat

Devs don’t use protection.

Halcyon_blue

What starkerealm said.

The complexity of programming and software development is really under-appreciated by many.

Just because you're posting on a forum, yes that feedback is captured. It then goes into a pool of other feedback, then prioritised, then allocated out based on capacity. This would likely involve a social media team, a product team, a dev team, a QA team and process interations. Then we assume every human in the team never makes a mistake and correctly identifies interactions with legacy code and their colleagues' work.

In other words, real life.

If we could get programming to be as smooth as half the forum goers want, we would already be dead from Skynet years ago.

Elsonso

Halcyon_blue wrote: »

If we could get programming to be as smooth as half the forum goers want, we would already be dead from Skynet years ago.

I'd be more likely that we would be currently living in a simulation where the bugs are deliberately injected into "reality" to make it more believable.

That said, you really cannot test quality into a product like ESO. It is too large and too complicated to completely test in the time allowed. The development is never going to be large enough to turn around fixes for all the known defects in the time allowed. That does not even start to think about the defects that are yet to be discovered.

ESO, if it is like most large software projects, is held together with the programming equivalent of duct tape and bailing wire.

"If builders built houses the way programmers built programs, the first woodpecker to come along would destroy civilization."

starkerealm

lordrichter wrote: »

Halcyon_blue wrote: »

If we could get programming to be as smooth as half the forum goers want, we would already be dead from Skynet years ago.

I'd be more likely that we would be currently living in a simulation where the bugs are deliberately injected into "reality" to make it more believable.

That said, you really cannot test quality into a product like ESO. It is too large and too complicated to completely test in the time allowed. The development is never going to be large enough to turn around fixes for all the known defects in the time allowed. That does not even start to think about the defects that are yet to be discovered.

ESO, if it is like most large software projects, is held together with the programming equivalent of duct tape and bailing wire.

"If builders built houses the way programmers built programs, the first woodpecker to come along would destroy civilization."

I mean, that's a pretty good description of the internet, when you get right down to it.

None of this, and I really do mean, none of this, was designed, from the ground up, with the idea that there would one day be a network with billions of client systems, passing data round the world. People like Gibson said, "this will happen," but all of the technology we work with is jury-rigged to hell and back.

Just the idea of writing netcode makes me want to vomit.

I still remember handing code over to my professor. The printer module didn't work. No one knew why it didn't work. It was written to spec. It should have worked. Syntax was good. No typos anyone could find. It did nothing. Wouldn't even print to file. No one could figure out why.

p4l4mu7

They're too busy busy making new content rather than fixing the game, also they don't care about issues until they start losing players because of them.

Facefister

New content? What content? Crown-store mounts?

SirAndy

That's what happens when you lay off your entire QA department ...

Sarousse

5 devs team. What do you expect ?

TheShadowScout

Shokasegambit1 wrote: »

How do bugs make it to live ?

You need to ask???

By strength of numbers, the bugstompers just cannot get them all!

xenowarrior92eb17_ESO

Sarousse wrote: »

5 devs team. What do you expect ?

my bad...did not know ZoS its just a small indie studio

vgabor

Shokasegambit1 wrote: »

How do bugs make it to live

They creep there under the carpet

Yuffie91

I think every single game has issues like this. Though Bethesda is known for their bugs and players making mods to fix them

WeylandLabs

starkerealm wrote: »

Shokasegambit1 wrote: »

...or am I wrong ?

A little.

Before we get started, I want you to look at this:



So, a couple things.

Large chunks of the code base are being rewritten. The stuff like memory management is going to require some pretty extensive rewrites of how those systems work. That's back end stuff. When you have legacy code you're replacing, that interfaces with more legacy code, you're going to get bugs. Someone wrote that code. It made sense to them. No one on the planet understands what the hell they were thinking.



When you're the only person writing code on a project, it's relatively easy. You understand your own logic, so your greatest enemy is typos (and off by one errors.)

When you're working with a large team, and you're all writing code, good luck. Good documentation, and coherent project design will help. But, stick enough programmers in the room, and inevitably one of them won't be strange enough. They'll implement something that makes sense to them, and no one else.

Come back six months later after they've moved on to a new project on the dark side of the moon, or Kansas, and good luck figuring out what they wrote. It looks like a coherent procedure call, but you're pretty sure it actually summons the Ghost of Christmas Past when you execute it in the wrong sub.

You wanted to know about the poisons. What could affect that? Dunno.

It should be a short list. The poisons should be connected to the item database, and that should be it. They're scaling, but that's an item database trait. They're not the only ones. It should apply a penalty to them, but that might have been removed. Other scaling items, like the Double Bloody Mara, have been altered. It's possible one of those changes affected the poisons.

But, if you ask the Ghost of Christmas Past, maybe they were given hard coded values, flagged for scaling, and now kinda do whatever they want. At that point, those values could be hooked into other things you wouldn't expect. So, when you change a value from an ability, it could turn out that the poisons are scaling off of that. They shouldn't be, but it's possible. I've seen some weird ass connections in games before. "This var returns the data I want, let's just call that and move on with our day."

So, how does that happen? ESO is a very big, very complicated, machine. No one person fully understands everything. Hundreds of people (if not more) have worked on the code base. How do bugs happen? Someone solved a problem without warning everyone else what they did.

I guess what I am saying also - is could this be done on purpose ? I know very little about coding - but a lot from a business standpoint.

The problem is reletivly easy to fix looking from a business perspective. I'm not trying to do research in how much these devs get paid but seems that the talent acquisition is very low. Rather paying a dev 60k from ITT Tech - to "save" money. You pay 120k for real talent that has the education and background to get things done.

Or... Unffortunaly what I'm thinking - is the proprietary systems they are using are completely garbage and out of date. And new devs are clueless to it at first thus having a retraining on this old - yet updated in house system they are using. And nobody wants to be apart of something so cringeworrhy developing and programming.

I just don't know but from what I heard from the bird - it's the talent. And ZOS's ability to not pay for knowledgable employees.

And it all just been damage control ... with dlc drops in between. It's us the players to feel what is acceptable and what's not. Catering towards roleplayers not fixing the game is what seems to be the real meta.

Regaurdless... we look at this process in so many ways that it's really tiresome. But it's no sweat off my back after 15k hours in the game, I don't play anymore I just find ways not too. 👍

Elsonso

Shokasegambit1 wrote: »

I guess what I am saying also - is could this be done on purpose ? I know very little about coding - but a lot from a business standpoint.

The problem is reletivly easy to fix looking from a business perspective. I'm not trying to do research in how much these devs get paid but seems that the talent acquisition is very low. Rather paying a dev 60k from ITT Tech - to "save" money. You pay 120k for real talent that has the education and background to get things done.

This statement bothers me on so many levels. SMH as I try to sort them in order of magnitude of who is being insulted.

zaria

starkerealm wrote: »

Shokasegambit1 wrote: »

...or am I wrong ?

When you're the only person writing code on a project, it's relatively easy. You understand your own logic, so your greatest enemy is typos (and off by one errors.)

When you're working with a large team, and you're all writing code, good luck. Good documentation, and coherent project design will help. But, stick enough programmers in the room, and inevitably one of them won't be strange enough. They'll implement something that makes sense to them, and no one else.

Come back six months later after they've moved on to a new project on the dark side of the moon, or Kansas, and good luck figuring out what they wrote. It looks like a coherent procedure call, but you're pretty sure it actually summons the Ghost of Christmas Past when you execute it in the wrong sub.

You wanted to know about the poisons. What could affect that? Dunno.

It should be a short list. The poisons should be connected to the item database, and that should be it. They're scaling, but that's an item database trait. They're not the only ones. It should apply a penalty to them, but that might have been removed. Other scaling items, like the Double Bloody Mara, have been altered. It's possible one of those changes affected the poisons.

But, if you ask the Ghost of Christmas Past, maybe they were given hard coded values, flagged for scaling, and now kinda do whatever they want. At that point, those values could be hooked into other things you wouldn't expect. So, when you change a value from an ability, it could turn out that the poisons are scaling off of that. They shouldn't be, but it's possible. I've seen some weird ass connections in games before. "This var returns the data I want, let's just call that and move on with our day."

So, how does that happen? ESO is a very big, very complicated, machine. No one person fully understands everything. Hundreds of people (if not more) have worked on the code base. How do bugs happen? Someone solved a problem without warning everyone else what they did.

This, I'm working with an page on an web application, pretty simple generate an table showing user progress in required reading. My problem is that the user mass is close to 100 its also over 100 subjects. Not all is require to read and check out all.
As this would require open lots of sql queries to ask in the way its required to be displayed so I streamlined this to two calls and wrote this into an virtual table who I could then write in one operation.
This had some bug, it was very hard to figure out how this worked as I could not figure how it work and I made it two years ago.

starkerealm

zaria wrote: »

starkerealm wrote: »

Shokasegambit1 wrote: »

...or am I wrong ?

When you're the only person writing code on a project, it's relatively easy. You understand your own logic, so your greatest enemy is typos (and off by one errors.)

When you're working with a large team, and you're all writing code, good luck. Good documentation, and coherent project design will help. But, stick enough programmers in the room, and inevitably one of them won't be strange enough. They'll implement something that makes sense to them, and no one else.

Come back six months later after they've moved on to a new project on the dark side of the moon, or Kansas, and good luck figuring out what they wrote. It looks like a coherent procedure call, but you're pretty sure it actually summons the Ghost of Christmas Past when you execute it in the wrong sub.

You wanted to know about the poisons. What could affect that? Dunno.

It should be a short list. The poisons should be connected to the item database, and that should be it. They're scaling, but that's an item database trait. They're not the only ones. It should apply a penalty to them, but that might have been removed. Other scaling items, like the Double Bloody Mara, have been altered. It's possible one of those changes affected the poisons.

But, if you ask the Ghost of Christmas Past, maybe they were given hard coded values, flagged for scaling, and now kinda do whatever they want. At that point, those values could be hooked into other things you wouldn't expect. So, when you change a value from an ability, it could turn out that the poisons are scaling off of that. They shouldn't be, but it's possible. I've seen some weird ass connections in games before. "This var returns the data I want, let's just call that and move on with our day."

So, how does that happen? ESO is a very big, very complicated, machine. No one person fully understands everything. Hundreds of people (if not more) have worked on the code base. How do bugs happen? Someone solved a problem without warning everyone else what they did.

This, I'm working with an page on an web application, pretty simple generate an table showing user progress in required reading. My problem is that the user mass is close to 100 its also over 100 subjects. Not all is require to read and check out all.
As this would require open lots of sql queries to ask in the way its required to be displayed so I streamlined this to two calls and wrote this into an virtual table who I could then write in one operation.
This had some bug, it was very hard to figure out how this worked as I could not figure how it work and I made it two years ago.

It gets worse as you go. Code I wrote for flatform databases in the 90s? I know what they did, more or less, but it's completely unreadable now, because of the absolutely bonkers syntax the database used. Still "technically" works, but you can't modify at all or the whole thing starts eating its own foot.

Rungar

if were going to be honest im amazed it works at all

starkerealm

Shokasegambit1 wrote: »

I guess what I am saying also - is could this be done on purpose ? I know very little about coding - but a lot from a business standpoint.

Yeah, in two ways.

Hang around the internet long enough, and you'll find stories, or delusional daydreams, from coders who sabotaged the projects they were hired to work on so the code wouldn't work without them, in order to give themselves job security. Usually this stuff comes from someone who was already on staff, and was tasked with creating a chunk of software without supervision, or working with anyone else. At that point, the difference between poor coding with bad documentation and sabotage is in the intent.

Is that happening here? No.

The other yes is that bugs can occur from intentional decisions by the programmer. At the time those decisions may have been sound, but because other systems, or other database entries, change, suddenly stuff that used to work fine doesn't anymore.

I mentioned this in the previous post, when someone made a rational decision that no one else could figure out. Through no intentional malice, you can write code that works fine, makes sense to you, and is a ticking time bomb for a guy five years from now.

Shokasegambit1 wrote: »

The problem is reletivly easy to fix looking from a business perspective. I'm not trying to do research in how much these devs get paid but seems that the talent acquisition is very low. Rather paying a dev 60k from ITT Tech - to "save" money. You pay 120k for real talent that has the education and background to get things done.

Or... Unffortunaly what I'm thinking - is the proprietary systems they are using are completely garbage and out of date. And new devs are clueless to it at first thus having a retraining on this old - yet updated in house system they are using. And nobody wants to be apart of something so cringeworrhy developing and programming.

When you're talking about programming, "proprietary systems" are a little different from what you might be used to.

ESO is running on an in-house engine. At some point ZOS had the Hero engine licensed, but moved to their own years before release. In-house engines can be a little tricky. The documentation is not there, but usually you've got most of the people who put it together in the first place, so even if you're new, you can talk to the person responsible for whatever bat**** weirdness you can't wrap your head around.

The upside is, in-house engines can do some insane things. Because you've got access to the original designers (even if it's just some of them), you'll be able to do things that simply wouldn't be possible with an off the shelf engine like Unreal. Also, once you have the engine, it's cheaper. You don't have to worry about someone outside of your project having a stroke of genius that screws up huge swaths of your code base.

That said, it's not cringeworthy, but eventually you'll realize that something written eight years ago is obtuse. It could be cleaner. Or, it's messy. Or it's simply poorly optimized. Maybe somewhere along the line something got revised. Drivers changed, and you can now do it more cleanly. At that point, it should be rewritten, but won't be, because it might break other things.

Now, what we're seeing is a refactoring. That's basically, going back to the drawing board, and starting from scratch. Rewriting the code. It's a huge endeavor. And it's necessary. It took a lot of back end work to make the dragons happen. I'm not sure how much, but that was something the engine straight up wasn't designed to handle. You can see the issues the engine originally had with flight when you look at harpies.

Shokasegambit1 wrote: »

I just don't know but from what I heard from the bird - it's the talent. And ZOS's ability to not pay for knowledgable employees.

This may surprise you, but Rob Garret came from Riot. He left the League of Legends devs to come over to ZOS. Granted, that's a data point of one, but I don't think pay is an issue. I do think, like a lot of MBAs, you do not fully grasp how complicated programming can be. This, isn't a knock against you, but it something to remember.

I don't have the hard number (obviously), but ESO's code runs into the millions of lines. Saying that the problem is simply not getting sufficiently skilled people is, frankly, a bit naive.

And, your little bird probably wanted to sound more knowledgeable than they were. Granted, you could get higher gross wages working for a dev in LA or The Bay Area, but, the downside would be that you were also living in The Bay Area or Los Angeles, where cost of living is significantly higher than Baltimore.

Shokasegambit1 wrote: »

And it all just been damage control ... with dlc drops in between. It's us the players to feel what is acceptable and what's not. Catering towards roleplayers not fixing the game is what seems to be the real meta.

There have been a lot of fixes over the years. There a lot of bugs. Some of those you really can't help. You just need to find them after they happen. Bonus points, some bugs don't occur until it hits live, and the server strain gets added. Who remembers non-respawning bosses at launch? That a bug that was impossible to detect before the system went live and was under full load. It simply didn't happen in low load conditions.

Shokasegambit1 wrote: »

Regaurdless... we look at this process in so many ways that it's really tiresome. But it's no sweat off my back after 15k hours in the game, I don't play anymore I just find ways not too. 👍

So how do bugs go live?

eso_lags

Shokasegambit1 wrote: »

How do bugs make it to live if there fixed or adjusted from the previous patch or dlc ?

I'm asking this becuase of disease damage status effect bug. As it was showing major defile but not actually giving the debuff ? It makes me think how long has that been a bug.

And how does new changed or adjusted abilities - passives - status effects- etc -etc get bugged if there working properly on the PTS.

Just a few questions - if any one has the knowledge to answer that would be most helpful.

I ask this becuase I seen the dev notes and they are looking into the crown poisons doing more damage than the 2x dot poisons. I'm thinking how is this possible - you'd have to purposely change it to make it stronger right. Becuase it wouldn't change by itself - or am I wrong ?

Thanks ... 👍

Its simple really, because zenimax doesnt fix things they are aware are broken before they put out an update. They will not delay the update if multiple things are broken. Also there might be some they are unaware of before the update goes out, but I've seen, more often than not, that people post about bugs that dont get fixed until after the update is out. Sometimes its weeks and sometimes months. Sometimes even years like with the undo bug which was found shortly after summerset went live, but the issue is still in game..

As for how something is fine on the PTS and then broken when it hits live, im not sure. My best guess would be that no one realized it was broken or bugged until it hit live, or that something within the game changed that caused it to have an issue. Even if zos decided to make a very small change to something right before they released the update, if that caused an issue people wouldnt be able to test it except for zos themselves. And i dont think they do too much testing tbh.

Kahmel

This thread made me rethink everything I thought I knew about video games. I was one of those guys that thought as long as you had the knowledge you could code video games with ease and bugs were just people not understanding their jobs. But now I see how wrong I was and boy was I wrong I'm sorry!

Also I'm glad I went into IT instead of software development xD

WeylandLabs

@starkerealm

You are a God at explaining things like a real human ! Thank you for your feedback - I was a bit sarcastic in my comment and you got the point.

That's crazy that you knew exactly what I was saying or trying to say. And I do live in L.A surrounded by talent from all walks of life. I guess when this business is centered out of Maryland you kinda got rethink stuff.


( Sarcasm * ) 👇

I'd figure damn - M.I.T is right down the street. Find the super nerd boss, you know the nerd that makes fun of all the other nerds for being stupid. And pay him to fix this whole thing for a (x) amount of money. Guess it's a bit more complicated than that.

It's crazy - if I would have known what you said 4 years ago I would have started a Gofundme account to save ESO. And hire the "Super Nerd Boss" for Zenimax.

Like - Here ya go your community dished out a million dollars so he could fix your game.

( no sarcasm * ) 👇

But again there would have been something wrong still becuase of what you said. 👍 I'm sure you understand what I'm saying.



Again thanks you for your explanation. That was trully the first full real comment I got in a while of a factual breakdown of everything.

Two more questions : If I may ...

Where do you see ESO heading ?

And in terms of development do you feel it that ESO will succeed past it's 4 1/2 year mark of dlc/content ? Meaning how long do you feel this game will last after its done with DLC's.

Thanks -

starkerealm

Shokasegambit1 wrote: »

I'd figure damn - M.I.T is right down the street.

I realize this was in the sarcasm, however MIT is in Boston. Not sure if this is new information, for you, but I had a guildie who thought Bert's trip to Boston meant ZOS was flying him out to see Elsweyr at the internal test. Nope, that was PAX East.

ZOS is based out of Hunt Valley, Maryland.

Shokasegambit1 wrote: »

Where do you see ESO heading ?

A lot of the quality of life stuff that's being worked on now, or being rolled out, is a long term investment in the game.

I mentioned this in passing, but refactoring software is not something you do when you plan on tossing it in six months or a year. So, realistically, if ZOS didn't think the game would last another four or five years, they wouldn't do it.

Will it last that long? I don't know. But, barring some dramatic shift in the market, or other unforseen factors, it'll be around for awhile.

Shokasegambit1 wrote: »

And in terms of development do you feel it that ESO will succeed past it's 4 1/2 year mark of dlc/content ? Meaning how long do you feel this game will last after its done with DLC's.

It won't. So, I don't know what the sunset plan for ESO will look like, and I wouldn't be able to speculate. At some point, yeah, new content will stop, and the game will be placed on maintenance mode. That's not going to happen until there's a persistent population drop off. The entire design plan for ESO is to keep churning out content until it's no longer profitable.

I don't know what happens when the next consoles roll out. I suspect that the PS4 and XB1 versions of the game will no longer be supported, that their existing servers will be converted over onto the new PS5 and XB4 builds of the game. I'm not sure, but I'd be willing to guess part of the push for account "cold storage" is in anticipation of that day.

For what it's worth, back in 2012 or '13, ESO was pitched as a 10 year life cycle. Something that a number of members of the team have told me in response to a number of questions about decisions back at launch is, "it's a different game today." That's not just a blow off, either. ESO underwent some major changes. The introduction of Champion Points was already in the works at launch, and Wrothgar set the stage for One Tamriel. So, looking at it now, I'd surprised if the game was on maintenance before 2024. Will they continue to turn out new quarterly content after that? I don't know, but given the overall trajectory the game's enjoyed, it's distinctly possible.

Now, if I sound overly optimistic, I'm not. There's a lot of caution here, because we don't know what will happen moving forward, but, I do think the game has a future.