EU Server Performance - Update from Matt Firor

Merlight

daemonios wrote: »

I stand corrected. Does ZEL distribute any other games? Are those numbers solely related to ESO?

Not just ESO. You can find details in the report
https://beta.companieshouse.gov.uk/company/06333300/filing-history
https://beta.companieshouse.gov.uk/company/06333300/filing-history/MzIxNDY2MzcyNmFkaXF6a2N4/document?format=pdf&download=0

I'm not quoting because the pdf is not text but scanned document lol, but the list of games is on page 4.

twev

Wolfpaw wrote: »

twev wrote: »

LiraTaurwen wrote: »

everlastingodeb17_ESO wrote: »

Zos uk had profits of 50+ million euros in 2017 and 40+ in 2016( dont have 2018 data) not to mention Zos France and Zos Germany, yet they cant fix the server... Get skilled people you clearly have the financial resources needed.

That money is going to the rich people that own this company, like the Trumps for instance, and they are not happy to give up any of it to improve a server.

Nice strawman argument.
Trump is one name out of about 40, and not at the top of the list.
I'm sure everyone else on the list is purely philanthropic by comparison.

And 'rich people' make things happen with investments.
The don't make return on investment by keeping their money under the mattress.

Maybe more people without access to investment funds should back the game.
I'm sure an operating budget of several thousand dollars would bring about a groundswell of development and bug fixes.

I only count 8 on the board of directors, & additional 9 including upper management. You have a different source?
https://www.zenimax.com/about

ZeniMax Media inc. Is a billion dollar company, where is the investments into a stable 5year old product we have been waiting for?

1.) Not just the board of directors make up 'The List', and the board of directors isn't a list of the investors who need to be paid back for all of the R&D money to finance the game since inception in 2007.
The board of directors are just the guys who make corporate decisions among other things.


The game is 'reported' (and estimated) by many sources to have cost over $200 million to develop over the 7 to 8 years it was being produced before it was released.
Thats all time the investment wasn't generating income.

Whether members of the board are actual investors or not is a different matter, but even if they are - they're not the only ones who ARE investors, and who expect to get return on investment.

We're not even talking about operating expenses here, which takes a big bite out of generated gross income for this game division.

There are a lot of people being paid salary as well as benefits as part of labor burden, infrastructure of buildings and utilities to keep the lights on.

As far as I can tell, the company has to pay taxes as well.

Add in all the staffing and development since release that are being paid since release, and all the money generated since release has to pay for a lot of stuff (with interest) since 2007.

ZO$ isn't low on cash, or course not, they make a profit, but it's not all gravy as many people prefer to believe.

It's all part of the mosaic of business, and business aint run as a 'not for profit' for the most part in the real world.
Otherwise, you'd be playing a game that was made out of the goodness of all the people's hearts who worked on this game since day one, while all the suppliers and utility companies were donating everything from paper/paperclips to servers for free.

Wolfpaw

twev wrote: »

Wolfpaw wrote: »

twev wrote: »

LiraTaurwen wrote: »

everlastingodeb17_ESO wrote: »

Zos uk had profits of 50+ million euros in 2017 and 40+ in 2016( dont have 2018 data) not to mention Zos France and Zos Germany, yet they cant fix the server... Get skilled people you clearly have the financial resources needed.

That money is going to the rich people that own this company, like the Trumps for instance, and they are not happy to give up any of it to improve a server.

Nice strawman argument.
Trump is one name out of about 40, and not at the top of the list.
I'm sure everyone else on the list is purely philanthropic by comparison.

And 'rich people' make things happen with investments.
The don't make return on investment by keeping their money under the mattress.

Maybe more people without access to investment funds should back the game.
I'm sure an operating budget of several thousand dollars would bring about a groundswell of development and bug fixes.

I only count 8 on the board of directors, & additional 9 including upper management. You have a different source?
https://www.zenimax.com/about

ZeniMax Media inc. Is a billion dollar company, where is the investments into a stable 5year old product we have been waiting for?

1.) Not just the board of directors make up 'The List', and the board of directors isn't a list of the investors who need to be paid back for all of the R&D money to finance the game since inception in 2007.
The board of directors are just the guys who make corporate decisions among other things.


The game is 'reported' (and estimated) by many sources to have cost over $200 million to develop over the 7 to 8 years it was being produced before it was released.
Thats all time the investment wasn't generating income.

Whether members of the board are actual investors or not is a different matter, but even if they are - they're not the only ones who ARE investors, and who expect to get return on investment.

We're not even talking about operating expenses here, which takes a big bite out of generated gross income for this game division.

There are a lot of people being paid salary as well as benefits as part of labor burden, infrastructure of buildings and utilities to keep the lights on.

As far as I can tell, the company has to pay taxes as well.

Add in all the staffing and development since release that are being paid since release, and all the money generated since release has to pay for a lot of stuff (with interest) since 2007.

ZO$ isn't low on cash, or course not, they make a profit, but it's not all gravy as many people prefer to believe.

It's all part of the mosaic of business, and business aint run as a 'not for profit' for the most part in the real world.
Otherwise, you'd be playing a game that was made out of the goodness of all the people's hearts who worked on this game since day one, while all the suppliers and utility companies were donating everything from paper/paperclips to servers for free.

So to sum up, half a billion dollar plus annual revenue, 5year old game, no more excuses.

Elsonso

Wolfpaw wrote: »

So to sum up, half a billion dollar plus annual revenue

Earned across a handful of studios, and games like Rage, Prey, Doom, Fallout, TES, ESO, TESL, Dishonored, etc.

Wolfpaw

lordrichter wrote: »

Wolfpaw wrote: »

So to sum up, half a billion dollar plus annual revenue

Earned across a handful of studios, and games like Rage, Prey, Doom, Fallout, TES, ESO, TESL, Dishonored, etc.

Yes, those games are mentioned in the link I gave for ZeniMax Media.

Cążki

Ok. They have so much money in a whole company. They can afford for group of programmers who can update the ESO in one week, with LFG system, performance ect ... Its all about managment and chemistry in company.


Metaphor:
-They most spend the time sitting under the treee in woods but when rains starts(playerbase scream) they plan build the tent for storm.

Delparis

LiraTaurwen wrote: »

everlastingodeb17_ESO wrote: »

Zos uk had profits of 50+ million euros in 2017 and 40+ in 2016( dont have 2018 data) not to mention Zos France and Zos Germany, yet they cant fix the server... Get skilled people you clearly have the financial resources needed.

That money is going to the rich people that own this company, like the Trumps for instance, and they are not happy to give up any of it to improve a server.

No you're wrong

The real investor with preferred Zenimax stocks

https://www.provequity.com/system/uploads/article/article_pdf/85/6197f0e7-173f-447b-832f-1b6829807edc.pdf

Vapirko

Firstmep wrote: »

Alcast wrote: »

"and we can now support the concurrency demand we are seeing"

Yeah, that statement is more worrying than the rest of his post combined.
@ZOS_MattFiror i invite you to play on PC EU during primetime, to see how "well" the server can support the concurrent playerbase.
It cant. The fact that i have better ping in Cyrodiil than in most overland zones often, is truly disturbing.

Don’t ask them to play, they’ll just wire themselves into the best possible connection to the servers and be like, “See? This game runs just fine.” I would pay to seen them do it on a normal connection from a residential home though. Something like two hours a night during prime time for three or four days covering the week and weekend. And stream it. I want to laugh in hateful joy.

Merlight

Vapirko wrote: »

And stream it. I want to laugh in hateful joy.

Stop laughing and go reset your router. Must be your poor connection lagging the stream.

ATomiX96

No biggie, just another 6 months of terrible performance with the slight chance of getting improvements.
Just buy some crown crates and cosmetics in the meantime LOOOOOOOL.

Cążki

Im asume and provide you should to stop playing games at this beautiful weather in EU 30oC nice girls outside :P lets wait to the winter then :P

WoppaBoem

All the hate, I feel sorry for you guys it must be really bad at your mega server. You know you can stop playing the game?

ZOS thank you for the update. Keep it up, steady progress is much appreciated.

And funny all these people of the forums who know (assume and think) they know exactly was is going on and what needs to be done with no inside information. In Holland we call this the best sailors are in the dry docks, they know everything better from the side line....

Verbal_Earthworm

WoppaBoem wrote: »

All the hate, I feel sorry for you guys it must be really bad at your mega server. You know you can stop playing the game?

ZOS thank you for the update. Keep it up, steady progress is much appreciated.

And funny all these people of the forums who know (assume and think) they know exactly was is going on and what needs to be done with no inside information. In Holland we call this the best sailors are in the dry docks, they know everything better from the side line....

As you are in fact poking your nose into a thread that doesn't apply to you whatsoever, you are in the dry docks yourself, sailor.

This thread is for PC-EU server players, not you, go away and take your condescending attitude with you.

Navras

It's great to have some feedback, thanks for keeping us posted.

After the premise, in all honesty, I don't understand why pushing update 23 with new dungeons (am I right? isn't it a dungeon dlc?) before fixing what is not working, which is also related to dungeons.

I know you guys have schedules and such but how can anyone enjoy a dungeon DLC with broken queues and all those issues we are experiencing?

twev

Cążki wrote: »

Ok. They have so much money in a whole company. They can afford for group of programmers who can update the ESO in one week, with LFG system, performance ect ... Its all about managment and chemistry in company.


Metaphor:
-They most spend the time sitting under the treee in woods but when rains starts(playerbase scream) they plan build the tent for storm.

I absolutely agree, and have been saying forEVAR that they need to get out the checkbook and pay whatever it reasonably costs to build a bigger bugstomp team, and theres no reason it can't be done.

I just took exception to the mention that there being a TRUMP on the board meant that assumptions could be made that he was they guy syphoning all the cash out from under us, and we were fated to be destitute because of it.

People look at the income of games and like to think it's 100% profit, while they ignore the huge debt-load the game racked up over years of development with no income.

If any investors don't get paid back, then investment money dries up, and we're not looking at AAA games anymore, but stuff that was coded by 3 guys living in Mom's basement.

Uryel

A bit off the track but I'd like to share a personal story.

Back 10 years ago, I used to work in IT. I used to be a sytem and network tech, that's what I studied at school and all. Now I'm a social worker, so I'm not up to date on modern issues, don't ask me to fix your rig XD

So, back then I worked for a somewhat large-ish company with subsidiaries all over the world. The company was doing very well, being leader on their specific market, but the company itself wasn't a major name and chances are good you never heard of it. IT Infrastructure was a bit cobbled together and not as clean as could be, but it was working fine, and it had to since every user from every subsidiaries in the world needed to log and work onto the main systems to work.

One day, one of our server failed. Sh*t happens, that's what redudancy is for. As it turned out, the redudancy in that case was a virtual machine. Problem is, the physical server running that virtual machine wasn't dimensionned properly, because costs were deemed too high, and was already used to run the virtual printing server. Yep, we had had a physical server dedicated to printing, but it had failed a month before, and orders were "let the virtual machine do the job, I'm not spending a dime on a computer that only does printing". Yeah, well, I suppose all those labels for categorizing the products or shipping them don't really matter, eh ? As you can guess, the printing server was a very busy one, so adding load to the same physical server wasn't gonna end well. And as it happens, the server that had just failed was the one that basically stored and exploited all data about what was made, where, who ordered what, for when, and all payment data. This thing was used by accounting, manufacturing and shipping.

Basically, if those data were lost, the company would lose 3 months worth of financial data, which back then amounted to about 80 millions euros. That's not factoring the possible loss of customers if we also lost who ordered what and couldn't deliver. And we were now running that server on an already overloaded virtual machine. I don't really need to spell the catastrophe at hand, right ?

So of course we wanted to replace the physical server as fast as we could. It didn't even need to be that great, a 5000 euros machine would do. We found one and a delivery guy that could bring it to us within a few hours. You know what took the longest time before we actually could get to work ? Convincing the boss that he really needed to spend those 5000 euros.

That jackarse would rather take the risk of losing 80 millions worth of income than have to explain to his boss why he allowed a purchase worth 5000 euros. He could have blamed the failure on us, but he couldn't blame the spent money on us, of course.

In the end, after much convincing and some shouting, that jerk opened the company wallet, we had to pay double for the delivery because it took so long that it was now friday evening and we needed it RIGHT NOW to do overwork on the week end when the load is reduced so that it wouldn't blow in our face on monday.

That's IT for you, folks. Guys who know squat decide how much it's worth to spend on hardware they don't even begin to understand and are in complete denial that they could actually lose huge amounts of money if they don't spend a tiny bit. The executives can always blam the techs if / when sh*t hits the fan. That is, assuming the fan isn't broken and unreplaced because of the cost deemed too high.

No matter the company, even those who rely entirely on their servers to make money, IT infrastructures is never seen as a source of income, only as an expense.

reoskit

Uryel wrote: »

That's IT for you, folks. Guys who know squat decide how much it's worth to spend on hardware they don't even begin to understand and are in complete denial that they could actually lose huge amounts of money if they don't spend a tiny bit. The executives can always blam the techs if / when sh*t hits the fan. That is, assuming the fan isn't broken and unreplaced because of the cost deemed too high.

No matter the company, even those who rely entirely on their servers to make money, IT infrastructures is never seen as a source of income, only as an expense.

Preach, @Uryel.

Also, thanks for making my work and play worlds collide.

Thrawniel

'Connection lost due to inactivity'. In the middle of fracking trial. Yeah. I was so inactive fighting the boss...

UPD: it took forever to load back too, so I got kicked(( Thanks for ruining my limited game time by your crappy servers. I won't have enough time to find a new group and star over today.

twev

Uryel wrote: »

A bit off the track but I'd like to share a personal story.

Back 10 years ago, I used to work in IT. I used to be a sytem and network tech, that's what I studied at school and all. Now I'm a social worker, so I'm not up to date on modern issues, don't ask me to fix your rig XD

So, back then I worked for a somewhat large-ish company with subsidiaries all over the world. The company was doing very well, being leader on their specific market, but the company itself wasn't a major name and chances are good you never heard of it. IT Infrastructure was a bit cobbled together and not as clean as could be, but it was working fine, and it had to since every user from every subsidiaries in the world needed to log and work onto the main systems to work.

One day, one of our server failed. Sh*t happens, that's what redudancy is for. As it turned out, the redudancy in that case was a virtual machine. Problem is, the physical server running that virtual machine wasn't dimensionned properly, because costs were deemed too high, and was already used to run the virtual printing server. Yep, we had had a physical server dedicated to printing, but it had failed a month before, and orders were "let the virtual machine do the job, I'm not spending a dime on a computer that only does printing". Yeah, well, I suppose all those labels for categorizing the products or shipping them don't really matter, eh ? As you can guess, the printing server was a very busy one, so adding load to the same physical server wasn't gonna end well. And as it happens, the server that had just failed was the one that basically stored and exploited all data about what was made, where, who ordered what, for when, and all payment data. This thing was used by accounting, manufacturing and shipping.

Basically, if those data were lost, the company would lose 3 months worth of financial data, which back then amounted to about 80 millions euros. That's not factoring the possible loss of customers if we also lost who ordered what and couldn't deliver. And we were now running that server on an already overloaded virtual machine. I don't really need to spell the catastrophe at hand, right ?

So of course we wanted to replace the physical server as fast as we could. It didn't even need to be that great, a 5000 euros machine would do. We found one and a delivery guy that could bring it to us within a few hours. You know what took the longest time before we actually could get to work ? Convincing the boss that he really needed to spend those 5000 euros.

That jackarse would rather take the risk of losing 80 millions worth of income than have to explain to his boss why he allowed a purchase worth 5000 euros. He could have blamed the failure on us, but he couldn't blame the spent money on us, of course.

In the end, after much convincing and some shouting, that jerk opened the company wallet, we had to pay double for the delivery because it took so long that it was now friday evening and we needed it RIGHT NOW to do overwork on the week end when the load is reduced so that it wouldn't blow in our face on monday.

That's IT for you, folks. Guys who know squat decide how much it's worth to spend on hardware they don't even begin to understand and are in complete denial that they could actually lose huge amounts of money if they don't spend a tiny bit. The executives can always blam the techs if / when sh*t hits the fan. That is, assuming the fan isn't broken and unreplaced because of the cost deemed too high.

No matter the company, even those who rely entirely on their servers to make money, IT infrastructures is never seen as a source of income, only as an expense.

Your tale is frighteningly parallel to how most lawmakers pass legislation controlling issues they don't have the first clue of the scope or machinations of.

Myrm

Verbal_Earthworm wrote: »

This thread is for PC-EU server players, not you, go away and take your condescending attitude with you.

I assume, then, that no PC player has ever made a comment in a console thread on this forum, or that no PC-EU player has ever commented in a PC-NA topic, including you?

What about those who play it on more than one platform? Do we have your permission to speak?

merevie

Basically stopped Eu character when queue started.

The people leaving are not new people -they are your long term customers that provide the leadership and services that keep the community going.

Telling us that we may get around to pvp in a year is a death knell. Telling pve guilds that it'll be 6 months.. well..these are your long term customer groups.

New players will come and go. Suggest reconsidering dg release and putting resources into critical issues.

Verbal_Earthworm

Myrm wrote: »

Verbal_Earthworm wrote: »

This thread is for PC-EU server players, not you, go away and take your condescending attitude with you.

I assume, then, that no PC player has ever made a comment in a console thread on this forum, or that no PC-EU player has ever commented in a PC-NA topic, including you?

What about those who play it on more than one platform? Do we have your permission to speak?

No.

Myrm

Verbal_Earthworm wrote: »

Myrm wrote: »

Verbal_Earthworm wrote: »

This thread is for PC-EU server players, not you, go away and take your condescending attitude with you.

I assume, then, that no PC player has ever made a comment in a console thread on this forum, or that no PC-EU player has ever commented in a PC-NA topic, including you?

What about those who play it on more than one platform? Do we have your permission to speak?

No.

Haha!! Tough luck! I’m a rebel!

Darkstorne

Myrm wrote: »

Verbal_Earthworm wrote: »

This thread is for PC-EU server players, not you, go away and take your condescending attitude with you.

I assume, then, that no PC player has ever made a comment in a console thread on this forum, or that no PC-EU player has ever commented in a PC-NA topic, including you?

What about those who play it on more than one platform? Do we have your permission to speak?

It would be weird if a PC player stepped into a console technical performance thread, told them to stop complaining about a console specific issue that was promised to be addressed months ago, informed them they could always just stop playing the game, and then praised ZOS for their work.

They'd deserve the same response as the poster above got.

maboleth

Tulare_Verlaris wrote: »

Six more months of paying and playing a totally broken game then? No thanks, unsubbing and stopping playing for 6 months then.

Maybe will return down the line, maybe not. Either way you need to seriously improve your appalling communication.

*Stops playing the game for 6 months. Logs in the other day.*

Xogath

I would go as far as saying that any and all further development/additions to the game need put on the backburner until the game can perform.

Cyrodiil latency is a massive issue. Server latency as a whole has risen 30-33% with the release of Elsweyr.. how is that acceptable at all? (hint: it isn't)

Latency aside, the overall visual performance of the game is below sub-par. The Engine itself is in need of a complete overhaul, with non-native features (multi-core support, for example, which was added to the game later.. but in what way I am unsure) built in.. or.. ported to a new Engine entirely that is up to date and has everything you would want and need for the game to both look great and perform well while doing it.

I would give up the next year of content for this to happen. If everyone else felt the same way, we might actually see things get done.

But no, we'd rather whine about an ability in PvP being overtuned or something equally pitiful.

Royaji

Xogath wrote: »

I would give up the next year of content for this to happen. If everyone else felt the same way, we might actually see things get done.

Will you keep paying for those fixes like you would for content? No? Neither will I and many others so this option is off the table.

Elsonso

Darkstorne wrote: »

Myrm wrote: »

Verbal_Earthworm wrote: »

This thread is for PC-EU server players, not you, go away and take your condescending attitude with you.

I assume, then, that no PC player has ever made a comment in a console thread on this forum, or that no PC-EU player has ever commented in a PC-NA topic, including you?

What about those who play it on more than one platform? Do we have your permission to speak?

It would be weird if a PC player stepped into a console technical performance thread, told them to stop complaining about a console specific issue that was promised to be addressed months ago, informed them they could always just stop playing the game, and then praised ZOS for their work.

They'd deserve the same response as the poster above got.

No, they don't deserve that response. That sort of response is part of what makes the difference between a toxic environment and a friendly one.

More on topic, it is important to remember that not everyone who plays on PC EU has performance problems, and those that do will have varying degrees of sensitivity to it when it happens.

ZOS has been promising performance improvements for years, now. ZOS delivers on this promise in drips and dribbles, rather than as a grand reveal. No one drip or dribble is going to fix things. People tend to expect otherwise, and that just ain't gonna happen.

It is clear that PC EU has a higher player population than PC NA, at least relative to the amount of server capacity that ZOS planned for. I play during PC EU off hours, normally, so I never had to deal with login queues, and the only impact that I have in my casual overworld quest/leveling play is an increase in overall latency (~150 ms -> ~180 ms). It has to spike pretty high before I really notice it in game play.

Delparis

Uryel wrote: »

That jackarse would rather take the risk of losing 80 millions worth of income than have to explain to his boss why he allowed a purchase worth 5000 euros. He could have blamed the failure on us, but he couldn't blame the spent money on us, of course.

@ZOS_MattFiror when are you gonna spend 5000 € to upgrade the servers ?
or are you going to explain to your bosses at www.provequity.com that you can't spend that much money because it's gonna lower their dividends. Are they aware that customer are cancelling their subscription because of this BS ?

ryh

You will all play this game anyway so no point to fix anything - MMOs are designed to addict people.
So cry and play.