Should zos make LIVE tests a regular thing?

MincVinyl

Ideally we wouldn't have to do live tests, but spending a week or two every couple of months with incentives/bonuses would be more than manageable.

There are plenty of other mechanical layers of the game that zos could look into....(imo these should be looked into before gutting aoes but hey)

• Cp
• Procs
• Cross healing/buffing/set proccing (many ways to implement, best: 1-4players can do outside of group 5-24 players can only interact with group)
• Over time effects stacking (previously how the game worked, less calculations, more skill/class/morph diversity,
• The hammer and destroyable gates (Leads to faction stacks)
• Keep ticks being the go to ap farm (resource incentives could spread out playes during seiges)
• Mountspeed being too fast (people bypassing field fights just to get to the ball seige at keeps)
• Abilities with too many mechanics (think of how complicated dizzy swing has become since it was a long cast hard hit knockup)
• Catapult seige and persistent ground effects
• The anticheat system (cant see zos removing this in the long run, but if they could disable it for a week it would be interesting to see what happens)

Onefrkncrzypope

Yes

Well yes and no. Not for a month too long. A weakend research sure

MincVinyl

Yes

Onefrkncrzypope wrote: »

Well yes and no. Not for a month too long. A weakend research sure

Yeah timing of course would have to be decided along with the incentives

SeaUnicorn

Yes

I think it's a good idea to make targeted tests like that because they will be more accurate. PTS does not reflect all specifics of live, so a lot of stuff goes unnoticed and is only discovered on live when it's too late to fix and we get stuck with it for a whole quarter.

2 things that need to absolutely happen for this to be good practice:
1. Meaningful rewards for participants. We are basically doing free QA for ZOS, all labor needs to be rewarded.
2. Fixes for major issues need to be at a faster pace. This must include balance fixes too. Things tend to be over-buffed and over-nerfed quite often and then we are stuck with imbalances for a whole quarter. This results in massive loss of players and hurts end game community a lot.

Skoomah

No

Can’t ZOS just hire staff to do the testing themselves?

Why do I have to lay a completely different game because the are cheaping out on staff?

They have logs of what everyone is activating for skills. Create some bots that can mimic button mashing?

TequilaFire

No

Create a special live campaign for it and leave the normal campaigns in peace please.
Call it a test campaign and leave it up.

Kadoin

Yes

PTS doesn't have enough people anyway...Bandwith isn't free either.

MincVinyl

Yes

Skoomah wrote: »

Can’t ZOS just hire staff to do the testing themselves?

Why do I have to lay a completely different game because the are cheaping out on staff?

They have logs of what everyone is activating for skills. Create some bots that can mimic button mashing?

You are asking zos to hire on 80+employees and teach them the game, hope they abide by the same meta when they can do it for free?

Pr0Skygon

Yes

Skoomah wrote: »

Can’t ZOS just hire staff to do the testing themselves?

Why do I have to lay a completely different game because the are cheaping out on staff?

They have logs of what everyone is activating for skills. Create some bots that can mimic button mashing?

I think you have the wrong idea my friend. ZOS is proposing the live server test, since there are certain test that requires a ton of players to make a good sample size.

Pr0Skygon

Yes

IMO, this is by far the best solution for the performance issue. You cannot fix an issue involving millions of players, with the sample size of a few hundreds on PTS.

Ajaks

Yes

They can open a Cyrodiil PVP campaign only for testing. I'd gladly take part in it if Zenimax stays open on the ideas for changes and the reasoning behind them. PVE will be harder to implement in similar ways however.

catnamedwill

Problem with this approach is that there are so many things to test, there will always one test or the other going on.

artal

No

Running some test is not issue. Changing game and running tests for 1 MONTH is issue.
They should make test campaign and reward players playing therr some way. Forcing broken game play on everyone is really... Well anoying tu put it mildly. and vast majority of players have strong feelings about every single proposition they made for this tests.

If you pay for the game you want to have fun playing character you like the way you enjoy. And not spending whole month playing either broken char, or what most people will do, single target build as much as you can.
This test will show nothing. I can already see cyrodiil full of nbs.
Have fun testing I'm gonna hit the beach some more. And if any form of cooldown goes through gonna finaly hang my weapons

Banana

Yes

They need to try something new.

Skoomah

No

Pr0Skygon wrote: »

IMO, this is by far the best solution for the performance issue. You cannot fix an issue involving millions of players, with the sample size of a few hundreds on PTS.

You do realize that they have player caps in cyrodiil. The zone gets no where near millions of people. It can barely handle 50 people on the same screen at once...

Skoomah

No

MincVinyl wrote: »

Skoomah wrote: »

Can’t ZOS just hire staff to do the testing themselves?

Why do I have to lay a completely different game because the are cheaping out on staff?

They have logs of what everyone is activating for skills. Create some bots that can mimic button mashing?

You are asking zos to hire on 80+employees and teach them the game, hope they abide by the same meta when they can do it for free?

Yes. I am asking for ZOS to properly staff their company so they can release products that are polished and major bugs free, have enough staff to address historical issues and new issues quickly. How many sloppy releases and sloppy patches and sloppy class balance attempts do you want continue experiencing?

They charge us one chapter per year for the cost of full blown games with way more content. They charge monthly subscriptions on top of that. I’m sure they can afford and should want to afford it if they want to grow and maintain the game.

Xvorg

Skoomah wrote: »

MincVinyl wrote: »

Skoomah wrote: »

Can’t ZOS just hire staff to do the testing themselves?

Why do I have to lay a completely different game because the are cheaping out on staff?

They have logs of what everyone is activating for skills. Create some bots that can mimic button mashing?

You are asking zos to hire on 80+employees and teach them the game, hope they abide by the same meta when they can do it for free?

Yes. I am asking for ZOS to properly staff their company so they can release products that are polished and major bugs free, have enough staff to address historical issues and new issues quickly. How many sloppy releases and sloppy patches and sloppy class balance attempts do you want continue experiencing?

They charge us one chapter per year for the cost of full blown games with way more content. They charge monthly subscriptions on top of that. I’m sure they can afford and should want to afford it if they want to grow and maintain the game.

though I agree, I think it won't be possible at this point. It could have been a good option when it was a Pay 2 play game like 6 years ago and there were little content (just base game)

Runefang

No

No, from a software development perspective making changes in live without sufficient testing is a bad idea. Production is a sacred place.

I can understand why they're doing this, simulating this test must be so difficult for them that a live test is preferable. But generally even performance testing like this should occur in a non-production environment.

Wolf81

Yes

Yes every patch cycle they should be doing tests on both the algorithms & spaghetti codes along with stress tests on their hardware regarding megaservers. I still believe they need a seperate server hub for cyrodiil to alleviate not only the stress on their megaserver but pvp in general...spend money to make money.

idk

Unfortunatly the poll is biased by the spin put into it by listing what the OP thinks should be tested vs a clean poll about focused testing on areas known to have significant impacts on game performance.

Elsonso

No

Live testing is not for the faint of heart. We have too many people with faint hearts. Let them rest in peace.

idk

Elsonso wrote: »

Live testing is not for the faint of heart. We have too many people with faint hearts. Let them rest in peace.

Specific well-intended and thought out testing is beneficial. However, most of the things OP lists as the focus of this poll have already been tested very thoroughly, such as CP, or are known to not have a meaningful effect, such as destroyable gates since we would be stacking at a keep if not at the gate.

Mayrael

No

Yeah... Because placing some prescripted bots simulating players actions in highly controlled environment and conditions isn't the way they could do test. It could not only show them general direction but also biggest offenders (skills) that cause most load, because same skill loops could give them repeatable results easier to analyze. Increase the amount of bots untill you recieve satisifing results. You stress test the servers in controlled way, trying to meassure what causes most load, then try to analyze what causes that load, maybe some nested methods or functions, maybe it's recurency.

Testing this on live will not give them representative data, as those testing conditions will lead to people changing their builds and the way they play, to adjust to those changes. Just like when we had siege bug. People started to spread more, lag was almost gone, conclusion? Lower damage of sieges causes more server load :P

TwinLamps

Yes

Yes
Especially for PvP

Vildebill

I think live testing is a good thing if it doesn't break stuff and make the game unplayable. Especially if there's an incentive to join the testing, like double AP. It's extremely hard to mirror a live environment in a testing bubble, especially with all the players.

Asardes

Yes

PTS is of limited use anyway, and of no use for stress tests, since few people are on it at any time. A lot of stuff couldn't be tested properly, for example the multi-bid system for traders last year.

Nemesis7884

Yes

Onefrkncrzypope wrote: »

Well yes and no. Not for a month too long. A weakend research sure

not weekends...a lot of ppl can only play on weekends and probably dont want to be guinnie pigs all the time

idk

Nemesis7884 wrote: »

Onefrkncrzypope wrote: »

Well yes and no. Not for a month too long. A weakend research sure

not weekends...a lot of ppl can only play on weekends and probably dont want to be guinnie pigs all the time

So people who work weekends and can only play during the week can deal with having to be guinea pigs all the time? That seems a little unfair to give such favoritism to one group. Just make the test last all week long to be fair.

CaperGuy

Yes

I think it's fair when it comes to real big core changes like this, I.E. - fixing bugs that happen from game design itself + the impact on the server -only- in stressful conditions.

The alternative is likely years of guessing games as to what they -think- they should do to help, which could result in never getting anything truely root cause fixed. They should absolutely do what they need to do to get the data, to fix/improve things.

Elsonso

No

idk wrote: »

Elsonso wrote: »

Live testing is not for the faint of heart. We have too many people with faint hearts. Let them rest in peace.

Specific well-intended and thought out testing is beneficial. However, most of the things OP lists as the focus of this poll have already been tested very thoroughly, such as CP, or are known to not have a meaningful effect, such as destroyable gates since we would be stacking at a keep if not at the gate.

Well, to be honest, I treated those things in the OP as "for example" not as "we should do these"... no matter what the intent was. I answered based on the question in the title.

Temeraire507

Yes

They should absoluetely do more live test events. Give players a pet or whatever for helping out and participating just like with the livestreams, to make sure more people help out.

There is a limit on how much data PTS and hired testers can provide. There's no way you would hire enough people to lock cyro just for one test and on live people are also experienced what can't really be said for testers you just hire once or twice. Live test events are the most effective and sometimes even the only way to get meaningful results on how changes would affect the live environment.