PTS Participation & Experience Survey

StihlReign · September 19

A one player interview does nothing. There's 25M players per ZOS.

The community deserves a survey communication roadmap. Look at the forums (everyone on the forums has an account and a vested interest in providing feedback). The players deserve a clear plan that details how ZOS will move forward with the player's concerns after survey completion. Show the consensus. This would help build trust, boost participation, and ensure clear understanding of the survey's purpose and outcomes.

Key elements should include setting objectives, crafting clear key messages, selecting appropriate communication channels, establishing a timeline, and presenting a plan for how they'll act on the feedback received. Even if it's nothing more than "we hear you, but we're going to do nothing"...which is where we are now.

There are companies built to do this, reinvention of the wheel isn't necessary.

Varana · September 19

The PTS generally comes much too late in the process. When it arrives, and people find something to report right away, it'll be the third week of a five week cycle until there is any chance of seeing a reaction to that report. If something comes up later in the PTS cycle, the chances of fixes or corrections are even lower. That leads to the general impression that feedback is usually just ignored.

That even applies to simple bugs (not meaning they're simple to fix but that they're just obviously bugs, not new mechanics or combat changes). I get that the latter may be a matter of opinion, or that there is a different design goal behind them that are done for a reason even if I may not agree with them.
But there have been many, sometimes severe, bugs in the PTS that have been called out early in the cycle - and still went live. And if something's on the live servers, these issues will often persist for months or years. Or a decade.
Or the bug goes live but is then fixed in one of the next patches. That's especially frustrating because it creates the impression that PTS feedback is totally ignored and sent to the garbage bin but you're starting to listen to bug reports again once the PTS is over. That is most probably not how it works in the background, but that is how it feels on the other end.

Between the PTS and live, there should be several weeks that are just spent acting on feedback, especially bug fixes.

And that also is true for combat changes or other things like Accountwide Achievements (I'm still not over that one, especially how it was handled in terms of communication and feedback). In all of the more controversial changes in the last years, complete novels of constructive feedback were written that tried to keep the original intent in place but make it better.
Still, crickets. And tumbleweed. Dozens upon dozens of pages of feedback and bug reports and not a single reply.

Just telling the reason for something - that is possible. Do it while your marketing and PR overlords are on the toilet, or whatever.
If you don't have the manpower to do something - you can just say that.
If the game is too old and was written in a way that makes something impossible - you can just say that.
If something takes more time than it looks from the outside - you can just say that.

In the end, the PTS experience has become deeply frustrating. It's okayish to just get a sneak peek at a new thing. But that's not what I think it should be about.
If you do think that that's what the PTS is for - just as a PR tool to get some hype or whatever - you can just say that.

And maybe get rid of the bug reporting and feedback thing so people use it as intended and don't get a false impression.

In the end, that's why I mostly stopped participating in the PTS. It's not worth my time and disk space. If I find a bug, it will go live anyway, so I might as well just not bother.

Finedaible · September 19

A little survey of my own for ZoS:

1. Is there anything worth testing after downloading the game for the PTS?
2. Is our feedback actually considered, or is only feedback with absolute, 100% validation and undying praise submitted for review?
3. What is being done about the bugs reported in week 1 yet did not receive a fix before launch?
4. Do you reject this reality and replace it with your own?

Thank you for participating in this valuable survey. We may or may not have answers in 10 years.

ZOS_Kevin · September 24

Just wanted to circle back here and thank everyone who has filled out the survey so far. And anyone who provided additional context in this thread. We will be discussing this over the next few weeks once the survey has wrapped up.

For anyone interested in submitting their feedback, there is one day left to fill out the survey. The survey will close tomorrow.

Thanks again to everyone assisting us to make PTS a better tool for both you and the dev team.

Treeshka · September 24

I have also submitted my feedback. But i hope at some point we do not have to login to PTS to earn rewards in Live server.

Destai · September 24

ZOS_Kevin wrote: »

Just wanted to circle back here and thank everyone who has filled out the survey so far. And anyone who provided additional context in this thread. We will be discussing this over the next few weeks once the survey has wrapped up.

For anyone interested in submitting their feedback, there is one day left to fill out the survey. The survey will close tomorrow.

Thanks again to everyone assisting us to make PTS a better tool for both you and the dev team.

Can you provide a summary of the feedback so we're clear that we're being understood?

nightstrike · September 24

ZOS_Kevin wrote: »

Hi everyone! We put together a short survey to help us better understand ESO Public Test Server (PTS) participation, and how we can improve the overall experience. This survey will be open until the end of September 25. We’d love to hear from you even if you’ve never participated in the ESO PTS. Thank you in advance for sharing your thoughts with us!

Link to the survey: https://beth.games/46niSAe

["You" in the following context refers to ZOS, or the survey authors I suppose, not Kevin]

This survey, like all the others you've had recently, requires stating yoru age bracket vs simply "over 18" (notably, the choice list doesn't say that "under 18" will end the survey, despite this being an 18+ game). If you decline to do so, the survey ends immediately. This is absurd, unnecessary, and rules out the entire chunk of people who see that as a privacy issue. Age has no bearing whatsoever on the relevance of a person's response to the survey, and your continued insistance on requiring that demonstrates your lack of understanding of how to properly conduct market research.

Here are my answers, since I refuse to submit the survey under these restrictions:
1) Have you participated in the PTS?
Yes, several times

2) Motivation?
Other: To be able to experience things that are broken or blocked on live, such as quests I can't reply, or, prior to recent changes, the previous inability to access various tutorial quest lines

Also other: Initially, I participated in the PTS thinking that it would give me access to help shape game design decisions. That is completely unworkable as demonstrated by the horrendous accoutn wide achievements change, where the widespread animosity towards the idea was completely ignored. This of course happens with thousands of other changes, but that's the one that killed it for me.

3) How satisified?
Very dissatisfied

4) Why?
See answer to #2, also other. Play testing on the PTS is a total waste of time.

5) Desired changes?
Other: You need a practical way to handle changes that the community doesn't want. Currently, the process is to ignore all complaints and push it through. This defeats the entire purpose of asking for comments in the first place. The PTS is too late in the process to reverse course on bad decisions. Though frankly, even fixable things get ramrodded through. Comments from the PTS that highlight major flaws to an otherwise desired design change are also ignored, making the whole process, as in #4, a waste of time. You need a way to actually finish featuers before rolling them out.

6) Misc.
Generally speaking, product engagement is a paid gig, and the PTS is free. So you are benefitting from donated time and resources of customers that pay you for a product. I imagine that you would get better feedback if you compensated testers, as is normal for every other job on the planet.

Dolgubon · September 24

I'm a PC player, but I'd really love to see console players get access to PTS, for a reason that I don't think I've really seen discussed.

Right now, any console players that are interested in creating addons for console need to re-putchase the game on PC to be able to efficiently create addons. They need to pay for the privilege of volunteering their time to make the game better for other players. That's a pretty high barrier. Allowing console players into PTS would remove it, and we'd likely see a lot more console add-ons and ports of PC addons.

Personally, if I'd needed to buy the game again, Lazy Writ Crafter would never have existed. (At least, from me)