Has the PTS in its current form outlived its usefulness?

moderatelyfatman

I understand the purpose of the PTS: it allows keen players to test out the upcoming changes and gives devs some valuable feedback.
However, one of the problems is that only a tiny percentage of the player base use the PTS, so the feedback given tends to be skewed towards the loudest voices.

Most other MMOs do not have a PTS, instead relying purely on internal but extensive testing. Do you think ESO might be better off using this system (Class Reps, external Dedicated play test groups etc) instead of just relying on feedback from the PTS?

colossalvoids

Yes

Long time ago I'd say. It's more of a cosmetic preview and a resource for the new content to come pre-solved by competent community members which often kills a lot of enjoyment of exploring and theory crafting on day one onwards.

It's also more of a marketing device nowadays, thanks to miners and how people looking forward only to new cosmetic options for the next patch. Looking at amount of reports that are acted upon compared to for example Summerset PTS (was a closed one), where most things were ironed out before release whilst nowadays almost every single bug and issues is staying until couple first incrementals after release. And most issues are known literally day one of PTS, we always reported those in masses but nowadays it feels like report system sends them to the void.

Ph1p

moderatelyfatman wrote: »

Most other MMOs do not have a PTS, instead relying purely on internal but extensive testing. Do you think ESO might be better off using this system (Class Reps, external Dedicated play test groups etc) instead of just relying on feedback from the PTS?

Why not use both, which is most likely what ZOS is actually doing today?

Danikat

ZOS does have internal QA, and gets feedback from other sources as well as the PTS, among other things feedback from this forum has made it into the game, they've talked about responding to how they see people playing in the actual game, not just on the PTS and they've sometimes run surveys or contacted guilds and other groups for feedback on specific topics.

I stopped using the PTS to actually test upcoming things because it seemed like the feedback was hardly ever used. The only changes which happened as a result of PTS testing were minor adjustments to the numbers for certain skills, which isn't the kind of thing I'm intersted in testing.

I understand why that happens - they wait until a release is almost completely finished (usually even the voice acting is done) before they put it on the PTS and then there isn't time to make more than minor adjustments before it's released. But it still means it's a complete waste of time to give feedback that will never be used, so I stopped doing it.

BetweenMidgets

If ZoS listened to feedback and bugs found on the PTS there could be use. Past several years show us that isn't the case, though.

fizzylu

No

If anything, the problem with the PTS currently is that the testing phases are far too short and Zenimax usually ignores the valuable feedback (which btw, is a reason a lot of the PTS testers from back in the day stopped bothering with it). Considering how little they already take our suggestions, feedback, and opinions into thought when designing this game.... I'd say getting rid of the PTS is the worst thing that could happen at this point when it comes to communication between Zenimax and it's players.

As it is now, PTS is just a short bug test (sometimes even barely that) and a way for some content creators to showcase the new collectibles/etc. What should be getting suggested is longer PTS phases and earlier so that Zenimax could actually alter new features/content to align with the feedback they get instead of just marching along with what they alone think is best.

Elsonso

No

moderatelyfatman wrote: »

I understand the purpose of the PTS: it allows keen players to test out the upcoming changes and gives devs some valuable feedback.
However, one of the problems is that only a tiny percentage of the player base use the PTS, so the feedback given tends to be skewed towards the loudest voices.

Most other MMOs do not have a PTS, instead relying purely on internal but extensive testing. Do you think ESO might be better off using this system (Class Reps, external Dedicated play test groups etc) instead of just relying on feedback from the PTS?

As players here in the forum have noticed, ZOS is not exactly super good at listening to obeying PTS feedback. Yes, there are times when they appear to do that, but I think that is simply because the PTS feedback aligns well with their own thinking.

As for the alternatives... Class Reps and external dedicated test groups are an even more exclusive group than what we get from PTS. They will be a tinier percentage of the player base, and may have an even louder voice. Additionally, doing things this way risks building blind spots, as these testers could shift towards what ZOS wants to a significant degree.

PTS is open for any of the PC players to join and make comments. The forum has convenient collection threads, but these are sort of a necessary evil. While they provide feedback, they are an echo chamber containing a mix of players that have been on PTS, have not been on PTS, and just agree or disagree with what sounds good or bad.

The main problems with PTS is that it is limited to PC players, leaving the console players in the back seat and just along for the ride, and it is only used at the very end of the development cycle, after all of the ("bad") decisions have been made.

Gabriel_H

Maybe (comment below)

Maybe, but not for the reasons you gave. I used to always go on the PTS. I found many a bug. I reported them all. 99% of the bugs made it into release.

As for other MMOs, there are plenty with test servers. Those that don't isn't because they have great internal QA, it's because they want to cut costs.

Soarora

No

Zos does use feedback from the PTS and I’ve seen minor things like floating carpets that I’ve reported get fixed. The housing community also uses PTS to plan builds.

The downside is people work on content on PTS first, so world firsts have been progging content before most people and I’m still mad about ZOS’ decision to remove the complete collection book. Have to dig through coffers to find sets and then reconstruct them anyways because they’re not in the right trait. I’ve stopped making builds.

LPapirius

Yes

The "testing" that takes place on the PTS should be done internally at ZOS. Or, ZOS should give players some kind of compensation for doing their R&D and testing for them.

spartaxoxo

No

ZOS makes changes literally every time based on pts feedback and testing. It might not be what players want quite often. But, changes are obviously made and the patch that goes live is always better than the one that PTS starts with

The_Meathead

Yes

It has if ZOS refuses to listen.

Which feels like a more common problem every Update.

Destai

Yes

Without clearly knowing why changes are posted to PTS, and what our input actually affects, I personally don’t find them particularly useful. Also, having to buy each expansion for PTS access now really limits it for me.

Thinking back to the AUA a few months ago, they had stated there’s a few reasons why PTS is used. It seems there’s an open door policy on feedback but it’s not necessarily going to sway any decisions. Knowing which design decisions are open to change based our feedback will cut down on drama.

I feel like they’re mostly open to tweaking combat numbers. Other suggestions like those around events, cosmetic changes, character design, quest givers, wayshrine placement, etc., rarely get changed in PTS. Sometimes they change things after it launches, which again, makes me wonder why they’re even asking us on PTS when it seems like only post-launch feedback truly sways them.

I do also think they need to be better about changing course when something doesn’t land well on PTS. Seeing something go live despite vociferous opposition is pretty confusing and ultimately, frustrating. More communication is needed when these things happen.

Artim_X

No

The answer is having it all!

I agree ZOS needs better internal/external testing and the Class Reps system restored in some manner. However, the PTS is always useful and a true boon for those who engage with it.

tsaescishoeshiner

If you follow PTS patch-by-patch, you'll see player feedback, bug reports, and critique patched before the live update, or notes for future updates. You'll also see a lot that isn't fixed in those 6 weeks, especially bigger picture things or bugs that aren't priorities to work on.

SeaGtGruff

No

One of the purposes of the PTS that you didn't mention is the ability to create new fully-levelled characters to test out build ideas, or to try out an upcoming class, skill line, or other new feature.

Plus, the PTS lets players check out a chapter's or DLC's content before deciding whether or not to buy it.

So I do think the PTS continues to be relevant, whether or not players who do extensive testing on it and give feedback feel like their feedback is doing any good.

FelisCatus

Yes

Definitely pointless having it when ZOS doesn't listen to feedback.

Finedaible

The thing is, it has been implied for a long time now that there is extensive internal testing, yet there is still the PTS to supposedly identify and correct bugs that internal testing cannot uncover. Ironically the community reports issues in the very first week of a PTS cycle and those same issues still make it to live servers. Of course this is determined case-by-case but many major problems only get fixed or adjusted many months down the line after the damage has already been done. It seems like there is an inefficient workflow at play, and I don't think it is the tester's fault but there's no way to be sure. I mean, the fact that 'hybridization' is currently a 3-4 year ongoing (?) project that is still incomplete does not fill one with confidence that things are being worked on efficiently, it just makes the development seem directionless.

Rkindaleft

Maybe (comment below)

The PTS in it's current form is fine. The problem is how ZOS interacts with the feedback.

I used to participate in PTS, I stopped during U35 testing because we kept being told "we hear you." There's a difference between "we hear you" and actually being listened to, if you say "we hear you" then proceed to double down on changes nobody wants you may as well not have said anything at all because it's clear the feedback has not been taken into account.

It also feels like it’s meaningless doing bug reports because 95% of them make it live anyway.

Avran_Sylt

No

So you want to take the PTS, and restrict the number of participants?

It's very likely that the system that you described already happens in conjunction with the PTS.

DreamyLu

Maybe (comment below)

For me, the key question is: usefulness for whom?

I don't think that the main objective are players. I rather think that it's useful for ZOS. Taking into account the amount of efforts (and associated costs) involved to manage a PTS version parallelly to the game, if ZOS would get no benefit out of it, they would have stopped long ago.

For the players, benefit seems to be limited (reported bugs don't get fixed?). But maybe the feeling to be part of game development? And to see things in advance?

Personally, I don't touch PTS simply because ESO content being rather "small", the PTS is too big a spoiler. I like as much discovery as possible after download, lasting long enough.