How would you rate Zenimax online studios on the Joel test?

Husan · March 2016

I ran into this on reddit, and I found it very amusing to run what I know about how ZOS operates through this simple test. Read about it yourself here: http://www.joelonsoftware.com/articles/fog0000000043.html

I'm not going to post my result just yet, but I can say it's not that great. Definetly not a passable grade.

What I want is to hear your opinions, how many points would you give to Zenimax on this test? Why? Where do you think they are lacking the most? Do you think they could improve on their work if they consciously started to include all 12 points in their work organisation?

Discuss

Phinix1 · March 2016

#5 seems to be where they suffer the most. They seem to have the same timeline priorities Miscrosoft did, just punching in the code clock and getting it out there, and the same lack of equally timely bug testing and resolution.

"...if you find a bug in code that has already shipped, you're going to incur incredible expense getting it fixed."

Case in point: Every time they completely break core functionality like video and audio rendering and push it out to live, then scramble for weeks or months "working on a solution."

How many people aren't playing and aren't paying for that time?

Better to relax on corporate deadlines and keep a stable product in the wild then push something out that breaks core functionality. You will lose more customers and customer confidence to the later than the former.

#10 seems relevant as well...

O.o

code65536 · March 2016

You realize that was from over 15 years ago, right?

Large-scale software development has evolved quite a bit since then. For example, of course everyone uses source control. It's so damn easy to set up these days even for amateurs and... how would you even go about without it? These days, even most amateurs use it. But 15 years ago, it was still a bit exotic to some.

And different types of projects require different things. For example, unit testing and automated regression testing is a very big thing these days, but those sorts of things work well in application development. In game development? Kinda hard to do, as you might imagine.

Anyway, as I've said in other threads, their biggest mistake with TG was announcing and setting in stone the March 7 release date. You can tell things were going to be rough when you saw that list of features that were coming but weren't enabled in the first PTS build because they're not ready yet, including the 64-bit client. A month is already a very short time to iron out bugs in a beta, assuming that the beta is the final product already. But when the beta has key components that aren't even done and ready for testing? You could tell from that first day of PTS that TG would not be fully ready come March 7. And lo and behold. And I doubt this has to do with development environment--this ultimately comes down to business decisions from up top. So the question is, are the suits making the decisions like "March 7 or bust" even listening to the front-line people working on the code?

Elsonso · March 2016

This checklist was intended to be answered by insiders, not customers. Customers have no idea what the answers are to those items, unless they are told the answers.

Do you use source control? Unless they say otherwise, we assume 'yes'

Can you make a build in one step? This is likely 'no' due to complexity

Do you make daily builds? We do not know the answer to this

Do you have a bug database? At least one

Do you fix bugs before writing new code? Almost no one does this, so 'no'

Do you have an up-to-date schedule? We do not know the answer to this

Do you have a spec? We do not know the answer to this

Do programmers have quiet working conditions? We do not know the answer to this

Do you use the best tools money can buy? We do not know the answer to this

Do you have testers? Yes

Do new candidates write code during their interview? We do not know the answer to this

Do you do hallway usability testing? We do not know the answer to this

Phinix1 · March 2016

lordrichter wrote: »

Do you have testers? Yes

How do we know the answer to this? Unless you mean paying customers on the PTS?

Elsonso · March 2016

Phinix1 wrote: »

lordrichter wrote: »

Do you have testers? Yes

How do we know the answer to this? Unless you mean paying customers on the PTS?

Well, aside from the fact that they have been seen on ESO Live?

Phinix1 · March 2016

Ah. I haven't watched enough of those. I've seen Gina, Kai, the sound guy, other devs, and some of the creative people like writers and the cute girl that does butterflies.