Class Performance in VMA - Statistics

LightYagami

Hi everyone,

I did a brief study about class performances on VMA leaderboard (not the weekly one).
I collected the data around a week ago but didn't have time to tidy up the data until now. Scores of top 50 of each class were summarized (totally 4 x 50 = 200 top players).

I've been working as an analyst for quit some years and I'm familiar with data.

As we know, VMA is a soloing challenge, so by evaluating performance in VMA, we can better focus on individuals and eliminate effects of others. In addition, simply spending longer time won't make people become one of the tope players on the leaderboard.
The reason I didn't include the two new classes is that VMA is an old content and people are less devoted to run VMA leaderboard.

This is not the best way to study class performance, but as ZOS does not publish better data like K/D/A ratios of different classes in death match, player base by class in Cyrodiil, frequency of class skills appearing on death recaps on PvP, damage dealing per second in PvP, etc., what we can do is study the leaderboards. My original intention was studying PvP performance, but Cryodiil leaderboard can be reached by simply playing for long time and with influences from others.

There's a huge problem that leaderboards CAN'T tell us whether the toons are stamina or magicka based. It will be much better if ZOS study death recap data.

A few Key highlights of the 1st table:

1. DK is obviously lagging behind. They have the lowest average, the 2nd lowest max, the lowest min. and the lowest median scores. Max and Min may be skewed by extreme data, median may be a better indicator.
Their Standard Deviation is much higher than other classes. High standard Deviation means that performance of the class is very wide-spread, some of them can be very good (but still not as good as other classes) and some are very bad, i.e. it's a difficult class.

2. Templars are also relatively weak. They have the 2nd lowest average score, the lowest maximum, and the 2nd lowest median score.
Their Standard Deviation is the lowest meaning that performances of different people are close. i.e. this is a consistent class and easy to perform, although the overall performance may not the the best.

3. NB is the best performing class as many players expected. The highest average, the highest max, the highest min, and the highest median. The sorcerer class is right behind NB.


The observation can be easier elaborated by the 2nd table with the chart below. I ranked all scores of the 200 players, and separated them in the slots of 20. Then I looked into contributions of

LightYagami

Oh sorry I clicked a wrong button... it was not finished yet...

Please see below for the summary of the 2nd table:

The observation can be easier elaborated by the 2nd table with the chart below. I ranked all scores of the 200 players, and separated them in the slots of 20. Then I looked into contributions of every slot.

For example, the first bar means the among the top 1 to 20 players, 2 of them are DKs, 11 of them are NBs, and so on...

From the 2nd table and the chart, it's much easier to visualize the difference between DKs (and Templars) versus NBs and Sorcerers...

NBs do exist at any positions lower than 120, while most DKs are below 120...

These are hard data.

I always wish to talk with data rather than only personal feeling.

I hope ZOS study their data and tell the players their summaries. They have the resources of data and time so I trust they can study in a much better way.

Gundug

It’s an interesting look at the scores, but not enough information to really make a meaningful conclusion about the classes, since it’s only 200 characters, and we have no idea of the relative skill level of the players and their mastery of each class. Some of these scores may be from the same player running different classes as well.

Unfortunately, we do not have access to a much wider range of scores, which might tell a different tale regarding player experience with the classes in the arena.

There can also be a certain bias based on player perception of what is the best to use, resulting in a higher use of that class by high end score pushers.

I did not follow the reasoning behind not including the Warden and Necromancer classes, however. It seems to me they are as relevant as any of the others.

LightYagami

Gundug wrote: »

It’s an interesting look at the scores, but not enough information to really make a meaningful conclusion about the classes, since it’s only 200 characters, and we have no idea of the relative skill level of the players and their mastery of each class. Some of these scores may be from the same player running different classes as well.

Unfortunately, we do not have access to a much wider range of scores, which might tell a different tale regarding player experience with the classes in the arena.

There can also be a certain bias based on player perception of what is the best to use, resulting in a higher use of that class by high end score pushers.

I did not follow the reasoning behind not including the Warden and Necromancer classes, however. It seems to me they are as relevant as any of the others.

As I said, these are one of the only data we can get. I have my own full time job as a financial / business analyst, and I won't have time to do detailed studies of a game. These works should had been done by ZOS and they are better to let players know.

There are many limitations, and I already mentioned some of those, but I trust it's better than knowing nothing.

Another key point is that I'm trying to start some data conversation, if anyone want to add or make new studies, it will be very good. People usually talk with emotion without any objective supports. ZOS usually don't let us know objective data.

I hope it can be a triggering point to encourage players and ZOS talk with data.

SidraWillowsky

A few things (I'm a statistician btw )

1. In the future, I'd recommend pulling data for Wardens as well. Magwarden is one of the easiest classes to run vMA on (and a few months ago the world record vMA score was on a magden), so I think it would be worth it to pull the data for the class if you decide to do more analyses. May as well pull Necros too, though I rarely see them used in vMA... that's an entirely anecdotal observation, however.
2. I'd be interested to see a concurrent analysis of the top 200 scores, period, to compare
3. For your rankings/bins, I'd be more interested to see average percentile by class (if you want to put the data into a Google spreadsheet, I'd be happy to take a look). Now that I think about it, though, I'm not sure how useful this is without pulling the raw top scores, because I'd imagine that the percentiles look even worse for some classes because they may be separated by many more scores from other classes that didn't get pulled because you'd already pulled 50

I worry about the validity of these analyses because of the lack of necros/wardens, and the way the data were pulled. I think it would be more meaningful to pull the raw data instead of pulling by class.

Where are you getting the data? I tried eso-database and am seeing a blank table for vMA.

LightYagami

More information:

Usually people always say that a particular class is too weak, please make it stronger, or a particular class is too strong, please nerf it.

There's no data supporting the statements of "too weak" / "too strong". It creates sort of undesirable discussion atmosphere as people are debating without supports / evidences and they get emotional...

If ZOS let players know more official data, it will be good for the whole community.

zvavi

LightYagami wrote: »

More information:

Usually people always say that a particular class is too weak, please make it stronger, or a particular class is too strong, please nerf it.

There's no data supporting the statements of "too weak" / "too strong". It creates sort of undesirable discussion atmosphere as people are debating without supports / evidences and they get emotional...

If ZOS let players know more official data, it will be good for the whole community.

I think your statistics are better than overall statistics of completion per class, because a lot of completion per class is biased (known as the one bar pet sorc). Because yours paints how easy the class is for completion better.

LightYagami

SidraWillowsky wrote: »

A few things (I'm a statistician btw )

1. In the future, I'd recommend pulling data for Wardens as well. Magwarden is one of the easiest classes to run vMA on (and a few months ago the world record vMA score was on a magden), so I think it would be worth it to pull the data for the class if you decide to do more analyses. May as well pull Necros too, though I rarely see them used in vMA... that's an entirely anecdotal observation, however.
2. I'd be interested to see a concurrent analysis of the top 200 scores, period, to compare
3. For your rankings/bins, I'd be more interested to see average percentile by class (if you want to put the data into a Google spreadsheet, I'd be happy to take a look). Now that I think about it, though, I'm not sure how useful this is without pulling the raw top scores, because I'd imagine that the percentiles look even worse for some classes because they may be separated by many more scores from other classes that didn't get pulled because you'd already pulled 50

I worry about the validity of these analyses because of the lack of necros/wardens, and the way the data were pulled. I think it would be more meaningful to pull the raw data instead of pulling by class.

Where are you getting the data? I tried eso-database and am seeing a blank table for vMA.

I got informt from in game leaderboard, no idea why you saw a blank table.

You can do your own study if you have time.

If you have requests, please ask ZOS to do official studies.

LightYagami

zvavi wrote: »

LightYagami wrote: »

More information:

Usually people always say that a particular class is too weak, please make it stronger, or a particular class is too strong, please nerf it.

There's no data supporting the statements of "too weak" / "too strong". It creates sort of undesirable discussion atmosphere as people are debating without supports / evidences and they get emotional...

If ZOS let players know more official data, it will be good for the whole community.

I think your statistics are better than overall statistics of completion per class, because a lot of completion per class is biased (known as the one bar pet sorc). Because yours paints how easy the class is for completion better.

Thank you!

I hope ZOS will do their studies and let us know the results. They should have the time and resources to do that, and it is their job to do that.

Probably they won't let us know the results, bit at least they will know what's happening by classes. A key point we don't know is stam vs magicka performance. It may be obtained from death recaps, bit it's extremely time consuming for players to do voluntary work... Hope developers will do their jobs...

LightYagami

Just to clarify, I'm not a staff of ZOS. I have to use my own time to do even simple studies. I'm not doing in depth study, and basically I'm trying to encourage players, more importantly ZOS, to do more object studies and let us know the results.

If someone wants more information, please take some time to do your own study. It's always good to have any kind of sources to support your view.

I have a Bachelor degree in mathematical science, and a Master degree in accounting, and I'm a licensed financial risk manager, working in a multi-national company. I'm familiar with data but won't have the time analyze a game in depth.

ZOS, please consider to publish your official data by class performance. Let us know what's happening with different classes.

Juhasow

LightYagami wrote: »

I'm familiar with data but won't have the time analyze a game in depth.

Took it out of my mouth. You know how to count statistics but in context of ESO You lack knowledge to understand them and potential mistakes You can make.

Oh and when it comes to sources of data other then leaderboard You can try ESO logs.

idk

I would suggest there are much better sources of data to analyze class performance. ESO Logs for trials would be significantly better at analyzing DPS performance in actual situations.

SidraWillowsky

idk wrote: »

I would suggest there are much better sources of data to analyze class performance. ESO Logs for trials would be significantly better at analyzing DPS performance in actual situations.

My hesitation there comes from the fact that many people don't take logs. I've been on the vMA leaderboard quite a bit but have never remembered to take logs. It also looks like OP is analyzing score data going back since vMA was introduced (2015? 2016?), so we're not going to have the logs from those runs anyway, since logs didn't exist back then.

I think it's pretty easy to predict which classes aren't going to do as well in vMA- most stam builds, and magDK and magplar (if you're using the melee-range jabs as your spammable).

A more apt analysis might include looking at *how* much worse some classes are than others. OP kind of gets at that in the second part of their post, but it's not accounting for any class scores that exist after they grab the top 50 scores. For example, there may not be a whole lot of overlap in the lists between the top 50 nightblades and the top 50 DKs. In that case, all 50 NB scores will get pulled before the top 50 DK score list even starts, so the NBs with higher scores than DK who aren't in the top 50 won't be counted in between the top 50 DKs.

Logs might still be interesting, and the lazy side of me gravitates toward that, since I think I can get the scores and classes output into a .csv file...

Ragnaroek93

You won't reach a balance in a solo arena nor should that be the goal (unless you want every class to be the same). Magblade is probably the best spec for vMA but it's not very good in PvP for example. And vMA is doable with every spec in the game, it's not like some classes or specs are excluded.

The only issue I see is how these arenas heavily favour ranged specs over meele specs, there are so many mechanics which slow down meele specs while there are barely any mechanics against ranged specs. Imo there should be far more reflects and negates to slow down ranged specs a bit. When I look at both arenas I don't see a single encounter which favours meele over ranged and that's an issue to be honest (Vateshran last boss mechanic has a ridiculous hitbox for meele specs for example).

idk

SidraWillowsky wrote: »

idk wrote: »

I would suggest there are much better sources of data to analyze class performance. ESO Logs for trials would be significantly better at analyzing DPS performance in actual situations.

My hesitation there comes from the fact that many people don't take logs. I've been on the vMA leaderboard quite a bit but have never remembered to take logs. It also looks like OP is analyzing score data going back since vMA was introduced (2015? 2016?), so we're not going to have the logs from those runs anyway, since logs didn't exist back then.

I think it's pretty easy to predict which classes aren't going to do as well in vMA- most stam builds, and magDK and magplar (if you're using the melee-range jabs as your spammable).

A more apt analysis might include looking at *how* much worse some classes are than others. OP kind of gets at that in the second part of their post, but it's not accounting for any class scores that exist after they grab the top 50 scores. For example, there may not be a whole lot of overlap in the lists between the top 50 nightblades and the top 50 DKs. In that case, all 50 NB scores will get pulled before the top 50 DK score list even starts, so the NBs with higher scores than DK who aren't in the top 50 won't be counted in between the top 50 DKs.

Logs might still be interesting, and the lazy side of me gravitates toward that, since I think I can get the scores and classes output into a .csv file...

The same hesitation of people taking logs applies to using the vMA leaderboard as it only looks at some players.

Further, if OP is going back to the beginning of thee vMA leaderboards or even across multiple years, they are not getting the picture of how the classes fare today. That has changed.

Basically, the information from vMA does not represent how the classes perform outside of vMA. The sorc is a great example of this. It has great self heals so it has been able to do fine in vMA even when it was underperforming outside of vMA. The self heals is probably why both the sorc and NB perform so well in the analysis provided.

In the end, the analysis is great. Thank you for the information as it is very interesting. However, it really only applies to vMA as my guess is the OP is experienced enough in data analysis to recognize this.

Runefang

One thing I’d mention here is that a class which is easy to complete vMA with doesn’t mean it makes the best class for getting top scores. You will see NB and Warden at the top for scores but still double pet sorcs are easier for the average player.

Also the best vMA players will run the best class for clearing and ignore others. This creates a skew that suggests the under played classes are worse than they seem.

Juhasow

@SidraWillowsky OP is analyzing scores from 3 weeks time period between launch of markarth and 1 week ago.

SidraWillowsky

Juhasow wrote: »

@SidraWillowsky OP is analyzing scores from 3 weeks time period between launch of markarth and 1 week ago.

When they say that they're analyzing overall leaderboard vs. weekly, do those only apply to the current update? I was thinking that the leaderboard was across the game's lifespan and was wondering why that existed, but it makes much more sense for it to only include info on this update.

In that case, I think it's difficult to draw any conclusions here because the game is so broken.

Juhasow

SidraWillowsky wrote: »

Juhasow wrote: »

@SidraWillowsky OP is analyzing scores from 3 weeks time period between launch of markarth and 1 week ago.

When they say that they're analyzing overall leaderboard vs. weekly, do those only apply to the current update? I was thinking that the leaderboard was across the game's lifespan and was wondering why that existed, but it makes much more sense for it to only include info on this update.

In that case, I think it's difficult to draw any conclusions here because the game is so broken.

Every major update there is complete wipe out of leaderbards.

LightYagami

Juhasow wrote: »

LightYagami wrote: »

I'm familiar with data but won't have the time analyze a game in depth.

Took it out of my mouth. You know how to count statistics but in context of ESO You lack knowledge to understand them and potential mistakes You can make.

Oh and when it comes to sources of data other then leaderboard You can try ESO logs.

This website?
https://www.esologs.com
Oh nice it's quit interesting! I'll Tata look at this. Hope ZOS will do something similar too.

VoidCommander

Not to be that guy, but utilizing proper grammar in your report goes a long way to displaying competence in your research. Getting distracted by 7 errors in your first post negatively affects your presentation, regardless of how good the information actually is.

LightYagami

Ragnaroek93 wrote: »

You won't reach a balance in a solo arena nor should that be the goal (unless you want every class to be the same). Magblade is probably the best spec for vMA but it's not very good in PvP for example. And vMA is doable with every spec in the game, it's not like some classes or specs are excluded.

The only issue I see is how these arenas heavily favour ranged specs over meele specs, there are so many mechanics which slow down meele specs while there are barely any mechanics against ranged specs. Imo there should be far more reflects and negates to slow down ranged specs a bit. When I look at both arenas I don't see a single encounter which favours meele over ranged and that's an issue to be honest (Vateshran last boss mechanic has a ridiculous hitbox for meele specs for example).

Death recap data may be helpful to know more about skills... We have limited accessibility on that...

In fact I'm a veteran Battlefield player, had been playing the game for around 10 years until I quited during the controversy BFV. There were a few websites providing excellent statistics like KD by classes, killing per minute by weapons, data on different game modes, time spent by players, accuracy of weapons, etc. I love that website. There was a famous website Symthic providing nice summaries of weapon characteristics, bullet spread patterns, etc.

Runefang wrote: »

One thing I’d mention here is that a class which is easy to complete vMA with doesn’t mean it makes the best class for getting top scores. You will see NB and Warden at the top for scores but still double pet sorcs are easier for the average player.

Also the best vMA players will run the best class for clearing and ignore others. This creates a skew that suggests the under played classes are worse than they seem.

Unfortunately we don't have the accessibly of most data... It's one of the easiest data to collect, and easy to study as a player...

The log website some fellows talked about is informative, good to know that

idk

Runefang wrote: »

One thing I’d mention here is that a class which is easy to complete vMA with doesn’t mean it makes the best class for getting top scores. You will see NB and Warden at the top for scores but still double pet sorcs are easier for the average player.

Also the best vMA players will run the best class for clearing and ignore others. This creates a skew that suggests the under played classes are worse than they seem.

This is what I was getting to earlier. All this data shows is what is easiest to do well in vMA. The data is irrelevant when looking at what the best class is for DPS, or any role, in a HM trial or even PvP.

It is by no means an indicator of how the classes are balanced overall.

But again, thx for the info because it is interesting to see it.

SidraWillowsky

Juhasow wrote: »

SidraWillowsky wrote: »

Juhasow wrote: »

@SidraWillowsky OP is analyzing scores from 3 weeks time period between launch of markarth and 1 week ago.

When they say that they're analyzing overall leaderboard vs. weekly, do those only apply to the current update? I was thinking that the leaderboard was across the game's lifespan and was wondering why that existed, but it makes much more sense for it to only include info on this update.

In that case, I think it's difficult to draw any conclusions here because the game is so broken.

Every major update there is complete wipe out of leaderbards.

TIL

@LightYagami I realize I'm being way too critical- thanks for doing this! Lots of inspiration.

LightYagami

SidraWillowsky wrote: »

Juhasow wrote: »

@SidraWillowsky OP is analyzing scores from 3 weeks time period between launch of markarth and 1 week ago.

When they say that they're analyzing overall leaderboard vs. weekly, do those only apply to the current update? I was thinking that the leaderboard was across the game's lifespan and was wondering why that existed, but it makes much more sense for it to only include info on this update.

In that case, I think it's difficult to draw any conclusions here because the game is so broken.

If you're interested in the current performance, it may be better to collect data across a few weeks to diversify abnormalities in one particular week, but it's even more time consuming.

ZOS should improve the transparency on their side, instead of leaving their duties to third parties...

LightYagami

SidraWillowsky wrote: »

Juhasow wrote: »

SidraWillowsky wrote: »

Juhasow wrote: »

@SidraWillowsky OP is analyzing scores from 3 weeks time period between launch of markarth and 1 week ago.

When they say that they're analyzing overall leaderboard vs. weekly, do those only apply to the current update? I was thinking that the leaderboard was across the game's lifespan and was wondering why that existed, but it makes much more sense for it to only include info on this update.

In that case, I think it's difficult to draw any conclusions here because the game is so broken.

Every major update there is complete wipe out of leaderbards.

TIL

@LightYagami I realize I'm being way too critical- thanks for doing this! Lots of inspiration.

No problem. If you have time and interested in other data, please do some studies. As mentioned a few times, I'm not an employee of ZOS and won't be able to do deep studies. I'm a finance professional in real life, I just want to relax with gaming after my work

One of my aims is simply reminding the community the importance of giving opinions more objectively. I noticed that many comments on the forum are based on feelings which could be bad for the development of the game.

Some YouTubers published their class rankings with fancy radar (spider web) charts, however they don't give data supports but only their experiences. (I'm not blaming them because I know we don't have the data, it's important for ZOS to do that and let us know.)

As I mentioned on another reply, I'm experienced in Battlefield and back then we had many good sources of data (mostly from third parties but there were still some official data. Yes, even EA did that and I trust ZOS is better than EA).
There were far fewer arguments between players based on personal feelings like ESO... I hope ZOS can do something similar, if possible.

Kurat

It's no secret that sorcs and NBs are ez mode for any content, esp solo.
If you wanna get on leaderboard play DK coz only 2 other people play them.

LightYagami

Kurat wrote: »

It's no secret that sorcs and NBs are ez mode for any content, esp solo.
If you wanna get on leaderboard play DK coz only 2 other people play them.

Lmao
Indeed...
I guess population by classes or frequency of using skills also tell us something. People stopped playing some classes or using some skills with reasons.

LightYagami

idk wrote: »

Runefang wrote: »

One thing I’d mention here is that a class which is easy to complete vMA with doesn’t mean it makes the best class for getting top scores. You will see NB and Warden at the top for scores but still double pet sorcs are easier for the average player.

Also the best vMA players will run the best class for clearing and ignore others. This creates a skew that suggests the under played classes are worse than they seem.

This is what I was getting to earlier. All this data shows is what is easiest to do well in vMA. The data is irrelevant when looking at what the best class is for DPS, or any role, in a HM trial or even PvP.

It is by no means an indicator of how the classes are balanced overall.

But again, thx for the info because it is interesting to see it.

Please do some studies per your interest and share us your findings.

Juhasow

LightYagami wrote: »

Kurat wrote: »

It's no secret that sorcs and NBs are ez mode for any content, esp solo.
If you wanna get on leaderboard play DK coz only 2 other people play them.

Lmao
Indeed...
I guess population by classes or frequency of using skills also tell us something. People stopped playing some classes or using some skills with reasons.

What if I told You that even when mag dk's were top DPS class not that long ago with very easy rotation and many top scores in trials , in vMA they were still one of the weakest classes score wise and one with the smallest amount of participants. To understand why You need to know a little bit more about the game then just recent leaderboard scores.

Also as I mentioned earlier Your data covers only 3 weeks time period because leaderboards reset with every major update.

kojou

I run it on my Warden because it is easy to get on (and stay on) weekly with the Warden... My reason has nothing to do with the overall power of the class in my case.

Nightblades were considered one of the easiest ways to get a clear because of Swallow Soul (RIP Funnel Health) and Impale being one of the better executes in the game. The healing over time coupled with shields was enough to keep you alive without costing you too much in terms of defense and self healing.

Sorcerers have Crit Surge, Shields, and the Matriach, so they have even more defense and can dish out a lot of damage while passively healing. They don't have the execute that Nightblades have, but have much better defense.

The Pale Order ring renders all of this irrelevant at this point though and is likely the Combat Team's answer to equalize all classes for solo content. With that ring you can be successful with any class in VMA, so we are likely to see more players venture in with their favorite classes if they haven't already.

I can also run VMA with relative ease on my Magicka DK at this point and I don't have to change anything in terms of skill load out because I get plenty of healing with the Pale Order ring where before I had to do silly things like back bar a Resto staff.