MOBA Competitive Analysis
2018
The project
Multiplayer Online Battle Arena (MOBA) games are notoriously difficult for first time users to learn. Due to their unique play style, even seasoned gamers may find the transition from a traditional FPS or RPG more difficult than expected. To exacerbate this, new players are often looked down upon in MOBA communities, making it even harder for them to make the transition and learn how to play.
​
The onboarding process of playing a new game is important for player retention and satisfaction, which makes it worth trying to solve this interesting and challenging user experience problem. The current study looks at the tutorial levels of three MOBAs:
​
-
Heroes of the Storm (HotS)
-
Defense of the Ancients 2 (DotA 2)
-
League of Legends (LoL)
​
The purpose of this research is to identify what challenges players face when loading up a MOBA for the first time, and what factors contribute most to their satisfaction. By analyzing three games, we present what each does best and worst so that a more rounded solution can be crafted.
User testing
20 users who had no experience playing MOBAs were recruited for this study. Each user played all three games, which were counterbalanced to account for order effects. Play sessions took about an hour and a half.
​
​
​
​
N = 20
1.5 hours
For each game, the player completed the tutorial level(s) and a game against bots.
3
Surveys
Users completed the NASA-TLX to assess workload and the Game User Experience Satisfaction Scale (GUESS) to assess satisfaction.
​
At the very end of the study, users completed a final survey that compared all 3 MOBAs against one another.
​
Method
1
Participant observation
Users were asked to think aloud as they played the game, and to verbally identify both positive and negative aspects of the game.
​
Semi-structured interview
After users had played through the tutorial(s) and against bots, we talked with them about their reflections on game play and comparisons with other games.
2
Overall Positives
Aesthetics
All three games were rated highly on visual aesthetics. Each games was appreciated for its own style, and users often remarked on the pleasing character design.
​
'It was fun and helpful tutorial, the graphics were super cool...I think the unique character design and skins are cool." - P7, LoL
"The graphics are fun and the character list seemed interesting when I saw it."
- P17, DotA 2
"I like the graphics, the way the game looked. The talents or abilities of the characters seemed cool and fun." - P18, HotS
Freedom of Playstyle
The extended options for character selection in the more advanced tutorials was noted as a positive experience, some noted they wish they could have selected their character earlier. Users identified that playing according to their preferred playstyle very early on was important to them. ​
I like how you can pick between 2 characters. I was more familiar with what she [Ashe] does, instead of having to learn all the guys [Garen's] skills. Options are good." - P20, LoL
"During the tutorial you could pick from only one of three [heroes], but there were others that I would want to try out. Basically would want to have options of the characters." - P14, HotS
Some users preferred to play a hero like Ashe, who stands in the back and fires attacks from a distance.
Others preferred a more direct hero like Garen, who is meant to be right in the middle of fights.
Tutorial Guidance
Users liked that the tutorials guided them through several aspects of the game, instead of simply providing large walls of text. Multiple types of guidance provided at once (e.g., LoL presents pop-ups and a voice-over narration) was particularly positive experience. This made it easier to understand how the game should be played.
"I really like these little bubbles [pop-ups]. They’re informative, tell you icons are in a simple way and give elaborations as to purpose and importance...I like the way she [the narrator] communicates. I like the guidance she gives in the tutorial."
- P18, LoL
"The guides were useful, the lit green arrow path told me where to go."
- P11, HotS
"I like that in the first one [tutorial] they show you the basics to destroy an enemy tower."
P20, DotA 2
Overall Negatives
Respawn mechanics
In general, players didn't understand the respawn mechanics and identified it as a primary source of frustration. Several players either did not notice their health draining, or were surprised at how quickly it drained. They often ran back and died in the same spot multiple times, not understanding why. This was noted especially often in Dota 2.
"Some hints on what to do to avoid the death when I die would be nice."
P7, DotA 2
"Every time I died was because of something stupid. It was hard to escape"
- P12, LoL
"When I died, there was so many things happening and I was kind of confused about what was happening when I was about to die."- P18, HotS
Repetitive Pop-ups
When approaching enemy towers without support from minions, players were reminded to spend their gold and stay behind their minions. While helpful at first, these quickly became annoying, as the users were often in the middle of something else, like running away from enemies or rushing to save their tower. This was found in League of Legends.
"I had just come from the store, and came back out and she said spend gold twice in a row. Did I do something wrong?."
P18, LoL
Comparisons
"Gold popup was annoying, I know and I don’t want to spend it." - P15, LoL
"It kind of sounded like she [narrator] was getting angry with me after the tutorial went on (like stay behind your minions. Maybe a popup warning initially and then blurb after that to not forget. I don’t want to hear that over and over.."
- P7, LoL
HotS was rated the highest for both playability and enjoyment, followed by LoL and DotA 2.
​
Users experienced the most frustration for DotA 2, and the least for HotS.
2
1
3
Final notes
While we are still crunching the data for results on satisfaction and workload differences between the three MOBAs, it's obvious from the qualitative data that HotS was the game of choice for new users. DotA 2 tended to punish it's players heavily with extended death timers and little feedback about what to do better. Several users quit partway through the DotA 2 tutorials, because they found them too frustrating. LoL had a better tutorial system, but the store was too confusing to new users, who were so busy trying to understand the basic mechanics that the constant reminders to spend their gold simply became an annoyance. However, it should be noted that since we ran this study, LoL updated their onboarding process to include a seperate tutorial for the store:
HotS tended to include the best parts of all three tutorials: step-by-step onboarding, a selection of heroes to choose from right from the beginning, and an aesthetically appealing environment to play in. Future updates may consider adding tips on why the player died, and how they might avoid making the same mistake again. Similarly, the respawn mechanic should be explained in more depth, as it was the most common source of frustration.