Rating systems

Formula X

The Formula X rating system is being developed by DotA-GC's own technical expert Strom. Formula X is being designed specifically with DotA in mind and also to rate a single player very quickly. It doesn't just use the outcome of the team match, but also a lot of DotA specific data it can get from the WarCraft III replay file.Game length, hero kills, deaths, assists, creep kills, denies, neutral kills, leveling speed, items etc.There are also some specific actions that increase a player's rating e.g. buying wards. It's very important to understand that everything is calculated dynamically based on several factors. The system doesn't necessarily favor a soloing Lightning Revenant over a tri-lane Treant Protector, because there are hero based calculations in place. It's also a well known fact that heroes can kill creeps faster the more the game lasts, because everyone has better items and there are just more creeps. This has been addressed in the formula aswell. There are many other similar situations that have been addressed. Every single thing in the formula is calculated dynamically, this means that there are not any hard hitting thresholds e.g. if the game length is less than 40 minutes then give extra points. Everything is relative - when a player plays good versus weak players then it won't matter as much as if it would have mattered against good players.

Formula X works very fast compared to other more universal rating systems. It takes only about 30 games to make a solid judgement about a single player.

Tip: Want to get a high Formula X score? The best way to do this is to work as a team and win games as quickly as you can.

TrueSkill

The TrueSkill rating system was designed in the 21st century at Microsoft Research, with multi-player and multi-team games in mind. The system uses very complex formulas and has greater precision than any previous rating system. The system was not built from scratch, it has heavy influences from ELO and Glicko. TrueSkill tries to lower the number of games needed to get an accurate rating for a single player. The system is extremly dynamic, allowing both quick movements and being able to slow movements down. TrueSkill also changes the ratings of different players differently if they are in same team. At some point or another a players rating will pretty much freeze. This is the goal of the system. The system believes that it has found the true skill level of that player. Unless that player will show unpredictable performance the rating will not change. In simple words - unless the player improves his real skill the TrueSkill rating will not change.

TrueSkill matching is sometimes viewed as outrageous and plain stupid. The idea here is that the system tries to make two teams as even as possible by making the average player rating equal the medium skill level.

An example:

Team A
2 good players, 2 bad players, 1 medium player
> 2 good + 2 bad = 4 medium
> + 1 medium
> A total of 5 medium players

Team B
5 medium players

This matchup is viewed as a good match because both teams have 5 medium players on average. Now it is arguable whether this holds water in the dota sense. Those 2 good players in Team A might carry the game and pull Team A to victory. On the other hand those 2 bad players in Team A could feed so much that Team B will win. The point here is that this is arguable but there is no universal answer. Thus we are using the universal statistical approach, which states that 5 medium average equal 5 actual medium.

Another factor that plays a role in matchmaking is the player uncertainty. It is shown that a new player is level 0, but in reality he is at the average level, with a very high uncertainty. It's unwise to think of all new players as beginners in dota. They might have played 4 years, just the first time in our system. Now it is again arguable that the average new player in our system has a skill below the average skill level in our system. Even if this is true at the moment - it wouldn't be true for long because the new players would bring down the average skill level.

Now having a new player with the average skill level and high uncertainty, it's best to match that player against a proven average player. A match between a proven average player and a unproven average player has the highest probability to give the system valuable information. Matching a new player against a known good player would give a lot of information given that the new player wins. But the fact is that most of the time the new player will loose, giving almost no information. The same way - matching a new player against a known bad player would give a lot of information given that the new player looses. But yet again the fact is that most of the time the new player will win. Matching the new player (an uncertain average player) against a certain average player is good because the new player has a 50/50 chance of winning. This result will give us information whether the new player is better than an average player or not. This is not the best information, but we have a high probability of getting that information, that's why this path is the best.

The thing is that many players view new players as bad players. Unfortunately this is not helped by the level 0 designated to new players. Once players start thinking about new players as uncertain average players, then the teams might look just a little bit better.