There's been a lot of talk about seeding lately, especially for The Grand Melee. One alternative seeding method could be as follows:
1. Select all matches played between qualified players (ranked and unranked, only 1v1, only RM/EW)
2. Simulate a ladder by replaying the matches in chronological order using Elo, trueskill, etc
3. Use rankings as seeds
This method has some useful properties:
- Blends ladder and tournament matches
- Excludes irrelevant matchups (ladder vs lower players, orthogonal tournaments, etc)
- Doesn't rely on subjective point awards or multipliers
- Reduces likelihood of manipulation
- Resistant to inactivity in ladder or tournaments
And some downsides:
- Only helpful for top-50ish since it exploits the right side of the skill bell-curve resulting in lots of common matchups
- Relatively opaque - can't look at a spreadsheet to easily see how a player got their seeding
Using the prediction scoring proposed by felixferoc we can compare seeding methods for a few recent tournaments:
- "Sim" is the simulated ladder seeding
- "Actual" is the seeding that was actually used in the tournament
- "ATP" and "aoe-elo" are using *current* snapshots - so they overperform a bit since they include adjustments based on the outcome of these events whereas "Sim" only includes matches prior to tournament start
Warlords
RBW
TTL
Posting this to generate some discussion on how to improve, pitfalls, etc. Not suggesting to use this for a real tournament in this state.
1. Select all matches played between qualified players (ranked and unranked, only 1v1, only RM/EW)
2. Simulate a ladder by replaying the matches in chronological order using Elo, trueskill, etc
3. Use rankings as seeds
This method has some useful properties:
- Blends ladder and tournament matches
- Excludes irrelevant matchups (ladder vs lower players, orthogonal tournaments, etc)
- Doesn't rely on subjective point awards or multipliers
- Reduces likelihood of manipulation
- Resistant to inactivity in ladder or tournaments
And some downsides:
- Only helpful for top-50ish since it exploits the right side of the skill bell-curve resulting in lots of common matchups
- Relatively opaque - can't look at a spreadsheet to easily see how a player got their seeding
Using the prediction scoring proposed by felixferoc we can compare seeding methods for a few recent tournaments:
- "Sim" is the simulated ladder seeding
- "Actual" is the seeding that was actually used in the tournament
- "ATP" and "aoe-elo" are using *current* snapshots - so they overperform a bit since they include adjustments based on the outcome of these events whereas "Sim" only includes matches prior to tournament start
Warlords
Code:
Player Sim Actual ATP aoe-elo Curr RM 1v1 Max RM 1v1
---------- ----- -------- ----- --------- ------------- ------------
hera 1 7 7 1 1 1
tatoh 2 2 1 4 16 13
theviper 3 1 3 3 4 3
accm 4 12 6 5 8 11
nicov 5 11 11 9 2 5
yo 6 6 2 6 3 7
liereyy 7 4 10 2 6 6
villese 8 5 5 7 5 4
vinchester 9 3 4 8 7 9
mbl 10 10 16 10 10 2
jordanaoe 11 8 9 12 12 8
valas 12 14 15 11 9 12
classicpro 13 16 13 16 11 14
capoch 14 13 12 14 14 15
daut 15 9 8 13 15 16
miguel 16 15 14 15 13 10
Method Total Ideal Accuracy
----------- ------- ------- ----------
Sim 580 860 67%
Actual 550 860 64%
ATP 480 860 56%
aoe-elo 530 860 62%
Curr RM 1v1 530 860 62%
Max RM 1v1 570 860 66%
RBW
Code:
Player Sim Actual ATP aoe-elo Curr RM 1v1 Max RM 1v1
--------- ----- -------- ----- --------- ------------- ------------
hera 1 6 6 1 1 1
tatoh 2 7 1 4 16 12
theviper 3 1 3 3 4 3
villese 4 5 4 7 5 4
yo 5 3 2 6 3 6
mbl 6 9 14 9 10 2
liereyy 7 2 9 2 7 5
accm 8 10 5 5 8 10
dogao 9 13 11 13 6 9
sitaux 10 11 13 8 2 8
jordanaoe 11 8 8 11 11 7
valas 12 14 12 10 9 11
daniel 13 15 15 15 14 16
capoch 14 12 10 14 12 14
daut 15 4 7 12 13 15
kingstone 16 16 16 16 15 13
Method Total Ideal Accuracy
----------- ------- ------- ----------
Sim 460 700 66%
Actual 310 700 44%
ATP 480 700 69%
aoe-elo 360 700 51%
Curr RM 1v1 200 700 29%
Max RM 1v1 240 700 34%
TTL
Code:
Player Sim Actual ATP aoe-elo Curr RM 1v1 Max RM 1v1
------------- ----- -------- ----- --------- ------------- ------------
tatoh 1 4 1 4 23 14
hera 2 10 7 1 1 1
theviper 3 1 3 3 5 4
jordanaoe 4 9 9 11 14 9
vinchester 5 3 4 8 10 10
villese 6 7 5 7 6 5
mbl 7 20 19 10 12 2
yo 8 2 2 6 4 8
liereyy 9 6 10 2 9 7
nicov 10 13 11 9 3 6
daut 11 8 8 12 17 21
accm 12 5 6 5 11 13
barles 13 18 14 13 2 3
classicpro 14 16 13 17 13 17
dogao 15 12 17 14 7 12
bact 16 14 18 18 24 20
capoch 17 11 12 15 18 18
saymyname 18 24 23 23 21 24
slam 19 19 21 20 15 19
miguel 20 17 16 16 16 11
badboy 21 21 20 22 19 16
songsong 22 22 22 21 8 15
thedragonstar 23 15 15 19 20 22
bruh 24 23 24 24 22 23
Method Total Ideal Accuracy
----------- ------- ------- ----------
Sim 1960 2670 73%
Actual 1850 2670 69%
ATP 1880 2670 70%
aoe-elo 1680 2670 63%
Curr RM 1v1 1320 2670 49%
Max RM 1v1 1410 2670 53%
Posting this to generate some discussion on how to improve, pitfalls, etc. Not suggesting to use this for a real tournament in this state.