Main Menu

Ratings School - Leagues vs Tournaments - Part 1

Ratings School - Leagues vs Tournaments - Part 1

Comparing Player Rating Calculations

Monday, December 19, 2016 - 21:33

In 2012, the PDGA began a new program called PDGA Leagues in which disc golf clubs that were already running weekly leagues at their local courses could sanction their leagues with the PDGA. Although there are several benefits to PDGA Leagues, it's fair to say that the most appealing benefit is that rounds thrown by PDGA members will be counted towards their PDGA Player Rating. However, that same appealing factor is also the one that has sparked the most common misconception about PDGA Leagues; lower ratings at PDGA Leagues vs PDGA-sanctioned events, on the same courses. 

If you already understand how Player Ratings work, skip to the Busting the Lower League Ratings Myth section. For those of that need a quick refresher, see the terms and definitions below. More details can be found on the PDGA Player & Course Rating System page as well as the Ratings FAQ page

Scratch Scoring Averages (SSA) - Your PDGA Player Rating (PR) is a number that shows how close your average round scores are compared to the course ratings for the courses you played at PDGA events, These course ratings are officially referred to as the Scratch Scoring Averages, or simply the SSA. Players that average the SSA on courses played will have a rating of 1000 and are considered “scratch players”. 

NOTE: It's extremely important to understand that the SSA for a course is NOT a static number. The SSA for a course changes EVERY round, with each new SSA determined directly by the propagators that competed during the round. Speaking of which...   

Propagators - PDGA members (current membership not required) that have a Player Rating based on at least eight rounds. Only the scores of the propagators are used to determine round ratings. On a PDGA event's results page, propagators have Player Ratings listed in bold.

Calculating Player Ratings - Player Ratings are calculated using a complicated algorithm, but the overall concept can be understood by anyone. From the very beginning, the Player Rating system calculations were set up so that the sum of the unofficial ratings earned by the propagators each round, whether in tournaments or leagues, will be the same or close to the same as the sum of their PDGA Player Ratings, regardless of how easy or tough a course played for that particular round.

When the round scores are uploaded, the algorithm determines the course SSA factoring in the average Player Rating of the propagators and the average of the scores they turned in for the round. Once the SSA is calculated, the round score for a 1000-rated round has been set, and all other round ratings can then be processed.

Busting the Lower League Ratings Myth

It's true. Ratings are in fact lower at PDGA Leagues compared to PDGA-sanctioned tournaments on the same course(s), but it isn't the round ratings that are lower, it's the course rating - the SSA. This simple fact is the basis for all of the confusion around the issue. However, a lower SSA during a league is completely normal, consistently, across the board, and should not be a concern. Players do NOT receive lower round ratings at leagues as a result. 

Not sure you believe this? Follow along and we'll show how you can prove this yourself for any PDGA event or league. 

The table below shows a group of 10 propagators with PDGA Player Ratings ranging from 820 to 1000 and an average of 910, which is close to what the average rating is for the entire PDGA member base. We'll pretend this group first played in a tournament with money and/or prizes on the line and then played in a league several days later with every player shooting three better than what they shot in the tournament. Why three better at league you ask? Good question.

An average difference of three between tournament and league rounds is much more common than unusual. We consistently see the same group of players carding scores that average anywhere from one to five lower at leagues than in tournaments on the same courses with the same layouts. In fact, in all of the research we've done, we've yet to find a single course where the players are averaging higher scores at league. 

Tournament vs. League - Scoring & Rating Example

Propagator Tournament League
Name Player
Rating
Score Round
Rating
Score Round
Rating
A 1000 53 1015 50 1022
B 980 58 970 55 974
C 960 57 979 54 984
D 940 63 925 60 926
E 920 62 934 59 936
F 900 69 870 66 869
G 880 67 889 64 888
H 860 70 861 67 859
I 840 73 834 70 830
J 820 67 889 64 888
Totals 9100 --- 9166 --- 9174
Averages 910 --- 917 --- 917
Scratch Scoring Averages 54.7 ← SSA→  52.3  

In the table above, the total round ratings for the Tournament (9166) and the League (9174) are nearly the same, both with an average rating of 917. This average is seven points better than the 910 average rating of the propagators. This indicates that quite a few extra rating points are being pumped into the system to partially account for players improving rather than declining.

This example shows how the very common lower round scores we see at leagues also lowers the SSA. The course SSA dropped from 54.7 at the tournament to 52.3 at the league, meaning a 1000-rated round requires a score 2.4 throws lower at the league than it did at the tournament. However, scan the round ratings received by each player and you'll see that they are about the same at both the tournament and the league, and in many cases, the league round ratings are actually higher.

Simply put, a lower round score does not necessarily equate to a lower round rating. Just because someone shoots one throw better at league than they did at a tournament a few days prior doesn't mean they will receive a better round rating. The equality of these small differences between tournament and league ratings happen all over the world, but it's disguised unless you put in the work to understand why.

NOTE: If you are crunching the numbers on your own using an event's results page, there are two things to consider if your numbers seem off. 

  • Propagators do not have to be current PDGA members. A non-current member's Player Ratings does not display on PDGA.com, meaning you won't be able to include that person's Player Rating in your calculations, even though they are being used in ours.
  • Any propagator that earns a round rating more than 60 points below their Player Rating is excluded from the ratings calculations.

Higher Course Ratings at Tournaments - Explained

Once we gathered sufficient statistical evidence showing that players are consistently averaging lower scores on the same layout in similar conditions in league rounds when compared to tournament rounds, it naturally begs the question, "Why does this happen?". We know that the round ratings are calculated exactly the same way for tournament and league rounds. There's nothing in the math that would explain the difference.

We've learned through our research that when the SSA comes out significantly lower in one round vs. another that there is often times something physically different involved.between the courses. This can be, perhaps, a pin position or tee that was moved to a different placement and/or playing conditions that were significantly different (e.g. wind, rain, etc.). However, when comparing a league round to a tournament round on the exact same course layout in similar weather, more challenging factors such as nerves and pressure appear to be the reason, often times combined with a stricter adherence to the Official Rules of Disc Golf.

As we mentioned earlier, we've found no PDGA League that has produced a higher SSA on the same course than the SSA produced during tournament play. That's statistically unlikely unless there's something about tournament play that always makes the round just a bit tougher to play for everyone on average. That something is pressure.

paul_saves_push_putt_on_2_sudden_death.jpg

One of the greatest examples of a high-pressure showdown was the sudden death playoff between Ricky Wysocki and Paul McBeth at the 2014 Pro Worlds, played in front of nearly 1000 spectators.

If you're skeptical that tournament pressure is relevant, studies about the effects of putting pressure in ball golf should help sway your opinion. Tournament players intuitively know this to be true; that we're more nervous during tournament competition than we are during recreational rounds. Additionally, tournament play often means risk/reward decisions may be made differently. For example, we may play more conservatively, e.g.laying up more often during tournament play than we would during casual rounds where we're more likely to attempt shots where the negative consequences of missing can be very high. A casual round player would likely worry less about roll-aways, out-of-bounds lines, kicking off of trees, etc.

Consider this league vs. tournament ratings scenario if it were flipped around to where the PDGA had created the ratings system to be used only for league rounds in the beginning, several years before starting to rate tournament rounds. Players might be saying, "Nice! I get a better round rating now for shooting a 52 at this tournament than I ever got for a 52 at league." In a sense, that is what should be happening, because the data shows that it's simply a bit tougher to play in tournaments than leagues.

However, because the PDGA did start with tournament ratings over 10 years before league ratings, league directors and players are looking at it the other way around, saying, "Wait a minute...why are the ratings so low?"

We hope that after reading this article that the league directors and players that have been asking that very same question have a better understanding of what's going on. The data and stats comparisons don't lie. Even though the course rating/SSA is lower in leagues, propagators will still receive the same overall ratings for each round, regardless of how much easier a course plays at leagues when compared to tournaments.

In part 2 of "Ratings School - Leagues vs. Tournaments", we'll dive deeper into this subject. Make sure to follow the PDGA on Facebook and Twitter to be updated when it is published!

*Story by Chuck Kennedy #4949, PDGA Consultant and Ratings Committee Chair. Edited by Matt Gregoire #28647, PDGA Media Manager.

Comments

Submitted by Jeterdawg1 on

Nice read and makes sense. Could it also be familiarity with the course? In tournaments, maybe 50% of players are extremely familiar with the course (10+ rounds played), but for leagues, I imagine that jumps to 95%+. Surely that also causes better scores at leagues.

Submitted by cgkdisc on

Certainly there is likely more knowledge among some players in some leagues. But this differential effect between leagues and tournaments is seen all over the world in cases where the same player pool with good course knowledge is playing both leagues and events on the same layout. Then, there's the other end of the scale where few players have better knowledge or experience when the league rotates among several different courses or new courses or special layouts are used. The point again is, whatever the reasons may be, it doesn't lower player ratings in leagues.

Submitted by jetasun on

I am a proponent of the PDGA tracking league ratings separate from tournament ratings due to the anomalies pointed out in the article above and because statistics like these are dependent on data granularity to achieve better accuracy, meaning the more people who are entered into an event, the higher the probability that the results will provide an accurate output.

I suspect that many leagues are like the league I decided to play this summer, with relatively low player turnout, a more “casual play” atmosphere, and random groupings that mix professional with amateurs on the same card…it really did not feel like competitive sanctioned play (at least, not at my local league). If you are into casual play and want a PDGA rated round, it serves a purpose. If you are a competitive person looking for an accurate rating related to tournament play, find a league that consistently fills the field with lots of current PDGA members that compete on lots of courses.

Leagues certainly provide benefits to the growth of our beloved sport. But separating tournament ratings from league ratings may, in my opinion, foster more participation in both types of competition, while also retaining rating integrity.