Mountain Project Logo

Route Ratings Changed to Consensus


nathanael · · Riverside, CA · Joined May 2011 · Points: 518
Brian wrote:Yeah, I have to agree with a bunch of the posts that this is going to invite people screwing with the grades on purpose like the recent posting of the Dawn Wall at 5.7. I've seen routes intentionally rated ridiculously low to skew the consensus grade. Example: mountainproject.com/scripts… I climb a lot at the Gunks so almost all roofs out west seem soft to me so I'm going to grade them low not necessarily according to the area. This could potentially cause havoc.
It takes the median not the mean. It takes the median not the mean. It takes the median not the mean.

As you can see from your very own example, the outliers don't matter.
rgold · · Poughkeepsie, NY · Joined Feb 2008 · Points: 526
grog m wrote:The gunks...the route desciptions say things such as "anywhere else this would be 5.10". Okay, why not just grade it 5.10? Ego? Douchism? Whats the point of grades if not to provide the climber a universal idea of how difficult the terrain he will encounter may be. Hopefully this new consensus grading system will "fix" the gunks.
Ha! As if...

Really, the "problem" is much more substantial than just the Gunks. It may be that grades should be universal, but they aren't and never have been. Grades reflect, first of all, some of the history of climbing in a region and, second of all, the local specialties required by the area's rock. Either of these two can have a substantial effect, offsetting the grades by at least one notch and not infrequently more than that. What isn't really at play is ego and douchism, at least not in general.

Leigh Ortenburger made a grand attempt in 1963 at creating a national climbing classification system that would make grades at all areas comparable. (supertopo.com/climbers-foru…) The effort failed, and no one has even begun to try to revive it.

The fact of the matter is that grades within a region are often fairly inconsistent, so the problem of consistency across regions isn't even the first thing one would want to tackle. But one does hope that at least within a region the grading will provide---for the accomplished climber---an idea of how difficult the local terrain will be.
Tom Nyce · · Flagstaff, AZ · Joined Nov 2010 · Points: 45

My preference is that rating within an area should be as consistent as possible. Those climbs are all in one guidebook, and there shouldn't be 5.8's that are harder than some other 5.9's for instance. When visiting climbers come to the area, they tackle some climbs cautiously to get used to the local rock type and ratings, and then can count of the guidebook consistency after that.

When you allow a "consensus" of climbers (often including tons of visiting climbers, not used to that particular style of climbing) to chime in on the ratings, some climbs get changed (to match climbs in totally different areas where the climber is from). But, not all of them get changed, and the "consistency" of the ratings in that area suffer. Due to this effect, I've found that the older guidebooks to an area are often more internally consistent than the newer ones. Of course the older books have generally stiffer ratings, but they don't seem to have the unpredictable scatter that the newer books have.

I'm talking about trad climbing rather than sport, because the local rock types make such a difference in the style required, and that is mainly what I have experience with. Of course, I'm not opposed to fixing up some true "sandbags," or over-rated climbs, that are not rated consistently with the other climbs in that same area.

Brian · · North Kingstown, RI · Joined Sep 2001 · Points: 740
grog m wrote:The gunks...the route desciptions say things such as "anywhere else this would be 5.10". Okay, why not just grade it 5.10? Ego? Douchism? Whats the point of grades if not to provide the climber a universal idea of how difficult the terrain he will encounter may be. Hopefully this new consensus grading system will "fix" the gunks.
I don't think the original ratings had anything to do with ego and douchism. Those were just the prevailing grades at the time. When the early Gunks climbers graded a 5.6 that was considered a hard climb in those days. If you look at the early grades as being the standard then everything else has been grade creep since then. Although as I mentioned earlier I like grade creep. It makes me a harder climber without working out.
Jon Frisby · · midwest/west circuit (Indiana) · Joined Feb 2013 · Points: 120
rgold wrote:I think Bill has a point, but it works both ways. Climbers who have specialized in one genre of climbing naturally tend to overrate less familiar types of climbing. And when it comes to trad climbing, there is the prospect of undergrading by people who have not "really" led the route. If we are going for precision, then reporting the standard deviation would be nice too. In principle, half the responders think the climb is as difficult or harder than the median grade, and half the responders think it is as difficult or easier than the median grade. I think there is quite a difference between nearly full agreement and a spread of ratings as much as, say, two full grades between the hardest estimate and the easiest. The standard deviation would at least alert the reader about the precision of the consensus grade.
Rich - take a look at redriverclimbing.com That's got a pretty good example of what you're referring to in terms of the standard deviation in a good visual depiction
Dylan B. · · Unknown Hometown · Joined Mar 2006 · Points: 521

I think this is much ado about nothing. Most folks will use the original rating or the current consensus rating as a point of reference, and only deviate one or two grades in either direction. As a rule, we don't conjure our grade judgments from thin-air, but rather begin with what others say about the climb, and how it compares to similar climbs, and then upgrade or downgrade based on a comparison. A few outliers will make radical grading decisions, but by using the median those will have minimal effect.

Loganator · · blue van, on the highway to no · Joined Jan 2012 · Points: 240

I imagine this affect will change the grades of all the classic climbing areas. Yosemite, Jtree, Index, the Gunk's etc will all see grade inflation. I have a feeling that this will cause a lot of internal bickering. But seeing how everyone on this forum is probably just wasting time at work anyways, I don't really see a problem with it.

I highly suggest trying to climb pitches without knowledge of the grades/beta. Usimg your eyes to judge a route, or the suggestion from a friend that such and such is a good route has always pushed me to climb better. This technique makes every climb much more enjoyable in my opinion.

William Thiry · · Las Vegas · Joined Dec 2011 · Points: 45
Loganator wrote:I highly suggest trying to climb pitches without knowledge of the grades/beta. Using your eyes to judge a route, or the suggestion from a friend that such and such is a good route has always pushed me to climb better. This technique makes every climb much more enjoyable in my opinion.
+1 on that, Loganator. Your statement captures more of the spirit of climbing than grade chasing.
Ryan Nevius · · Unknown Hometown · Joined Dec 2010 · Points: 898

It looks like the consensus grade shows up in search results, but that the search is still retrieving matches based on the original grade. For example, if I search for .10a routes in the Estes Valley, I end up with results for 5.8-5.10b routes (consensus grades): mountainproject.com/scripts…

Is this the intended functionality? It's a bit confusing to see 5.8 or 5.10b routes if I'm looking for 5.10a specifically (just as an example).

Nick Wilder · · Boulder, CO · Joined Jan 2005 · Points: 3,879
Ryan Nevius wrote:It looks like the consensus grade shows up in search results, but that the search is still retrieving matches based on the original grade. For example, if I search for .10a routes in the Estes Valley, I end up with results for 5.8-5.10b routes (consensus grades): mountainproject.com/scripts… Is this the intended functionality? It's a bit confusing to see 5.8 or 5.10b routes if I'm looking for 5.10a specifically (just as an example).
Oops - fixed! Thanks for the report.
MattH · · Unknown Hometown · Joined Sep 2011 · Points: 420

Here's a perfect example of why listing the "median" grade suggestion fails when there are few route grades: mountainproject.com/v/omaha…. The route is a consensus 14a but one guy with zero site contributions dropped a 5.12 rating last year and instantly the route gets downgraded to 5.12 and shows up in searches as 5.12 because the only other rating on MP is from the original posting. I'd rather find a route by its original grade and find out later that it's soft/hard by looking at the consensus and seeing who ticked it and their reasoning.

SCherry · · Boulder, CO · Joined Jan 2006 · Points: 561

I totally agree with MattH.

The route grade should be listed as what the locals or guidebook authors believe it to be (or its original grade by the poster). Then the consensus on MP listed so if you want info on what other people on MP think you can find it.

His example of Omaha Beach is exactly why...lots of routes like this on MP.

Plus the consensus grades on MP are always jacked up by a sandbagger or someone who thinks routes are sandbagged...everyone I know takes them with a grain of salt.

SteveF · · Fort Collins, CO · Joined Aug 2007 · Points: 31

I doubt many old and sandbagged climbs will stray too far from their original grade due to anchoring bias.
In the thread "Lies, Damned Lies, and Statistics" teece303 makes a pretty good point worth repeating.

teece303 wrote:Anchoring bias play a big part in reducing deviation. Look it up if you've never thought about it. Once a route is assigned a grade, people will, in general, have a very hard time suggesting a grade very far from the assigned grade, for fundamental psychological reasons. And that's true even if they think the assigned grade is just plain wrong. I've seen this in action. Climber think she is warming up on a 5.9+. The route is VERY hard. Half way up her and the belayer think this is more like 10+ or 11-. Pretty stiff for a 9. When they figure out what route they were on, it's a 12a. This climber climbs 12a, so she knows what that feels like. But because she was anchored to the belief that the route was a 9+, she was unwilling to de-anchor and stray that far from the assigned rating. It's a fascinating psychological problem. So not only are grades somewhat subjective (although, they are nowhere near as subjective as many climbers imply), and a grade will vary by climber (some routes exploit our weakness or play to our strengths, and thus are legitimately different to different climbers), but ALSO, the very act of assigning a number to them makes it difficult to get an "accurate" grade of the route, if that initial number is somehow "wrong."
The "Wisdom of the Crowd" may work surprisingly well with groups guessing a value that's objective when individuals aren't influenced by the opinions of others. It's likely less effective when dealing with something like climbing grades which are somewhat subjective and individual opinions are heavily influenced by the existing grade.

I'd love to see a normalization of grades across areas so I know what to expect when I get on a climb anywhere in the world. But I'll bet localization in grading will remain a fixture of the climbing world for many years to come. Crack climbs in sport climbing areas like Shelf Road will still be rated much harder than the same climb if it existed in a crack climbing area like Indian Creek. Many people recognize this fact, but the problem still remains. See the comments for I, Claudius at Shelf Road for an example.
Nick Wilder · · Boulder, CO · Joined Jan 2005 · Points: 3,879
MattH wrote:Here's a perfect example of why listing the "median" grade suggestion fails when there are few route grades: mountainproject.com/v/omaha…. The route is a consensus 14a but one guy with zero site contributions dropped a 5.12 rating last year and instantly the route gets downgraded to 5.12 and shows up in searches as 5.12 because the only other rating on MP is from the original posting. I'd rather find a route by its original grade and find out later that it's soft/hard by looking at the consensus and seeing who ticked it and their reasoning.
This is either wrong, or someone JUST added another grade to it. Both the consensus and original grade are 14a, and the 12 is still in there. And it shouldn't work that way - I added code a couple days ago that biases a route towards it's original grade in the cases where there are very few votes.
Dylan B. · · Unknown Hometown · Joined Mar 2006 · Points: 521
SteveF wrote:I doubt many old and sandbagged climbs will stray too far from their original grade due to anchoring bias.
+1. This is my hunch, I just didn't know the term.
Shirtless Mike · · Denver, CO · Joined Mar 2006 · Points: 5,410
Nick Wilder wrote: This is either wrong, or someone JUST added another grade to it. Both the consensus and original grade are 14a, and the 12 is still in there. And it shouldn't work that way - I added code a couple days ago that biases a route towards it's original grade in the cases where there are very few votes.
Thanks for fixing this Nick, my only 5.14 and Mountain Project had it downgraded to a 5.12 :)
MattH · · Unknown Hometown · Joined Sep 2011 · Points: 420
Nick Wilder wrote: This is either wrong, or someone JUST added another grade to it. Both the consensus and original grade are 14a, and the 12 is still in there. And it shouldn't work that way - I added code a couple days ago that biases a route towards it's original grade in the cases where there are very few votes.
It's definitely the latter case - when I posted, there were 2 grades, the original 14a post and a second 5.12 post. The consensus was listed on both the page and the sidebar as 5.12. Now there's a 3rd grade, of 14a, which pushes the median back to 5.14a. Every route with 2 grades that I've seen has this problem - it takes the 2nd grade suggestion as the median instead of the first. An example from my ticklist that I noticed: mountainproject.com/v/try-o… was v8+ when submitted but is now v7 after just one v7 suggestion.

As I mentioned earlier, it might be useful to have a "silent grade suggestion" of sorts for people who tick a route and don't give a grade suggestion because they tend to do this only when the listed grade is seen as accurate and this listed grade can now change. If you gave them a half-weighted silent grade suggestion of whatever the listed grade was at the time they ticked it, you'd likely have more accurate consensus grades. For all ticks before the switch, it'd be easy to figure out the listed grade at the time of the tick, as it'd just be the submitted grade. Now it might take a bit more work obviously.
Nick Wilder · · Boulder, CO · Joined Jan 2005 · Points: 3,879
MattH wrote: It's definitely the latter case - when I posted, there were 2 grades, the original 14a post and a second 5.12 post. The consensus was listed on both the page and the sidebar as 5.12. Now there's a 3rd grade, of 14a, which pushes the median back to 5.14a. Every route with 2 grades that I've seen has this problem - it takes the 2nd grade suggestion as the median instead of the first. An example from my ticklist that I noticed: mountainproject.com/v/try-o… was v8+ when submitted but is now v7 after just one v7 suggestion. As I mentioned earlier, it might be useful to have a "silent grade suggestion" of sorts for people who tick a route and don't give a grade suggestion because they tend to do this only when the listed grade is seen as accurate and this listed grade can now change. If you gave them a half-weighted silent grade suggestion of whatever the listed grade was at the time they ticked it, you'd likely have more accurate consensus grades. For all ticks before the switch, it'd be easy to figure out the listed grade at the time of the tick, as it'd just be the submitted grade. Now it might take a bit more work obviously.
My prior claim was wrong - apologies! I wrote the code to bias the original grade, but forgot to run every existing route through it and get them updated. I've done so now. Hope I got it right this time. Please let me know if you see any more problems like this one.

Thanks!
nathanael · · Riverside, CA · Joined May 2011 · Points: 518
mountainproject.com/v/regul…

A slight problem is people logging aid routes as a free routes. For example the RNWF should show up as either 5.12 (actual free grade) or as 5.9 C1 (hardest mandatory free climbing), but people logging it as just "5.9" means the consensus grade is now 5.9, and it's showing up as the best 5.9 free climb in the Valley. [obviously this ignores the recent changes to the routes, a different discussion]
Christian Roda o Back · · Casa do Cacete · Joined Jul 2005 · Points: 1,486

Even w cutting out the outlier 13b and weighting the original rating more when there's less than 5 votes, how do the ratings below work out to a "median" of 5.8?

It seems like in these cases, the effect of the algorithm is not "overweight the original", but more like "take the original, period"? Perhaps a a case where a mean would be better?

Don't remember the bolt spacing, but from other comments, I'm not sure 5.8 leaders being super confident about getting on this route is necessarily a good thing.

Jump for Cholla

Guideline #1: Don't be a jerk.

Mountain Project News
Post a Reply to "Route Ratings Changed to Consensus"

Log In to Reply