ATC or Grigri? Official MP survey


Original Post
Corey Buhay · · Unknown Hometown · Joined Aug 2015 · Points: 0

Ever been dropped? Ever dropped someone? Collecting some data for a Mountain Project analysis to be featured on the blog in the next month or so. Goal: determining once or for all which is safer in the hands of an average user: an ATC or a grigri. Anonymous 60-second survey below:

https://docs.google.com/forms/d/e/1FAIpQLSdbzXqcLzDApcQwZ1pCmHGpkD5Q3O1sPfmzrbynvi-tJ-nkHQ/viewform


shoo · · Unknown Hometown · Joined Aug 2010 · Points: 0

You really think that a survey of people who self-select into taking your survey from a forum post has any validity whatsoever? This doesn't even pass the most generous sniff tests for survey validity, and serves only to mislead.

You want to answer this question for real? PM me, and we can talk. In the meantime, take the "official" down before you embarrass yourself.


Ted Pinson · · Chicago, IL · Joined Jul 2014 · Points: 40

There are mountains of scientifically collected data on this that are a much better basis for drawing a conclusion. I'll save you the Googling: Grigris are, by far, safer. Their accidents are just over reported because they're so surprising.


Medic741 · · Red Hook, New York · Joined Apr 2012 · Points: 85

Needs a comparison for how often each style is used.

E.g. I only leave the ground if belayed with a grigri, cinch, or jul, so if a user like me were to log a report the data is skewed without your dataset having a variable that you can use to correct for which device I choose to use


Corey Buhay · · Unknown Hometown · Joined Aug 2015 · Points: 0

Ted, that's what I'm trying to find out. If you have access to more scientifically collected data, please PM me; I'd love to see it! However, I've been unable to find any conclusive evidence for the safety of one device over the other (outside the gym) as of the posting of this survey.

Shoo, I agree that survey respondents will be self-selecting in that they're more likely to respond if they've been dropped, but I have no reason to believe Mountain Project users would be predisposed to use a Grigri over an ATC or vice versa. Therefore, a comparison of the two should still be valid if corrections for a small sample size are applied. And though any evidence will still anecdotal, some numbers would shed more light than current forum discussions do. This is also meant to be used in conjunction with other sources, not as standalone evidence. If you have any more concerns, please feel free to PM me; I appreciate your feedback.

Medic741 - That's an excellent point. Please mention that in the comment/clarification section of the survey, if you haven't already. I'll see if I can add another question.


shoo · · Unknown Hometown · Joined Aug 2010 · Points: 0
CoreyLynn wrote:Shoo, I agree that survey respondents will be self-selecting in that they're more likely to respond if they've been dropped, but I have no reason to believe Mountain Project users would be predisposed to use a Grigri over an ATC or vice versa. Therefore, a comparison of the two should still be valid if corrections for a small sample size are applied.
You are both wrong about your assumption that MP users (and moreso MP users reading a very specific sub-forum who bother to click your link) will not differentially respond based on own experience AND ALSO missing a hugely important second set of factors that are completely unaccounted for.

1) This is a charged issue, particularly for readers of this forum. People feel very strongly about "dangers" of the grigri. You are going to get a huge amount of people who are out to prove either that grigris are dangerous / totally safe based on their anecdotal experience. Your problem is that you have no way of knowing the relative strength of those biases. Some of those biases are in opposite directions.

You are making the mistake of believing that ambiguous net direction of bias strengthens conclusions, when the opposite is in fact true. If you know you are biased, but don't know what direction you are biased in, you cannot make any conclusions. If you know what net direction the bias is expected to be, you have a chance that the data will come out in such a way that some conclusions can still be made.

2) You are completely failing to understand the differences in how the two devices are used. The kinds of people and scenarios in which people use the two devices are totally different, and not in a way that is easily accountable (nor do you even attempt to do so with a BS measure of "years of climbing"). These are NOT fair comparisons. Again, you have an ambiguous direction of bias problem, complicated with the self-reporting biases outlined above, although in this case one general direction of bias is likely to be a bit stronger overall.

3) Forget about the "small sample correction" nonsense. That is not even close to your biggest problem here.

CoreyLynn wrote: And though any evidence will still anecdotal, some numbers would shed more light than current forum discussions do
No. Bad data masquerading as good data serve no useful purpose in public discourse. Your results are meaningless, and are actively misleading except in the absurdly unlikely case where all of the numerous biases in your survey magically cancel each other out.

Again, there are good ways to do this and actually put some real numbers on this, but you are not doing any of them. You have good intent, but you are doing a bad thing. Please cancel this.

Cory F · · Blacksburg, VA · Joined Jun 2016 · Points: 0

Is this for a school project?


JohnnyG · · Unknown Hometown · Joined Nov 2009 · Points: 0

CoreyLynn - I think it absolutely fantastic that you are doing this survey.

It seems the trick is you need a common denominator for comparing the two devices-- and this denominator is really hard to get. Like number of drops per number of falls. Looks like you are going to do drops per years of climbing. But then there is the factor that people who are catching lots of falls (sportclimbers) tend to use a grigri. Maybe people could estimate the number of falls they catch per year.

Shoo - sooo, rather than just take aim at this survey, how would you set up a survey that, in your **opinion**, is not biased?


Ancent · · Reno, NV · Joined Apr 2015 · Points: 37

Don't listen to Shoo: his comments and opinions are as meaningless as your internet survey may ultimately be. It's a MP-forum discussion after all. Nonetheless it's interesting data, and I enjoy seeing what people are using.


Morgan Patterson · · CT · Joined Oct 2009 · Points: 8,332

Surprised that twice as many people have reported being dropped on ATC vs Gri Gri. Well I mean not really but I didnt think it would be that high.

When your life depends on it, GriGri


EthanC · · Unknown Hometown · Joined Jun 2013 · Points: 219

Nothing to raise people's hackles like calling something official for no reason other than to sound important...

Also, where's the love for Smart/clickup/megajul style devices?

Wish there was a hear-say category...


Andrew Yasso · · Las Vegas, NV · Joined Sep 2012 · Points: 90

To the 3 people who have claimed to been dropped 10 times by their belayer... what?!


Greg D · · Here · Joined Apr 2006 · Points: 511
EthanC wrote:Nothing to raise people's hackles like calling something official for no reason other than to sound important... Also, where's the love for Smart/clickup/megajul style devices? Wish there was a hear-say category...
It worked for the Federal Reserve and Federal Express, neither of which are federal.

shoo · · Unknown Hometown · Joined Aug 2010 · Points: 0
JohnnyG wrote:Shoo - sooo, rather than just take aim at this survey, how would you set up a survey that, in your **opinion**, is not biased?
Basically any sampling strategy that relies heavily on self-selection into the sample is out, pretty much automatically.

Getting a real answer to this question would require some leg-work in the best case scenario, with reductions in bias depending strongly on just how much leg-work you're willing to do. An absolutely minimal amount of work would probably require cold call phone polling from a "climber" database (AMC, gym records, etc).

I am happy to do some pro bono work designing a study here, but my professional/research associations would prevent me from from doing much more than design and advising.

Ancent wrote:Don't listen to Shoo: his comments and opinions are as meaningless as your internet survey may ultimately be.
My "meaningless opinions" have credentials to back them, and you're almost certain to get the same answer from anyone with a relevant/comparable background. Once again, PM me if you want more info.

Owen Witesman · · Springville, UT · Joined Feb 2014 · Points: 53

Shoo is right in terms of survey design. This is not a sample survey that can be generalized to any population. It is only a questionnaire of those who respond.

And yeah, those people with 10 drops need to find a different hobby.


Morgan Patterson · · CT · Joined Oct 2009 · Points: 8,332
shoo wrote: my professional/research associations would prevent me from from doing much more than design and advising.
I wouldn't argue with this man about stats... sounds like you'll loose 100% of the time.

Cory F · · Blacksburg, VA · Joined Jun 2016 · Points: 0
Owen Witesman wrote:Shoo is right in terms of survey design.
Agreed. Is the survey for personal interests? Or part of something larger?

Brett Kitchen · · Unknown Hometown · Joined Apr 2016 · Points: 0
Cory F wrote: Agreed. Is the survey for personal interests? Or part of something larger?
You've asked twice now, but CoreyLynn says what the info is for in the originating post...

JohnnyG · · Unknown Hometown · Joined Nov 2009 · Points: 0
shoo wrote: Basically any sampling strategy that relies heavily on self-selection into the sample is out, pretty much automatically. Getting a real answer to this question would require some leg-work in the best case scenario, with reductions in bias depending strongly on just how much leg-work you're willing to do. An absolutely minimal amount of work would probably require cold call phone polling from a "climber" database (AMC, gym records, etc). I am happy to do some pro bono work designing a study here, but my professional/research associations would prevent me from from doing much more than design and advising. My "meaningless opinions" have credentials to back them, and you're almost certain to get the same answer from anyone with a relevant/comparable background. Once again, PM me if you want more info.
I see your point, and I agree with you.

This gets to really basic questions about data and decision making. I don't think it's appropriate to completely disregard this study because of self selection, especially since the alternatives is a much more costly study.

I work with a lot of imperfect data as a geologist, and we make decisions on imperfect data in all fields. Basically, we have to make decisions whether there is perfect data or not. And I would argue that some data is better than no data.

shoo · · Unknown Hometown · Joined Aug 2010 · Points: 0
JohnnyG wrote:And I would argue that some data is better than no data.
Some data is better than no data for those that have the education / training / experience / time / attention span to understand the limitations and biases of those data. Unfortunately, the typical MP user does not have those luxuries.

It is 2016. It is easier than ever to collect and spread information, good or bad. If it is more likely that the information that will be spread is misunderstood than understood by those who would be making decisions on that data, then no data is better than some data.

Knowing this, it is irresponsible, and possibly downright dangerous, to produce shitty stats and endorse them with an otherwise respectable brand, no matter how many disclaimers you use.

Cory F · · Blacksburg, VA · Joined Jun 2016 · Points: 0

Brett, I'm an idiot. I read the OP before coffee. embarrassing...


Guideline #1: Don't be a jerk.

Post a Reply

Log In to Reply