Strategy for debriefing & accident analysis - dissociating outcomes and decisions

This past weekend I took AIARE-1 with Marc Chauvin, and there was one point about risk management, decision making, and debriefing that I found particularly thought-provoking:

The nature of risk is that a good decision won't always lead to a good outcome, and a bad decision won't always lead to a bad outcome.

While obviously we want to make decisions in intelligent ways in order to mitigate risk, associating outcome with decision  too strongly--and in particular, letting ourselves assume that because things turned out well, we made good choices--can prevent us from learning from our decisions. It goes the other way, too: it's too easy to look at an accident and simply see bad outcome = bad decisions ("I would never do that").

Marc suggested a way to circumvent this that I really liked. When debriefing a trip that went well, ask yourself, "If we'd made the same decisions but they had led to an accident, what would the accident report look like? How would we have handled it, and how would we reflect on our decisions afterwards?"

Conversely, when reading an accident report, you can ask yourself, "If I were reading about the same set of decisions but they'd ended well, what would I think?" (It might be something along the lines of: "That person was bold! I wish I were bold/smart/strong enough to do that, too.")

I found this perspective really refreshing and plan to implement it in the way I reflect on decision making and risk in my climbing. Might make our reactions to accidents a little more circumspect/humane, too.

Etha Williams wrote: This past weekend I took AIARE-1 with Marc Chauvin, and there was one point about risk management, decision making, and debriefing that I found particularly thought-provoking:

The nature of risk is that a good decision won't always lead to a good outcome, and a bad decision won't always lead to a bad outcome.

While obviously we want to make decisions in intelligent ways in order to mitigate risk, associating outcome with decision  too strongly--and in particular, letting ourselves assume that because things turned out well, we made good choices--can prevent us from learning from our decisions. It goes the other way, too: it's too easy to look at an accident and simply see bad outcome = bad decisions ("I would never do that").

Marc suggested a way to circumvent this that I really liked. When debriefing a trip that went well, ask yourself, "If we'd made the same decisions but they had led to an accident, what would the accident report look like? How would we have handled it, and how would we reflect on our decisions afterwards?"

Conversely, when reading an accident report, you can ask yourself, "If I were reading about the same set of decisions but they'd ended well, what would I think?" (It might be something along the lines of: "That person was bold! I wish I were bold/smart/strong enough to do that, too.")

I found this perspective really refreshing and plan to implement it in the way I reflect on decision making and risk in my climbing. Might make our reactions to accidents a little more circumspect/humane, too.

Bad decisions, if done repeatedly, will always end badly. 

I look at it like Casino gambling. Sure you might win once and get away with it, but eventually if you continue gambling, the casino will clean you out.  

Same for climbing. If you knowingly or unknowingly follow bad practices, eventually a set of circumstances will occur that will be sub-optimal. It may be being benighted or it may be the death of your partner/yourself.  

That has no bearing on the "act of God" accident, where everything was done "right", and tragedy still ensued. Good friend was struck by lightning while climbing, and there was no rain forecast for the day.  She recovered, but is pretty timid now when it gets cloudy.

Buck Rio wrote:

Bad decisions, if done repeatedly, will always end badly. 

I look at it like Casino gambling. Sure you might win once and get away with it, but eventually if you continue gambling, the casino will clean you out.  

Same for climbing. If you knowingly or unknowingly follow bad practices, eventually a set of circumstances will occur that will be sub-optimal. It may be being benighted or it may be the death of your partner/yourself.  

That has no bearing on the "act of God" accident, where everything was done "right", and tragedy still ensued. Good friend was struck by lightning while climbing, and there was no rain forecast for the day.  She recovered, but is pretty timid now when it gets cloudy.

Sure, but where I think it's different from monetary gambling is that the probability/consequence calculus is more complex.

If you make a gamble where you have a 99% chance of winning $100 and a 1% chance of losing $1000 (great odds!), as long as you do it enough, probability tells you that you'll end up with a net profit of $89 per gamble (.99*100-.01*1000). As long as you have enough of a short-term safety net, that one time in 100 that you loose $1,000 won't matter because you more than make up for it with the 99 times you win $100. (This is the model the house operates on in the casino gambling comparison.)

If you make a decision on rock where you have a 99% chance of reaching the anchor uneventfully and a 1% chance of taking a life-altering fall, the 99 times you had a safe climb don't really matter once that one life-altering fall happens (at least, not from the perspective of consequences).

To put it in a different way, I think there are a lot of decisions that fall somewhere in between "act of God" and an obviously stupid decision. Probably most of the choices we make in the mountains lie somewhere in that gray area.

Etha Williams wrote:

Sure, but where I think it's different from monetary gambling is that the probability/consequence calculus is more complex.

If you make a gamble where you have a 99% chance of winning $100 and a 1% chance of losing $1000 (great odds!), as long as you do it enough, probability tells you that you'll end up with a net profit of $89 per gamble (.99*100-.01*1000). As long as you have enough of a short-term safety net, that one time in 100 that you loose $1,000 won't matter because you more than make up for it with the 99 times you win $100. (This is the model the house operates on in the casino gambling comparison.)

If you make a decision on rock where you have a 99% chance of reaching the anchor uneventfully and a 1% chance of taking a life-altering fall, the 99 times you had a safe climb don't really matter once that one life-altering fall happens (at least, not from the perspective of consequences).

To put it in a different way, I think there are a lot of decisions that fall somewhere in between "act of God" and an obviously stupid decision. Probably most of the choices we make in the mountains lie somewhere in that gray area.

My point was, if I wasn't clear, is that mistakes aren't a single roll of the dice, and that ONE bad decision rarely kills you (unless catastrophic). In my experience it is a cascade of bad decision taken in summary that kills you. 

So not checking your knot, in itself is rarely fatal. But a habit of not checking your knot, while not itself a failure point, could lead to death. And more to your point, even if you check your knot, you still may have the failure.

So lets say I start up the climb, don't do any double check, place a low piece and don't extend it (pulls out as soon as the rope come taut), climb and place another piece behind an expando flake, climb and place a tipped out #1, place a couple of small brass nuts in a flaring crack, then run it out and fall.  Individually, none of those "bad" decisions would be catastrophic, but taken together, I rip the entire pitch and die.

So, if you do the math (I'm too stupid) I think the probability of dire consequences goes way up the more bad decisions you make in a row.

It happens in Alpinism more frequently, when your hypothermic and your mind gets dulled.  Decide to bail (good decision, maybe) off of janky anchors (all you have got) with no knot in ends of rope (too many cracks) with no third hand,  to an unseen stance.   You see where this is going.

Ciao

Etha and Buck a fabulous book that I enjoy that goes into this in depth is "Who Lives Who Dies and Why". It is a great look into risk assessment and the cascade of bad decisions like Buck is talking.

"If we'd made the same decisions but they had led to an accident, what would the accident report look like? How would we have handled it, and how would we reflect on our decisions afterwards?"

I really like to look at my choices as I make them this way. I feel like it tones down my machismo drive of showing off or proving something to myself.

Jon Hillis wrote: Etha and Buck a fabulous book that I enjoy that goes into this in depth is "Who Lives Who Dies and Why". It is a great look into risk assessment and the cascade of bad decisions like Buck is talking.

"If we'd made the same decisions but they had led to an accident, what would the accident report look like? How would we have handled it, and how would we reflect on our decisions afterwards?"

I really like to look at my choices as I make them this way. I feel like it tones down my machismo drive of showing off or proving something to myself.

Just speaking for myself, but I have been around climbing for a while, and have seen mistakes that were made that didn't end badly but could have, and then made changes to alleviate that mode of failure. I feel pretty confidant I am not going to make a gumby/n00b/obvious error in judgement, since my SYSTEM is in place and I rarely if ever deviate from it.

AND....it has been a long time since I felt any machismo.  Declining T levels will do that. I have nothing left to prove to myself or anyone else.

Buck Rio wrote:

Bad decisions, if done repeatedly, will always end badly. 

This is vague enough to be close to meaningless. How many times must the bad decision be made or for how long before it ends badly? On a sufficiently long time scale, all decisions result in death, no matter how "bad" or "good" they happen to be.

Don't get me wrong, I appreciate that you're approaching this with systems-level thinking (which I think is probably the way to go). However, I think it's good to caution that while bad decisions increase the chances of leading to bad outcomes, the corollary doesn't necessarily hold that good decisions will lead to good outcomes. I think it's healthy to recall from time to time how little we might actually control in the climbing environment, particularly in alpine settings.

Buck Rio wrote:

Just speaking for myself, but I have been around climbing for a while, and have seen mistakes that were made that didn't end badly but could have, and then made changes to alleviate that mode of failure. I feel pretty confidant I am not going to make a gumby/n00b/obvious error in judgement, since my SYSTEM is in place and I rarely if ever deviate from it.

AND....it has been a long time since I felt any machismo.  Declining T levels will do that. I have nothing left to prove to myself or anyone else.

Dude my comment was about me... Was not implying anything about you lol. 

Jon Hillis wrote:

Dude my comment was about me... Was not implying anything about you lol. 

Dude: everything is about me ;-)

Buck Rio wrote:

My point was, if I wasn't clear, is that mistakes aren't a single roll of the dice, and that ONE bad decision rarely kills you (unless catastrophic). In my experience it is a cascade of bad decision taken in summary that kills you. 

Precisely true. There's a whole scientific literature, hundreds (if not thousands) of studies, on this. Most of it comes from studies of errors in aviation, engineering, medicine, military command, the nuclear industry, wildland firefighting, and the management literature. It's informally called the "Swiss cheese" model. Imagine a stack of random slices of Swiss cheese. If the holes line up, the bad thing happens. Most of the time they don't, which is how people get into the habit of making bad decisions. I've kind of wondered for a while why the climbing world hasn't in general taken advantage of this large body of science, even though some of the case studies come from mountaineering. How I make money for climbing involves applying this science to medical practice. Maybe in my retirement (843 days and counting!) I'll look into that.

I like the shift in perspective this trick brings. I think a key to creative thinking is that - force questions or framework that are counter intuitive, somewhat.

The hard part imo is how the get the right mindset when in the action and hard decisions have to be made, as Buck described. This kind of thinkin helps for sure.

Lee Green wrote:

 Maybe in my retirement (843 days and counting!) I'll look into that.

Congrats on nearing retirement. I still have 4,416 until I can hang it up, maybe sooner if the markets would cooperate.

I have had a few moments where a safety check has caught a misthreaded GriGri, a stopper knot came undone on a rappel (the rope hit the ground so it was not needed then), and a few other things that could have led to problems but didn't. I try to use those examples to improve my processes.

my company has all of us watch a talk by a former astronaut about "normalization of deviance" and though it doesn't always help at my job I have found it to be really helpful for my climbing process checks. it is a bit technical but I think the ultra high reliability requirements of rocket launches applies to how we should approach climbing safety.

https://www.youtube.com/watch?v=Ljzj9Msli5o 

Buck Rio wrote:

Congrats on nearing retirement. I still have 4,416 until I can hang it up, maybe sooner if the markets would cooperate.

After this week, I'd say you might be looking at another few years before retirement.

Etha Williams wrote:

Sure, but where I think it's different from monetary gambling is that the probability/consequence calculus is more complex.

If you make a gamble where you have a 99% chance of winning $100 and a 1% chance of losing $1000 (great odds!), as long as you do it enough, probability tells you that you'll end up with a net profit of $89 per gamble (.99*100-.01*1000). As long as you have enough of a short-term safety net, that one time in 100 that you loose $1,000 won't matter because you more than make up for it with the 99 times you win $100. (This is the model the house operates on in the casino gambling comparison.)

I've never seen a casino where the player wins 99% of the time. It's closer to the HOUSE winning 99% of the time. And when the player wins, it's a 50:1 payoff. So the house still comes out ahead. 

OK, so those numbers are just for discussion purposes. Roulette odds are simple to understand: half the numbers are RED (or black, odd, even etc). You win that you get 2:1 payoff. Sounds fair at first glance. But the zeros mean the house gets about 6% of all those bets. So the player still loses long term.