Three hikers. That’s the number that stuck with me this week. Three people walked up California’s Mount Shasta with a packing list and a route plan from Google’s Gemini, and three people came back down with help from the Siskiyou County Sheriff’s Office after descending in the dark and losing the trail. They didn’t have enough food. They didn’t have enough water. The AI had told them what to bring, and what it told them wasn’t enough.
Nobody was hurt, which is the best part of this story. But if you’ve ever asked a chatbot to plan something for you, and I’d guess most of you have, this one deserves a few minutes of your attention. Not because AI is dangerous, exactly, but because of what this incident shows about the gap between what these tools sound like and what they actually are.
Confidence is not the same as knowledge
The thing that makes chatbots so pleasant to use is also the thing that gets people in trouble. Ask Gemini or ChatGPT what to pack for a mountain, and you get a clean, organized, confident answer. Bullet points. Sensible categories. Reasonable-sounding quantities. It reads exactly like advice from someone who has been there.
But it isn’t. A language model doesn’t know the mountain. It has never checked the weather, never watched a trail wash out, never noticed that the snowmelt source everyone relies on dried up early this year. What it has is a very good statistical sense of what mountain-packing advice tends to look like in writing. Those are two completely different things, and the output looks identical either way.
This is the part I try to explain to friends who are new to AI tools. The model isn’t lying to you. It also isn’t telling you the truth. It’s producing text that fits the shape of a good answer. Most of the time that’s close enough, which is exactly why the times it isn’t catch people off guard.
Why trip planning is a bad first test case
If a chatbot gives you a mediocre email draft, you notice and you fix it. If it gives you a slightly wrong recipe, dinner is disappointing. The feedback is fast and the stakes are low.
Outdoor planning breaks that pattern in two ways. First, you can’t check the answer until you’re already committed. You find out the water estimate was low when you’re thirsty and hours from the trailhead. Second, the errors compound. Not enough water leads to slower movement, which leads to a later descent, which leads to hiking in the dark, which leads to a wrong turn. That’s roughly the sequence the sheriff’s office described. No single catastrophic mistake, just a chain of small ones that started with a packing list.
The sheriff’s office had a specific piece of advice in response, and I think it’s the most useful sentence to come out of this whole story: check with local authorities and the Forest Service instead. Those sources have something a model structurally cannot have, which is current, local, verified knowledge of the actual place.
A more honest way to use these tools
I’m not going to tell you to stop using AI for planning. I use it constantly. But there’s a distinction that keeps me out of trouble, and it’s worth adopting:
- Use AI to figure out what questions to ask. “What categories of gear do people usually forget on alpine day hikes?” is a great question for a chatbot. It’s drawing on general patterns, which is what it’s good at.
- Use humans and official sources for the actual numbers. Water quantities, current trail conditions, sunset times, permit rules, avalanche risk. Anything tied to a specific place on a specific day.
- Treat every specific fact as unverified until you check it. If the model gives you a number, find that number somewhere else before you rely on it.
- Notice when you’re outsourcing judgment instead of research. Research is “how much water do I need.” Judgment is “should I keep going or turn around.” Models are much worse at the second one, and they’ll answer anyway.
The uncomfortable middle ground
What I find interesting about the Mount Shasta rescue is that these hikers weren’t being reckless in any obvious way. They planned. They asked for guidance. They followed the advice they got. In a lot of contexts, that’s responsible behavior. The failure wasn’t a lack of effort, it was misplaced trust in a source that presents itself as authoritative without being accountable for being right.
That’s the real lesson for the rest of us, and it applies well beyond hiking. Medical questions, legal questions, financial questions, anything where the answer has consequences. These tools are genuinely useful helpers and genuinely poor authorities, and the interface gives you no signal about which one you’re getting.
Three hikers found out on a mountain. You can find out from your couch instead.
🕒 Published: