Critical thinking for leaders in the age of AI

Gemma and I have reached the sixth tool in the strategic toolkit we're building this series, and this one was Gemma's pick: critical thinking. Unlike most of the tools we've covered so far, this one comes with a genuine threat attached. We think it's a skill under real pressure from how we're all now using AI, so we looked at it through both the risks and the opportunities rather than treating it as a straightforward win.

The definition that didn't survive five minutes

We wanted to start with a proper definition, so Gemma read one out: critical thinking is the active process of analysing, questioning and evaluating information to make a logical and well considered judgment rather than accepting things at face value. Then she admitted it was AI generated, because that's what comes up first in most searches now.

So we didn't accept it at face value. We went looking for another one, landed on a university definition from Oxford Brookes, and it turned out to have far more depth. Critical thinking, it said, covers a series of processes we use to ensure we have good reasons to support our arguments, and it's best thought of as having a questioning outlook on life rather than accepting any reason at face value, always asking what do I think about this, and why do I think that. We genuinely hadn't planned it, but refusing to settle for the first answer turned out to be a live demonstration of exactly the skill we were talking about.

Why AI makes this both harder and more important

Here's the thing we kept circling back to. AI is plausible, but not necessarily right. I picked that phrase up at a workshop years ago and it's never left me. AI generated text can look articulate, polished, completely free of spelling mistakes, and still not actually answer the question you needed answered. You can't judge whether it has, though, unless you were clear about the question in the first place.

This connects straight back to one of our favourite models, what, so what, now what. There's a real magnetic pull with AI to jump straight to the now what, the finished output, without properly doing the what, defining the actual problem you're trying to solve. If you skip that step, you can't meaningfully critically evaluate whatever comes out the other end, because you've got no criteria to judge it against. It's the same discipline as that old story about the CEO who wouldn't look at a slide deck and only wanted you to talk him through your thinking with a pen and a whiteboard. Could you still do that, even with AI doing some of the heavy lifting? If not, that's worth noticing.

The AI sandwich

I did a brilliant workshop a couple of years back with a Bristol based consultant called Kerry Harrison, and she described a structure that's stuck with me ever since, the AI sandwich. The bottom layer is your own human, brainy thoughts, dumped down in whatever messy order they arrive in. I have ADHD, so mine come out as a proper jumble, but that's fine, the point is just to capture everything.

The middle layer is AI, used as a thinking partner rather than an answer machine. Ask it to challenge you, to ask you difficult questions, to help you structure, prioritise and cut repetition, not to decide what you think. The top layer is you again, bringing your own voice back to actually write the thing, now that the thinking is genuinely done. Using AI to help you figure out what you think is a big tick. Using it to outsource the thinking itself is not, and that distinction is the whole episode in one sentence.

A two hour lesson in what happens when you skip it

I've got a properly useful cautionary tale here. A leadership team I was working with was reviewing a role profile for a new CEO, drafted by a recruitment company. We all took it at face value, assumed it would be broadly right, and started making small wording tweaks. Two hours later, it became obvious there was repetition in there, some points didn't make sense, others flatly contradicted each other. It was a plausible, impressive looking document that never actually answered the real question, what does our next CEO need to look like, because nobody had done the thinking about what had gone wrong with the last one and what good would actually look like before the profile was written.

We'd been seduced by the quality of the language. If we'd known from the start that we were looking at an unreviewed first draft, one or two people could have sense checked it in ten minutes and decided whether it was even worth bringing to the wider group. Instead it cost an entire leadership team two hours.

Bot sitting and bot shitting

Gemma found two wonderful terms from an academic paper that capture this perfectly. Bot sitting is the largely unrecognised, unbudgeted labour of actually making AI usable, feeding it context, supervising its output, debugging its mistakes, cleaning up after it. It's the tail wagging the dog, we think we're saving time when really we've just created an administrative mess to work through.

Bot shitting is the follow on, shipping AI generated work that you haven't verified, don't fully understand, and couldn't confidently defend. It usually happens because you've spent so long bot sitting that you've run out of time and just send it anyway. Reading those definitions back, they're really the same point we'd been circling the whole episode, can I defend this, can I explain what's in it and why, or have I just sent out something that looks finished without being able to stand behind it.

That question matters just as much when you're the one reviewing someone else's AI assisted work as a leader. Rather than asking whether something was AI produced, which at this point you can probably assume, it's worth asking the person who brought it to you to talk you through it. Tell me your key perspectives. What's the what, so what and now what here. If they can't, that's the signal worth paying attention to.

Worth a listen if...

You want a genuinely honest look at how AI is changing critical thinking, both the risk it poses and the opportunity it offers.

You're after a practical structure, the AI sandwich, for using AI to sharpen your own thinking rather than replace it.

You've ever sent out a polished, AI assisted document and privately wondered whether you could actually defend everything in it.

Fi Craig is the founder of GoodThinking, a Bristol-based team and executive coaching practice working with senior leaders and their teams across the UK and internationally. The Strategic Leader podcast is co-presented with Gemma Bullivant.