Learn how to check that an AI tool will deliver on what it promises, with Craig McKee.
AI is a hot topic in education at the moment. There’s not an hour that passes where I don’t see something on social media about AI in education – the latest must-use prompts, the latest tools, the latest hack. But do these deliver on what they promise?
The potential
I’ll start off by saying that I love AI. I really do! I do believe it will change how we work and hopefully make a positive impact on the workload of school leaders which will hopefully go some way towards addressing the ongoing recruitment and retention crisis.
It has a huge potential. Like 184,999,999 others, I was one of the 185,000,000 users who signed up to ChatGPT early in 2023 – and my first impression was: “This is amazing!” I type in a short question or instruction and sit back as my work is being done for me. What isn’t to love? This could handle my emails, school improvement planning, SEF, governor reports, parent letters, complaint responses, pupil support plans, monitoring schedules – the list really is endless and I initially felt like the proverbial kid in the candy store, but…
When I actually read the responses, I realised they weren’t very good.
Was it a prompt issue? I must have got my instructions wrong or I wasn’t clear enough. Let me try again. OK, a bit better but still not right, something was still missing. This is when I realised that these tools simply didn’t have the appropriate knowledge to give me a high-quality answer, regardless of how long I spent crafting and refining my prompt.
Three types of AI queries
As I thought about this more, I realised that AI queries generally fall into three categories: general, informed and specific.
- General – I need general help with something like proofreading a document or rewording something. You could use any of the AI tools available for something like this because it doesn’t need specific knowledge about you, your role, your school or even education.
- Informed – I need help with school improvement planning targets or appraisal objectives. Now we need to start looking at what information the AI tool that you are using has access to. For the examples above, you would want it to have details of the OFSTED framework, EEF research projects, the teaching standards etc. so that it can give you an informed response rather than a general reply that perhaps wouldn’t be very good.
- Specific – I need to write a response to a complaint letter or create a subject medium term plan. For these examples, I would need the AI tool to have specific knowledge of my complaints policy or the curriculum progression in my school; without this information the reply is probably not only going to be not very good, it could potentially cause me problems. This could be handling a complaint in a way that is not in line with your policy or result in teachers teaching or planning lessons/units that don’t follow your school’s curriculum progression.
Critical evaluation
Custom GPTs go a long way to help you achieve this, since you are able to upload a number of files so that the AI tool is aware of the content and can use this to inform its response. This means that you can now get an informed response or even a specific response and have more confidence in the accuracy of the response you have received.
Moving forward, this means that critically evaluating AI tools is going to be really important. The first question to ask yourself is, “Do I need a general, informed or specific reply?”
If it’s a general reply, you’re pretty safe with most tools. If it is informed or specific, you need to ask yourself, “What information does this tool have?”
If you keep this simple approach in mind, it will help identify tools that are going to be a benefit to you in your role and also identify those that could potentially cause big problems.
As I said earlier on, I am a big fan of AI and want to reiterate this. I think it will have a positive impact on the workload of school leaders and am excited to see how this develops in the future. I do think we will see lots of tools that promise a lot but simply aren’t able to deliver, come and go. And that is why it is important to look past the slick website and apps to be able to really evaluate the tool for what it is, to see if it can deliver what it promises.
