My own argument had been our district didn’t necessarily need something new, like Khanmigo, because we already had a game-changing teaching product, Tutor.com, - being almost entirely unused by the district – right at our fingertips.
Well, I threw my own argument out the window. My curiosity got the better of me. How would Khanmigo compare to Tutor.com? A human tutor-for-hire versus AI-for-hire. For a quick explainer on what Khanmigo is... it is a "pre-prompt" engineered program, based on Chat-GPT. The AI prompt engineers set Khammigo to be Socratic, give positive/tutor-like feedback, and scaffold as best it thinks it needs to for the learner. Fortunately, funds were available to do a pilot of Khanmigo. We implemented it with the 8th grade math class. It took time to get everything in place, so it was not truly up and running until January 2024. Here’s the usage data by mid-February: Users: 108 8th Graders (out of 136 possible). Average minutes of use: 40 Average # of chats per user: 4 Average # of interactions with Khanmigo per chat: 11 I spoke with one of Khanmigo’s researchers around the same time. The researchers were learning about as quickly we were ourselves (granted they had much larger, and more in depth, data access). It was new for everyone. By March, data/observation evidence showed that teachers were having the most success, and engagement, if they are strong at modeling on how to use Khanmigo and why. If it was done with purpose and guidance students were more engaged. It if it was just made available as an option by the teacher usage was, I quote the research report, “varied.” “Varied” is kind wordage of saying it was inconsistent, and potentially weak. Some kids engaged, a whole lot didn’t. “Varied” was definitely true. According to the researcher, it got a 3 out of 5 average for student satisfaction out of 1600 students surveyed. I ran my own survey. Out of 105 responses to the question, “Would you recommend Khanmigo to a friend to help them with math...” 15 were NO 51 were MAYBE 39 were YES By no means a grand slam. But we did get a chunk of kids authentically engaged with it as a learning tool. I followed up with some qualitative interviews. There were students who said it was too wordy, or stilted, or inauthentic, or inaccurate. There were also students who found it useful to get support when the teacher was not available. As they reflected on their own learning practices, they saw the need for patience and just building the habit to make the effort to refer to it. Two months later, we got our 7th grade math students to give it a go. After a few weeks using Khanmigo we ran the same survey. 55 more responses to the same question ('recommend it to a friend...') 5 were NO 21 were MAYBE 29 were YES Speaking with the researcher after reviewing both my collected data, and Khanmigo’s data, we landed on this for a starting point: 1. Teachers had better consistently model usage of Khanmigo to get traction. 2. Conversation is key to be successful with Khanmigo. BE COVERSATIONAL when you interact. A bigger learning/teaching opportunity that can come from this: What are the qualities needed for a productive conversation (we're not talking about an easy conversation, like about the weather, or sports). There is a nuance, the need to be willing to be patient, push, and press to get more out of it. These are skills to learn. They are skills that must be taught in tandem with AI.
0 Comments
Leave a Reply. |
AuthorArchives
December 2021
Categories |