Veritasium: What Everyone Gets Wrong About AI and Learning – Derek Muller Explains https://www.youtube.com/watch?v=0xS68sl2D70
Perimeter Institute for Theoretical Physics Apr 9, 2025
AI is advancing faster than anyone predicted—and it’s already reshaping industries around the world. But what does that mean for education?
AI might change how we teach and learn, drawing on insights from past tech shifts and core principles of cognitive science. While AI presents exciting opportunities, it also introduces real risks—especially when it comes to how our brains build knowledge and expertise.
Yotube-transcribe (under edit) I wanted to sort of shoehorn AI into this
talk a little bit, because what I'm really passionate about is education.
So my question is really, how is AI going to play a role in that? But first we have to understand the bigger picture of how education actually works or doesn't work in some cases..... (example)
...AI tutors have only gotten better, I'm sure, in that time. I think if you had told me five years ago that such a technology would exist, I might not have believed you. AI, as we all know, has gone incredibly quickly. So the question is, is this the future?
Is this how everyone's going to be learning in the future?
Well, I wanted to reflect on two things.
The first is that you can always complain about education. You can do it now. People were doing it 10 years, 100 years ago. You can always complain about the state of education, and I can kind of see why....
You could always complain about education. You can always say, you know, it's no good.
So we can ask the question, why aren't people learning? You might point to certain things. Maybe the education system is stale. Maybe it was made to create factory workers. Maybe it's because we didn't have AI tutors yet. I'm going to come back to this question, but I think it is an important one to think about what is really going on.
What are the outcomes we're getting from the school system and why.
The other thing I wanted to point to, the second thing is this word revolutionize when it comes to education. Here we have a book by Salman Khan. Brave New Words. How AI Will Revolutionize Education. But I mean, he's not the only one saying this.
How AI Is Revolutionizing Education: Benefits and Challenges https://www.lisedunetwork.com/how-ai-is-revolutionizing-education-benefits-and-challenges/
By Md. Ashikuzzaman
Four ways AI could Revolutionize Education Technology or How AI is Revolutionizing the World of education. So clearly the expectation of a revolution is here. But the truth is people expecting a revolution in education have been around for at least 100 years. We have Thomas Edison. This is in 1922. -- The motion picture is destined to revolutionize our educational system. And in a few years it will supplant largely, if not entirely, the use of textbooks. You remember how we got rid of textbooks, right? 100 years ago he said something like,
education from a textbook is 2% efficient, whereas education from a motion picture is 98% efficient.
I have no idea where he came up with those numbers. I don't think anyone does. But you can imagine that in his time the motion
picture would have seemed like magic. In the same way that first clip I showed you of an AI agent tutoring
a kid and doing it somewhat competently, that also seems like magic.
So you can see why this argument for revolution is kind of convincing. In the 1930s, you had people saying
radio would revolutionize education. Why? Because you could get rid of all the teachers. You could just have like babysitters
in the classroom and then you'd have the expert radio in and they could radio into a thousand classrooms.
So immediately you have this economy of scale and people see the same thing. Today Elon Musk says AI education will be like
having Einstein as a teacher for every child. The same idea. We're going to scale
up the best and the brightest. You know, we're going to have this most intelligent entity teaching everyone
instead of, I don't know what we have today. Okay. So in the 1950s, it was TV.
Academics actually conducted studies where they would have a lecture going on in one room like this one.
Then they'd have it videoed and closed circuit TV in the lecture room next door. And they would test the students
to see who learned more or less the answer. No significant difference.
No big surprise because they're getting basically the same experience. Assuming that TV system works reasonably well,
then in the 1980s, I think people at MIT anyway thought that they had it. They thought they had the revolution
in education because the thing we had been missing was interactivity. And now you could actually
interact with the computer. You could go back and forth with it, you could program. So there was this idea that if we
taught kids how to program, say how to program a turtle, then they would
get better reasoning skills overall. What happened? The kids got proficient at programming
the turtle, but those skills did not transfer to other sorts of reasoning. We didn't get better thinkers.
And that was the concept. That was why computers and interactive computers and doing this whole turtle programming was meant to
revolutionize education once more in the 1990s, you get this. The use of video discs in classroom
instruction is increasing every year and promises to revolutionize. What will happen in the classroom of tomorrow?
I don't know. Many of you here may not know what a videodisc is. It was a huge CD like thing, like a dvd.
I used to use these when I was in high school. So, you know, they were there. Did they revolutionize education?
No. There's one more. I'll talk about MOOCs, massive open online courses.
About 13 years ago, this was all the rage. The big idea that can revolutionize
higher education, it's the MOOC. Will MOOCs revolutionize higher education? At least they're asking a question.
Revolution hits. The universities know it's hit the phenomenon of MOOCs revolutionizing education.
In the digital age, the online education revolution drifts off course.
Get sad. The MOOC revolution may not be as disruptive as some had imagined. Why MOOCs won't revolutionize higher ed.
What if MOOCs revolutionize education. After all, a story in 10 headlines.
Anyway, I hope you take my point. People are all too excited, all too ready to
put the word revolutionize next to education. This is old school, new school, it's the same school.
What is going on? We shouldn't be using this word revolutionize. You keep using that word.
I do not think it means what you think it means. You know, it's a good talk if there's a Princess Bride reference.
Okay, so why didn't these revolutions materialize?
Or why didn't they eventuate? You could say educational institutions,
they have a lot of inertia. They're not willing to change. We're all just set in our ways.
You could say it was just technological overhype, or you could say we didn't
have the capabilities of AI yet. This time. This time is different.
Or maybe it's something else. So I'm going to explore this concept a little bit.
I think one hint comes from this clip, a question that I asked
to some people in Los Angeles. So let's take a look.
You go into a toy store, and there's a toy bat and a toy ball.
Together, they cost $1.10. The bat costs a dollar more than the ball.
How much does the ball cost? 10 cents. 10 cents. We're all wrong, aren't we?
What's the answer? Think about what is the answer? What is the answer?
Five cents. Yeah, that's very good. Now, I don't think these people are silly. I don't think they're stupid.
I think they could work it out. If the ball was 10 cents and the bat was a dollar more than
the ball, it would be $1.10. And so together, they would be $1.20. So the correct answer, obviously,
you guys know it, $0.05. So why do they all say $0.10? It's because that number came to their
head as they listened to the question. They don't know where that number came from, but it was in their head and it sounded right.
So they blurted it out all at the same time. Really quite amazing, right? Something I love about that clip.
So why do they do that? Well, the insights are in this book, which is one of my favorite books of all time,
Thinking Fast and Slow by Daniel Kahneman. So in the book, he's talking about our two systems of thought, that we've
got two things, two kinds of processes going on in our brains at one time. The fast processes system one
and slow processes system two. And at the start of one of his chapters, he imagines what if there
was a Hollywood movie where these systems are turned into characters?
So I decided to take that inspiration and run with it. So I created these
two characters inside my head. System one is on the left and System two is on the right.
Now, System two is kind of the person who you think you are, that's the voice in your head.
It's a slow, effortful system. Thinking takes some effort.
If you ask him to multiply like 13 times 17, he's going to say, I don't want to do it, but you could force him to do it
and he can go through a series of steps. So that's the thing about System two is it's slow, it's methodical, it can
catch mistakes, it can work through processes, it can think about thinking. Now System one, on the other
hand, is rapid fire. This guy is doing things fast in the background. You have no idea that he's doing it.
He's collecting all the information from your senses, he's pulling out the relevant information and he is getting rid
of everything else because you're just getting bombarded with stimuli. So he's pulling out those key pieces
of information that you're going to work with. And he does all of this without you, the System two thinking, conscious thing,
really being aware that he's doing it. And you noticed also that he was like in a library setting.
So he's sort of associated with all of your long term memory. So he's got all of that and that's what allows
him to work so quickly and so effectively. And when Those people said 10 cents to me,
he served up the answer 10 cents. And this guy system two was just like, yeah, 10 cents.
I'm not going to think about it, I'm not going to check that answer. He could check the answer, but he doesn't want to because he's lazy. And fair enough, the goal for us should not
be always to use System 2, but it should be for us to know when we need System two versus
when System one will be fine. So the idea is, hopefully you want to optimize, allow System one to handle everything
that they can handle and only hand off to System two things that you need to. Now, in my travels, I came across these
two characters and something that I thought about later was that they almost embody
System one and System two together.
So the question that I asked them was like, how long does it take for the Earth to go around the sun?
Okay, so how long does it take for the Earth to go around the sun? What do you Reckon Carlos, sort
of 24 hours, obviously a day. Yes. Okay.
I asked the question to System two. He was like, I don't want to. Like, what do you think? You know, System one over
here, what do you think? He was like, obviously, a day. It sounds good. Yeah, a day.
Sun comes up, sun goes down. Yeah. And then what did he do? He just spat it out. He didn't stop to think about it.
He was like, obviously, yeah, it's a day. What was amazing about this conversation
was this conversation moved on and I had a whole set of questions that I asked them. I think at this point, I was asking them,
did dinosaurs live at the same time as humans or something silly like that? Anyway, that's what I was doing.
And then I noticed a pause and I was, like, looking at his face
and you could see that he was thinking. You could see that System
two had snapped into action. So I just didn't interrupt. Okay, watch what happens.
Hey. The Earth doesn't take one day to get around the sun textually.
Yee.
You see the shock on his face. The whole time he was just running System one.
And then at the very end, it was like, whoa, hang on, wait a minute.
All right. The thing about System two is that it is very limited.
So this paper came out in 55. It's a classic. The magical number seven, plus or
minus two, the limits on our capacity for processing information. So the idea here was really that you
can only handle seven new things in your working memory that System 2 can deal
with at one time, plus or minus 2. It turns out over the years, that estimate
of 7 has been revised down to about 4. And there's a test you can do to kind of assess
this and figure out, okay, what is my span? How many pieces of information can I just work with at one time?
So the way this test goes is I show you some numbers that you can look at briefly, and then I take them away.
And what you try to do is, is say each number on the beat.
There should be like a beat going, I don't know how we're going to do that. But you say each number, but you add one to it.
So increment each digit by one. After the numbers have disappeared,
I think I'll do it for you. So you guys don't feel weird about this, but I just wanted to demonstrate how we can show that this span is quite limited.
Ok, so hopefully I get this right, because I know what the numbers are. OK, so here we go. There's some numbers.
6, 5, 0, 2, right? Okay. That wasn't too hard.
You can make the task harder by instead of adding one, you add three. And instead of having four numbers,
you can go to five or to six to really push yourself and see what happens. What's amazing about this task, what's
interesting about it, is that while you're doing it, while you're really activating system two, there's physiological
responses that take place. Your heart beats a bit faster, your skin gets a little sweatier,
and interestingly, your pupils dilate. So when you're really thinking
hard, when you're using that system 2 thinking, your pupils are actually
expanding just a little bit. So I got a super macro lens and some friends
and I showed them numbers and then got them to do the add one task. And so we can try to see if you can
spot the growing in the pupil of this eye while he's doing the task. Here we go.
4, 3, 9, 7, 2. Okay, there it goes a little bit.
5, 4, 0, 8, 3. I feel like in the middle, when he's like
in the middle of the numbers, and he's still got to remember the remaining numbers, and it's like, you know, like, it's just.
And then at the end, it sort of starts collapsing back. He's like, okay, now I can relax a little bit again. So it's interesting to see
just how effortful this is. This measure is known as cognitive load, or
another way of thinking of how much mental effort you are investing in something.
And we can break down how the cognitive load is being expended into three categories.
So there's intrinsic cognitive load, which is just the amount of cognitive load you need for a particular task.
So in that case, if I'm giving you six numbers, it's going to require a large intrinsic cognitive load.
Or sometimes when we're doing something in physics, even something simple like F equals ma, which to physicists
and stuff kind of is not that hard. But for a new student, each of those things
is new and sophisticated. It's a complex concept. And so, in fact, even that just teaching
that has a large intrinsic cognitive load. There's also things that are totally extraneous.
These could be things like, I don't know, maybe there's someone chewing next to you, or your seat's uncomfortable.
There's just distractions and things around you that are taking away from your ability to focus.
So extraneous cognitive load is not great. And then finally, we have germane cognitive load, which is, can you
actually use your attention. Use System 2 to do some maybe thinking about Thinking, like observing kind of
what you're going through as you're thinking through a problem, or noticing things that might be helpful patterns for
you to use later. So germane load is great if you can get it.
Now, when it comes to this is another really classic study, because one question you
might have is, if we have such a limited system too, how do we accomplish anything?
And if you go back several decades, people were wondering, what makes a chess master a chess master?
Do they have really high iq? Do they have great spatial reasoning? Do they have a really large working memory
span, which is the kind of thing that we've been looking at what makes them unique.
And these researchers got together and they did a study where they showed some different level chess players,
from beginner, intermediate and grandmaster. I don't know if they were grandmaster, maybe just a master, but someone
who's very experienced in chess. They showed these different people a chessboard like this that was basically
stopped in the middle of a game. They showed them this chessboard for five seconds.
They had it in front of them. Chessboard's there. And then they covered up the chessboard and they had another blank
chessboard beside it. And they said to these chess players, okay, you've seen it for five seconds.
Now put the pieces on the board exactly as they were in that chessboard that you just observed.
When you ask novices to do this, they've only seen it for five seconds. When you ask the novice to do this,
they will get about four pieces correct after looking at the board just once.
In further iterations of the study, they were allowed to look at it for another five seconds and then put some more down.
And so they were sort of counting up how many it took. But just on the first glance, a beginner could do four pieces.
The grandmaster could do 16 pieces after one glance, after just five
seconds of looking at it. Why were the grandmasters so good
at remembering where all those pieces were on the board? It's a lot.
It's because of a phenomenon called chunking, which is where you can see things
that look like disparate bits of information, separate pieces of information, but you can actually see them as one thing.
So in the case of the example I gave you earlier, if you reverse the order of these numbers, you get 1945,
which is no longer four numbers. It's one thing the year that the Second World War ended, right?
And so that makes it much easier to remember. And this is true for basically everything we come into contact with.
We're not normally dealing with it in its Most basic chunk. So words, you know, you don't have to parse
out every letter, even phrases, you know, they have meaning and, and you can think of them as one thing, or equations like this one.
For a physicist, those become one thing. Even the Schrodinger equation, yes.
For a lot of people in the room, I imagine that's just a single entity in working memory. And you don't have to, you know, you could write that down.
You could close your eyes and write it down again. So the point is that the more we get
experience with things, the more we practice, the more we interact, the more we use System 2 to work through
problems, the more we develop this long term memory. And that long term memory allows us to chunk
the things in our world, and that allows us to deal with much more complex situations.
So that's exactly what's happening with a chess master.
I have heard a lot of people say this. That class really taught me how to think,
or that teacher taught me how to think. And I think that's more of a question
than it might seem at first. I mean, at first it might seem like, yeah, that feels reasonable.
I think a lot of us have this experience where we feel like, oh, yeah, after that, I really learned how to think, I learned how to reason.
But if you go back to this chessboard study, the next thing they did and as part of the study was they rearranged the pieces.
Same number of pieces as before, but now they positioned them as though they would never occur in a regular chess match.
They just put the pieces all over the place at random. You would never, ever see this
in a regular chess match. And they tested again, the novices and the masters.
And now the masters did no better than the novices. The thing that made them so good at remembering
where the pieces should go was that they had seen a lot of chessboards, they'd seen a lot of configurations just like that.
And if the pattern wasn't there, then they were no better off than the novices.
The exact same situation. My argument here is that
there is no general thinking skill. There is no general problem solving skill.
What there is are these complex webs of long term memory.
This is what we build up over our lifetimes. So a physicist is not necessarily going
to be a good chess player, and a chess player is not going to be a good physicist. This applies to all domains, I would say.
I mean, you can make arguments for specifics where you could say, yes, such and such skill will transfer.
But I think in general, the idea is an expert in One field is not an expert in another. And it's just because of this.
It's because that long term memory that your System one is working with is really
specialized to reflect the experiences you've had and to have recognized all the
patterns in those experiences to allow you to chunk new situations when they
come to you. It's so powerful when you're System one is this really complex web.
You don't even feel like you're thinking in the same way that those people said 10 cents to me.
If you ask Magnus Carlsen, what is he doing when he plays chess, most of the time I know what to do.
I don't have to figure it out, right?
He's saying I don't need System two. System one is so well developed
that it can solve any problem at a glance. For him, chess is a game of recognition.
He recognizes the board in the same way that we recognize faces or physicists
recognize a physics problem. Right? That's the way we get to be really good at things.
So really what we want to be doing in education is using System 2's resources very
carefully and repeatedly, such that we store
information in long term memory to allow our System one to do things that effectively feel
automatic. So what are the implications
of this for education? The first one is that we should
eliminate extraneous cognitive load, which is fairly obvious. So you should have a comfortable seat, you should be able to see the board.
Everything should be legible. The sound from my microphone should be pristine and pure.
I shouldn't have an accent. If I have an accent, it makes it harder for you. That's extraneous. Just trying to think.
What was that word? Stuff like that. Having subtitles sometimes helps. My wife loves to watch all
TV shows with subtitles. So now that's what I do too.
So how can we limit intrinsic cognitive load? This is point number two. We don't want to overfill what you can handle.
And my guess for you is that the reason why a lot of physics lectures fail sometimes,
if they do, is because the professor doesn't limit intrinsic cognitive load enough.
They go through too much not novel material in the one lesson. And it's just too much for anyone to handle
in terms of their working memory. Once you're overloaded, there's basically nothing you can do. Your system too doesn't
know how to deal with that. So one thing I would say when you're teaching is you have to try
to obviously start where your students are. And you have to keep the work kind
of bite sized because as soon as you start introducing 4, 5, 6 novel concepts in one lesson, you're going to lose people.
Another way to think about limiting intrinsic cognitive load when it comes to music is get
people to play songs that they already know. This is why we do that, I think.
Right. Learning to read music takes time and effort. So the first things you're going to do with students is get them
to play songs they already know. Then they know what the rhythm should be. And so even though, yes, in theory, they
should know how to read these notes and, and they should know what a quarter note is and understand the beats. I can tell you, as someone who played
French horn for several years, I didn't really know how to read music for at least a couple years after I started
playing the music told me what fingers to push down because I could have understood the notes.
But in terms of rhythms and things like that, really being able to sight read music, I think it took a couple years. So I think this is one of those things where
we limit intrinsic cognitive load by getting students to play songs they already know. Another way to limit intrinsic
cognitive load is to slow things down. So anyone who's practicing music, for example, like this harpist who's a friend of mine,
she describes her practice like this. You can practice everything exactly as it
is and exactly as it's written, but at just such a speed that you have to think
about and know exactly where you are and what your fingers are doing and what it feels like.
So she's really talking about using System two. There's very slowly, deliberately,
effortfully thinking through every little thing that she's doing in those moments.
And then when she does that enough times, enough repetitions of this, you get something that looks like this.
Yeah. So good, right? Every time I see superhuman performance,
the thing that I'm thinking about is how they have used System two very slowly and
effortfully to build up that structure in long term memory that allows them to do
things that look so amazing and effortless.
It's also at this point that I bring up this idea that discovery learning can be dangerous.
You know, when I was going through school, I'd say the dominant paradigm in education was something called constructivism.
And the basic tenet of constructivism is that students are active constructors of their own knowledge.
Something I do not disagree with. It's basically what I'm saying up here, that System two is about being
active and effortful in how we engage with material, how we practice. The problem is, I think some people
didn't know how to implement Teaching for this constructivist paradigm.
Their thinking was, if the students need to be active, then telling won't work.
But that's not true. I'm telling you stuff right now, but I
bet you're still being active and effortful in your brains, which is the place where you need to be active and effortful.
So that was the problem with constructivism, and it led to some classrooms where people really pulled the scaffolding away
from students, I would argue too early. So it was, we want to teach problem solving.
Here's a problem, solve it. You don't know how to do that. Figure it out, construct.
This is just my personal pet peeve. I have some hangups about constructivism,
but this is where I say discovery learning can be dangerous. I can see flip sides of it.
For example, if you go to a new city and you don't use a gps, it's really hard
to figure out where you have to go. But if you do it and you use your system
2 to figure out and follow every street and look at all the signs, signs around, read these things, and yeah,
you're looking really carefully at everything. At the end of that, you'll probably be able to retrace your steps.
If you use the gps, it'll be much easier for you. You're offloading processing from system two.
You're offloading it to the gps. Can you find your way back? No, you're going to need the GPS
to find your way back. I think there has to be some balance in education between the kind of GPS
guidance and the internal guidance. Ideally, I would love to see, like a gradual phasing out of the support
to allow, you know, the person to figure out the directions on their own. This has been, I think, well backed
up in research of things like the worked example effect. And this is even a further idea of sort
of fading out the assistance as you go. But the idea of like, first giving someone, hey, here's a problem
and here's how you solve it. Here's a problem, but it's not quite at the end. Here's a problem partially done, and here's one
for you to do from start to finish, right? But it is this idea of like, I'm going to reduce your intrinsic cognitive load.
I'm not just going to give you this problem at the end and say, figure it out. I'm going to give you this scaffolding
because I know that what your working memory is, is very limited.
That system too has very limited resources, and I'm not going to tax them with you trying to think about all sorts of things at the same
time, I'm going to give you this assistance. So to me, this is the scaffolding that helps System one move through and learn new problems.
I think, again, this is kind of a problem we have in complex domains like physics, where
to the physics professor everything's perfectly clear because their System one is so fully developed.
But to a student it's not. So this is the expert novice divide.
The professor can't see with the student eyes what that problem looks like. Right.
Only the student can see it that way. Okay, so what else do we need to do
in education? We need to repeat that effortful practice until we achieve mastery.
Why is mastery so important? Because when you show a skill with mastery,
that means it's Now a System 1 domain. For example, for all of us here in the room, our basic times tables,
I imagine, are mastered. They're in System one. You don't have to think about it. What's so useful about that?
When you come to do other problems which require that information, you're not going
to overload system 2 when you do it, because all that stuff is automatic. So that's the key.
That's how we're going to get better and better performance, is by building up more and more capabilities which are automatic.
If you never get to the level of mastery, then moving on is going to always cause you problems because you're going to be
having to use extra areas in your working memory to deal with thinking about that first before you can think about the other
thing next. We have increased your main load.
So how do you get people to engage in really active thinking, extra level active thinking?
One thing that was tried, for example, with this problem. Okay, so the bat and ball problem is actually from a test.
I think it's called the cognitive reflection test. And I think there's actually three questions on this test. The bat and ball problem is only one.
But this test has been given out to thousands and thousands of people around the world, including
incoming students at very elite colleges. And when you give this test, you typically find that about 90%
of people get at least one wrong. At least 90% get at least one of the three questions wrong.
And they're all kind of like the bat and ball problem in a certain way. So then what some researchers tried was
they thought instead of printing out this test in like really clear font on
nice white paper, we are going to print it out in like a terrible, very hard to
read font, maybe photocopy it a few extra times. So it gets really difficult to read.
Crumple up the paper or something, and then we'll hand it out. And what happens?
The error rate drops to 35%. Making the test harder to read made students
more likely to answer it correctly. Why? Because they didn't have that
easy, like, okay, I can read it. And an answer immediately comes to mind. They didn't have that experience.
Instead, what they had was, oh, this is challenging to read. Which kicked their system two into action.
And so it was activated and was able to find the correct answer. So I'm not suggesting this is a good
strategy, just making things more confusing, but I will say it can be effective.
And I honestly, I feel like I'm seeing advertisers apply this sort of principle sometimes.
In the old days of advertising, like the 50s, 60s, I feel like it was all about having a jingle showing what your product
did, why it was better than your competitors, and then having this great jingle that would just get stuck in your
head and you'd all remember the name of the brand and why it's better. And then these days, I think we've got
so bombarded by advertising that System one has got great at tuning it out.
And so now a new strategy I see some advertisers employ is let's confuse people.
This was a billboard that I saw in Sydney near the beach, and I was definitely intrigued.
The power of un unexplained television. Un cost the earth.
Unpay more, unspend more. It didn't really tell you anything.
It's like, how is this good advertising? A few days later, at a bus stop, I saw this unexplained.
With un, there is no stress, just unstress no hassle, just un hassle. With un, you can undo what you did.
You can un drive through the car wash with a window down, or un break dance in front of your teenage son and his
mates, or unflood the downstairs laundry that now doubles as the pool room. Un makes life relaxing and unreal.
Un your life be happy and live for now. Don't worry. Unworry.
What is this an ad for? Insurance? Yes, this is an insurance ad.
But how lame are insurance ads? And this is pretty great. So they really got me.
So what role is AI going to play? If you believe in this picture of education that I've painted?
I think the positive role that I see for AI is that it can provide timely
feedback, and that's essential when you are learning any skill. You're going to go through the repeated rounds
of practice and you're going to need immediate feedback in order to learn. When you're playing the harp, if you miss a note, you know, immediately, that's great.
Immediate feedback. If you're practicing tennis and your shot's in or out, you probably know if it hits
the net or goes over, you know, right away. So these are really helpful in terms of training your brain.
There are some different industries and some different professions where people seem
like experts, but they're actually not. Stock pickers is one example, because
you may get some feedback that maybe this was a good decision or a bad decision, but honestly, the stock
market is, at least in the short term, incredibly random. And so that feedback is never very informative
for what you should do the next time. This has also been shown with political pundits and some economists.
Anyone who's looking to predict these future trends, which are much less reliable
in terms of the validity of the environment. So one thing I see AI doing is providing
this timely feedback, which will really help, as we saw in the first case with Sal Khan and his son.
But the thing that I'm really worried about is how AI has this opportunity to reduce effortful practice.
I have four kids who are 8, 6, 4 and 0. And I worry about them that, you
know, if they're going to be. Will they write an essay, will they write 100 essays?
If there is a generative AI that can write for them, what forces them
to practice crafting those sentences? And if they don't craft those sentences,
what happens to their brains? The argument here is that you get good
at your command of the English language. You get good at being able to speak in front of people, at being able to express
your thoughts in writing by doing it again and again and again and again. And you should suck at the beginning,
and you shouldn't let that stop you. And you should keep going and going and making slight tweaks and improving and getting feedback and.
And getting going. If they never do that, I really worry what gets
into system one, you know, what is that? Do they have an amazing network of connected
knowledge that they can draw on? Do they have things that are automated? I fear that they won't.
How do we force people to have to do that painful, effortful work when there's a magic machine that will do it for you?
That's a big concern. What about drawing? You know, if you can just ask it to make a picture of whatever you like.
The bat in the ball was AI, by the way. I can't draw, so.
But again, like, what will happen to people's artistic abilities. So this is, I think my biggest concern
is if it prevents us from going through this painful, effortful process which
is the core process of learning. Using your limited system two resources
to engage with things and practice again and again and again, even when it's hard, even when it doesn't feel good, even when you're not great at it.
That is my big concern. I want to come back to these two big questions. Why aren't people learning?
And why haven't the education revolutions materialized?
To the first question, I want to be a little bit more generous to the people we saw at the beginning of this talk.
Our brains are designed to help us be effective in this world, which means
finding food and shelter, finding a mate, integrating socially so that we're not
ostracized, being able just to hang out and have fun. All of those things
are what we should be doing. And maybe it shouldn't be such a surprise that people don't know what elements
it takes to make water or even that CO2 could pose an existential threat. I know everyone in this room will
agree that's important. We should all know that and we should figure out how to work with that.
I guess what I'm saying is I think it's understandable that a lot of people don't focus on that, don't know that, don't think about it.
It's not part of the world that they exist in just because they're so busy with social
media and Instagram or whatever, because that's about connecting with other people. So I want to be a little bit generous there.
And when it comes to this question of why haven't education revolutions materialized
with film and TV and radio and computers and MOOCs and now AI, part of me wants to
say I think we might have already found the best thing being in a room with other
people, other learners, a teacher, and some
time to talk. Education is a social activity.
You know, people care about other people. I think that the tech hype comes
from a place of believing that the problem of education is not being able to get the information to the student.
That's not the problem. It's not the problem now, and it wasn't the problem 100 years ago.
When you have books, I mean, the information is all there. Assuming people have access, the students
have access to those books. And yet they're probably not going to learn very much unless they have a great teacher, unless they have a group
of like minded peers to go through that with them, unless they have a reason to
do it. An analogy for you. You know, the world is full of heavy objects,
and yet most people are not ripped.
Do you see where I'm going with this? The world is full of field.
Not many people running on them. There's plenty of ways to get exercise,
but obesity is a huge problem.
I hope the analogy. I'll get there. I think about teachers because I was one.
I kind of am one. I think about teachers a little bit like personal trainers.
Like, the gyms are there, but unless there's someone who you're going and you're meeting there and you're held accountable
to and someone to say, another one. Another. Give me another rep. Keep going. It's burning. Yeah, keep going.
You know, someone to tell you the homework and someone to hold you accountable and someone to really energize
you and maybe a group of other people who are doing it at the same time. And we're all, like, going to this together. That's when you see results.
This doesn't just happen in a vacuum. So I guess that's my big thought, that teachers
are some of the greatest people in the world, doing an incredible job of connecting with
students and creating communities of learners. And that's what it's about. It's about that social experience.
It's about getting excited and holding people accountable and forcing them to put their reps in. So for me, that's why none
of these technologies are ever going to revolutionize education. Thank you.
All right. Thank you, Derek. That was a fantastic talk. All right, so now we have some time for questions.
So for those in the theater who would like to ask a question for the speaker, we have a microphone set up there on
the stairs, so please make your way there if you have a question. We particularly encourage young members
of the audience who have a question to feel free to come up and ask. And just a reminder for anyone and all
folks either online or who are in the theater asking a question, please keep
your questions short and sweet to give us enough time to answer everyone's questions. So.
All right, we'll start with Giu. Go ahead. Hi, Derek. Thank you for coming today.
You kind of touched on it a little bit when you talked about your kids, but the question that I have is, what do you think about using AI to
answer your question when you're working on a problem as opposed to looking up for the
answers yourself in a maybe more complicated way? Like in the books or whatever?
Like in a. Which. Sorry. In the books. In the books. I think that as long as the AI is Sound.
And a lot of it is reasonably sound at this point. Right. Then I would say I don't think there's a difference really.
I think you may as well just talk to the AI, assuming some. Assuming the AI is reasonably knowledgeable and is not going
to hallucinate things like that. But in terms of just the. Is it better if it takes me longer?
Like, no, I think that's like it's strictly a question of efficiency. And at this point I wouldn't worry about
being, you know, you can be more efficient. I think it's fine. Yeah. Okay, thank you.
All right, we have a question from online. All right, so do you have any suggestions
for how to get these insights about education enacted into policy?
Is that something that should come from teachers or from the government or from public? What are your thoughts?
My thoughts are that policy is very challenging. In Australia recently, they've implemented
what they call a direct instruction policy, which is kind of what I'm talking about, which is that you can like telling is not
wrong, which was kind of taboo for a few decades. I think education is a really hard field.
It's a difficult field to research in because there's so many variables and also because the researchers are not
disinterested in the outcomes. Typically the researchers want to create something good that improves people's schooling, but that also encourages them to p
hack and do all sorts of other things and probably without even being conscious of what they're doing. And it just leads to a level of research that
I feel like does not hold up to scrutiny. Having said that, I feel like there are
some strong leadings from the research, like the things I was showing tonight
that I think do point us in this direction of there are some things we clearly need to do. We need to build up that long term memory.
We need to have this effortful practice. We need to push people beyond their comfort zone.
So. So I don't know exactly where the movement comes from, but there are education researchers, so hopefully it comes from there.
As I say, it's happening in some places, it's happening in Australia. Hopefully it'll spread throughout the world. Yeah, fantastic.
All right, so now we have quite a line for questions. So just a reminder to keep it to one question.
That way hopefully we can get through everybody. So go ahead. Hello. I'm curious because you mentioned art and I
found that many people that choose to go into the arts aren't necessarily doing it for. For money or for a reward at the end.
So do you think that. And I'm reluctant to Use the word fear. But do you think that our fear should be that people start chasing art?
Or the people who observe the art no longer care of the origin of where it was made?
I'm not entirely sure I understand your question. So when you mentioned generative AI and how will people still
do things when AI can do it for them? People who chase art already are doing it
without unnecessary promise of a reward. Yeah, yeah, yeah. Do you think that? I think that's great.
And probably that's really encouraging that people are just into making art for the sake of art.
And so maybe I should be less worried about that. I mean, like, I personally, for my kids, I'm just worried, like, are they going to learn how to write?
If you look at what's happened to our handwriting, you know, all of us, because we don't really handwrite, so what do you expect to happen?
Well, it's going to get a lot worse. We're better typists than we are handwriters. And what if we become not that great at.
What if we become not that great at writing at all?
I think this is one thing that I didn't quite mention in the talk, but this idea of it is that vast structure that I think allows
experts to have great insights. I was thinking about this yesterday,
almost something I put in the talk. It was about how you see a lot of
great advancements in thinking physics coming from young physicists. And part of the reason for that is I
think when they build their networks, the teachers who teach them have slightly
different networks that follow the historical progress of the science. And at some point there were some dead ends
that didn't work out in physics. And those are still in the minds of the teachers, but they're not in the minds
of the students because they don't get taught to the students. And I think that frees up those structures to
find new possibilities, which is kind of why I think Einstein maybe was open to this idea of, well, maybe space and time are malleable, but
for the people that came before that, their structure just wouldn't have permitted that kind of like flexing that new idea to come
around. Anyway, I think it's really important that you have a very well developed structure
of writing in order to be able to express yourself and express ideas. And if there's a machine that does it for you,
maybe we lose all kinds of important insights. Thank you. Yeah, I don't know that I answered your question, but I'm glad that
people will still be making art. Hi, Derek. Huge fan.
So you kind of talked a lot about System one and System two and how System one is quick
and it doesn't make much effort to think. And I have observed that with AI coming over,
we are using System one quite more often and like, keeping System two more on rest. And I feel like System two is designed to be
more curious and it takes the process slowly and understands the problem to solve it.
With System two being on rest and not being able to be much curious, and I think, like,
curiosity and learning goes hand in hand. How do you feel about the learning evolving?
Or let's say my question is like, if you had AI at the age of 20, do you think Veritasium would exist?
These are good questions. If AI was around when I was 20, would veritasium exist?
I don't know. That is a crazy counterfactual. I don't know.
Hopefully we're still all encouraged to follow the things that really
intrigue us and engage our interests. I think as a first gentleman who talked
about, can I use AI as this to look up or look it up in a book, it doesn't matter.
So in some ways, I think AI provides a speed up that way. And even these days, I haven't
really explored that area very much in terms of the things I create. But I'm intrigued to think about what could
I create in future using AI and maybe it's possible that I can create some things
that are really incredible and powerful. I don't know.
I feel like AI just. It's tainted a little bit because there's so much like, AI slop out there.
But I try to imagine a world in which you have really high quality AI and maybe it's amazing.
So I don't know. It's a short answer to your question, but I will experiment. Maybe you'll see some Veritasium that sort of
has AI in it, and you can tell me whether you think it's better or worse. I look forward to that.
All right, so everyone who's in line for questions, stay there, but I think we'll get through this line and then
we'll move to the atrium. So go ahead. Hi. I was wondering how you'd like, solve
the problem of scaling, because, like you mentioned, the best way for teaching would be like establishing a personal connection.
Right. Like, that's easy to do when I'm teaching, like, 10 people, but if I scale that up to 100, it's harder for me to build a personal
connection with 100 people and, like, you know, having them be accountable and, you know, be encouraging to, like, each one of
them individually. So, like, how would you, like, approach that problem? And I think that's somewhere, like, where AI can be helpful.
Yeah. Or how do you scale a personal trainer? How do you scale a plumber?
How do you scale an electrician? You don't. You just have lots of them. And I think that's the answer.
I think that's the solution. I just think people are always going to complain about education.
People are always going to be like, they don't know anything, whatever. I mean, I feel like it might
be close to the optimal. You know what I mean? And that's why it's so hard. I am more optimistic about us disrupting, say,
healthcare in the US than I am about us disrupting education. It just so hard.
I don't know. Right. I guess that just comes down to, like, resource management, I guess, because.
Can't have enough teachers. Yeah, I mean, we do have enough teachers in a way, and the goal should always be
to get more of them and to make them better. And I think, like, that's a system problem, but, like, we can do that.
I think that's probably the way to go. Okay, thank you.
Hi, Derek. Earlier you touched on the way that people learn, like an effective way that people
learn in the classroom, where it's like you have a framework and you slowly take away line by line of the equation kind of within
this context. How do you think, or where do you think the place of AI would be within this,
this way of learning, if at all? Yeah, it's a great example, a great question. I think, you know, AI can
offer lots of scaffolding. You can, like, say, hey, can I get a hint? You know, so there's, there's a remarkable
capability for it to sort of fill all kinds of gaps in learning. Or like, you could say to the AI,
hey, I need to learn about, you know, this period of Canadian history. Ask me 50 questions about this, like, who?
What a great tool to be able to have if you can use it effectively. Right.
So all of those ways that you can use it as an educational aid, I think will be great.
I think the place where it won't be great is where it allows you to do the work without doing the work.
And it's that work which is essential for learning. So, like, that's my big concern. Yeah.
Thank you. Hello. You know, just going to say, you know, the, the unadvertisement, I still think
it's one of the most effective ones. It's. I've Never forgotten it. Yeah, but my question was, it's like, with the onset of, like, chatbots and things,
it's become, like, so easy to get, like, assignments, like, just done without work. Right.
But it's like, I've personally never used them because it's like, you know, I feel like it's against my pride to,
like, you know, But I mean, other people, like, you know, in post secondary, right.
It's like people just. It's like, they're like, oh, yeah, no, I use, like, replit. And it's like, well, I mean, but it's like.
Do you think it would be, like, better to just, like, outright, like, it. Remove it from, like, the early stages of education and for.
Only to come back for, like, when you're, like, older, when you've, like, developed your
effectiveness at work and stuff? I absolutely think there's gonna have to be some of that. I think it's a little bit like, you know,
when I was at school, we had two TI82 calculators, maybe TI83s, and we were
allowed to use them on a portion of the exam, and then there was a portion of the exam where we weren't. And I think it's the same thing.
I think when it comes to writing, there'll probably be some writing where you can have an AI aid and somewhere it has to be no AI.
And I totally think that's essential. Especially at the lower levels of schooling, we're going to have to have
writing sessions for hours in class just to get people that experience of going through this challenging process.
So. Yeah. Yeah. All right. Thank you. Hi, Derek. Thank you for being here.
I just wanted to ask. To touch on the previous questions. I think that you were going to make
veritasium either way, to be honest. And I fear that many of the people in this room know how to break their
brains and get dopamine out of it. So how do you think we can effectively get people excited about
breaking their brains on their own? Because I think that's the key thing. If I can share that. What do you mean, breaking their brains?
Well, I mean, I feel like everybody here might understand, like, when you don't understand something, it's exciting.
A lot of people when they don't understand something, it's not exciting. So how do you think we change that?
That is hard. I feel like that's, like, that's the existential question. And this comes back to, like, me asking
random people on the street, you know, what elements does it take to make water? What's happening to CO2 in the, in the atmosphere right now, it is not hard
for me to find people who don't know the answers to those questions. Like most people don't know the answers
to like most of these simple questions. And so like I say, I don't know
that you can ever take someone who is just not interested and make them interested. It's kind of like people are going
to be into what they're into. And this is where I just have like compassion for like, we're kind of not
as a species, we're not evolved to seek out theoretical physics and yet a small, tiny minority of us do.
But then we shouldn't look around at everyone else and be like, what are you guys doing? Like, look how awesome this is.
You know, like, I get it, I get it for this room, but like, I don't get it for the public, you know, So I don't know.
That's my thought. Like, maybe there isn't a way. Tough question out. Yeah, it is. It's a tough one. Thank you.
Sorry. She wanted me to go first, so I'll do one first. So you've migrated from short
form video to longer form. And so my question is, is there a sweet spot where you get content overload?
You know how you're saying, like, there's only so much content you can pack into like a lesson, for example. So I teach, so I'm just curious, like,
is there like a minute or like a content? Like, I love how deep you are able to go to stuff. I was just curious if there's like,
do you have to think about content overload when you're doing videos? Yeah, I mean, maybe sometimes we do overload people with stuff. Honestly, like YouTube has
algorithmically pushed us. I don't know if it's apparent or not, but YouTube has algorithmically pushed us toward longer videos.
So in the early days, the videos were three minutes, two, three minutes, and now they're 30. And why?
Because if we make a good 30 minute video, it gets shown a lot more. And that's just the name of the game.
So I know that sometimes as we approach sort of 15 minutes into a video, 20 minutes into video, and we'll start putting integrals
and, and derivatives and stuff into the video. And I'll recognize that at this point we may be losing some people and maybe
this is too much, but then it's there for the people who want to keep watching. And I feel like if you've
watched 20 minutes, why not 25? Let's keep going.
So I don't know. This is also kind of a business question for us, which is that what do you put in the first minute?
That's pretty essential. You don't want to start with crazy math. What do you put in the first five minutes?
You know, if the writer that I'm working with comes to me and says, yeah, I want to go through this whole derivation in minute 25, I'm like, let's do it.
You know, by that point, the only people around are still the true believers. Or another thing that's great about videos.
If someone is feeling like they don't get it, they can always go back re watch. They could pause it. They could go take out a textbook,
they could write some things down. So hopefully the videos are still, you know, can be effective for a certain population.
But I do recognize that we may lose people sometimes and later on. Thank you.
What's your favorite video you've ever made in your life?
It's such a hard question. I do feel like every video is like one of my children.
Like, it's very, very hard to love some more than others. You know, the one that's done the best
for us is on Goodall's incompleteness theorem. So I wonder if I would say that one also.
There's the shade balls one on the reservoir. Have you seen the shade balls one? Yeah, I was gonna say the radioactivity one is my go to.
Which one? Radioactivity. Like the most radioactive places on Earth. It's an oldie but goodie. Yeah, an oldie but a goodie.
There's one that I really like which is about. It's only like four minutes long,
but it's about the most common cognitive bias out there. And I just feel like it does such a good job
of hitting on what science is all about without really talking about it
in a sort of boring methodological way. Like, it's a really concrete way to think about it.
So I was excited by that video. Thank you. Thank you.
So if you find that in person, education succeeds because people are forced to engage with the content they're forced to.
To put a conscious effort in. Why do you find that online edutainment succeeds when people can check out so easily?
Yeah. So I kind of think maybe they're doing different things. Like, if I was your teacher or lecturer,
then there would be stuff to slog through as well as stuff to enjoy. I think being an online educator, I get
to just pick some highlights and make some exciting stuff that hopefully energizes people, shows them what's possible.
I feel like my job is sort of a combination of educator, but also
magician, also comedian or something. Not that I'm particularly funny,
but yeah, I think it's all those things.
So I think there's a place for that and then there's a separate place for like now you have to sit down and practice
and that can be more tedious. So, yeah, hopefully one leads
to the other, at least in some cases. Yeah. So do you find that people tend to like self select for edutainment 100%?
Yeah. I mean, one of the reasons we could do what we do is because we aggregate from across the world. If I was ever on a.
Just a broadcast channel somewhere, like, you know, the CBC here or ABC in Australia,
like, I could never go into the depth. As soon as I would put down derivative
on screen or something, they'd be like, no, no, no, no, no, you cannot. No, don't do that. Let's do an analogy to like skiing or something.
You know, you just lose all the rigor. So it's. The thing I'm so grateful for is the Internet came along at a time when I wanted
to do that and there was people across the world who were hungry for it. So I just feel so incredibly lucky to be able
to do this and go into depth and. Yeah, reach a lot of people with it. Thank you. Thank you.
Hi. My question is if you like start talking
to AI or like if AI was in a human body, like talk like a human, acts like
a human and you like become friends with it, would it count as social interacting?
I think that is an amazing question. Amazing question.
Let me say this, that one of the best ways to learn that's been shown has been
if you have like a one on one tutor. There's what's called the two sigma effect, that the performance of those students is two
standard deviations higher than everyone else. So it's a huge effect. And if we could replicate that effect by
having something that is so natural and so human seeming that it would just feel
like you're there with another human, that might be really, really, really powerful.
I wonder. I feel like this is such a big question, like, how much do we need to know that it's a real human there?
Or how much could a fake human actually make us feel in the exact same way?
And I think that's what it comes down to. That's such an amazing question. Thank you. Thanks.
All right. That's a tough question to follow.
Thanks. Great talk. I'll start here. So I definitely Agree that revolutionized
isn't the right word, but I do think that all of those technologies have changed education in some way.
So for me personally, I know that the accessibility that was gained by so much content being on the Internet
has completely changed my educational journey. So my question is, if revolutionized isn't
the right word, what would you say is? Yeah, I feel like a lot of technology
just becomes a tool. You know, I think of them as tools.
I just think the contrast is so weird when you think about how has the Internet changed our lives?
It's really profoundly changed our lives and like, it's touched on education and yes,
like independent learning journeys and so on. Yes, the Internet's huge for that. But in terms of like, what has it done
in the classroom, what's it done to pedagogy, curriculum, all that sort of stuff taking people up?
I don't think it's done that much. And I just think it's so remarkable when I think about the way all these technologies really have revolutionized
things like smartphones. Right. Again, totally changed our lives. Now we're all on social media and we're all depressed.
But, like, you know, it's done that. It's had a huge impact.
So, yeah, just come back to, like, what happens in education. What just seems like these things become tools in the hands of educators.
They're tools to be used and exploited and so on. So, I mean, that's the word that comes to mind for me.
Thank you. Of course.
Stand up, stand up here. There we go. There we go.
If we didn't completely remove AI from education, what tools do you think we would
add and what tools do you think we would remove if we didn't completely remove?
If we did or didn't? Didn't. If we don't completely remove AI from education, what kind of tools are we going
to add and what are we going to take away? I don't know if I fully grasp it.
Maybe. What parts of AI would you keep in the classroom? Yeah, yeah. I mean, for me, the big role of AI is
that kind of like drill and practice type stuff, really getting people to do the reps, giving immediate feedback
with whatever someone's working on. I think there can be some powerful use cases. Yeah, the real risky use case is the one
where people use it to do their work for them, which is, I think, the one where it's being used most unfortunately. So that's the one that's concerning.
Yeah, that's the one we gotta get rid of. Thank you. Good question. Thank you. All right, last question from the audience.
I'm very short, so you can keep it like this. Thank you for your wonderful talk.
My question was hinted at prior and I'd just like to make it a little bit more prepared, precise.
So I'm a grader for this quantum mechanics course and I graded my
students midterms recently and the performance was extremely poor. It was abysmal. And this in no way was indicated
by their performance on assignments. And of course, I think it's very reasonable to conjecture that this is due
to the advent of ChatGPT. Students can just copy the solutions from ChatGPT.
Right. Res on the midterm, they don't have that access. And I think it's very reasonable to
argue, oh, perhaps we should change the evaluation methods for classes to be more exam heavy, less assignment heavy.
But at the same time you can argue that exams aren't the best methods of evaluation either, due to performance
anxiety and all these other things. So I'm wondering, what do you think is a, like a preferred rubric or evaluation method
for courses nowadays with the advent of AI? Yeah, I mean, the first thing that came to mind
is maybe the way that assignments are done now shouldn't be the way they're done in future.
Like maybe assignments need to be done in the same way that midterms are done to remove that risk.
Because I really worry that that is how you build that long term, that huge structure. That's what we're after.
And if we're not building that, then, yeah, by the time you get to a midterm or the final, it's too late.
The practice has to come in the lesson. I can tell you that. When I was lecturing first year physics
at UTS University of Technology in Sydney, I had stacks of cards at the
front of my lecture that said in huge letters, A and B on 1 and C and D on the other.
And I would force the students to come and get these cards at the beginning of every lecture. It was a 400 seat lecture theater.
And then as I would go through, I would ask a question maybe once a minute or once every two minutes, and they'd have
to hold up these cards. And that's really helpful for me as a lecturer
to make sure that they are where I think they are. It's also really helpful for them
in that it keeps system two engaged. It keeps them having to make that
decision every few minutes, which stops them from, for example, falling asleep, which deactivates both system one and two.
But it's also also really easy to tune out. And then, you know, that's also not helping.
So the whole point of getting this back and forth, even in a large lecture theater, just with these cards, I mean, was my attempt at
making this interaction and getting people to be thinking all the time, being really
mentally active. So I'd say we have to do the same thing with this work that they're
doing with the assignments. Like it has to be done. We got to find some way to make an AI isolation booth or something.
I will. Can I just say one more thing? So I think though, when you get into higher education, especially in physics,
the higher up you are, the longer it takes to complete a question. For, for instance, on a three hour exam,
there are probably only like three questions because it takes an hour to do each. And so reconciling that in a university setting
is much more difficult because you can't just complete a question within the span of a few minutes and then give an answer.
Right. So could you comment on that? No. I mean, it's a really good point.
I just don't see any way of getting away from the idea of there's going to have to be assessments at some point.
It could be the midterms and the finals. And if people are failing those because they're not doing the assignments properly,
it's just something that people have got to come to terms with. And I know that sucks from like a structural and logistic perspective,
but, like, there's no other way. If you're not learning the material, you're not graduating just the way it is.
I thought that I was an amazing lecturer for my time and I also thought that I gave very fair and very reasonable questions.
And on the midterms, my students were scoring 40%. So on average, I feel that pain.