What just happened?

To kick off series 6, Dan interviews Ray Fleming about 'What just happened?' in terms of the landing on Generative AI and ChatGPT into society.  We lookat how it might change assessment, courses and more. 

• AI Business School Artificial Intelligence Courses - Microsoft AI

 

________________________________________

TRANSCRIPT For this episode of The AI in Education Podcast
Series: 6
Episode: 1

This transcript was auto-generated. If you spot any important errors, do feel free to email the podcast hosts for corrections.

 

 

Hi, welcome to the AI podcast where we've been exploring all the latest developments and trends in AI over the last few years and the impact they've had on society and education and all kinds of industries. I'm Dan and today we've got a special guest, Ray Fleming. Uh if you remember back in the podcast initial days which was gez about three or four years ago now. Rey and I started this the podcast has kind of morphed and changed and people have come and gone and we've gone full circle back around and welcome Ry. How you doing?
I'm I'm good Dan and do you know it makes me feel old when you say three four years ago
uh when when the podcast started I think it was five years ago but what is um
five years ago where's my life going?
It's absolutely fascinating is that I think people think that AI has just started.
Yeah.
Like people out in the general population, not the people that have been listening to this podcast because they know it's been around for a while, but I think to a huge number of people, AI means that chat thing, that thing that writes stuff for me. And and that's really where they're starting with AI. And I I was looking back last week, I did my first AI course in 2015. That's eight years ago. And Seven years ago, in fact, eight years ago, I was talking about AI to education customers when I was doing keynotes and stuff like that. It was all about AI, but but I think there's this consumer level of friendliness about AI that's happening at the moment.
Yeah. And I remember you actually doing the the AI business school course and sharing that with um with customers years ago.
Yeah, that was 2019 uh early 2019. So, yeah, five years ago.
That is amazing, isn't it? Cuz that that course is just as relevant today. I I I tweeted about it or shared on LinkedIn I think uh like a couple of weeks ago I completely forgot about it and there was there was all these business schools for education and for retail and things like that and it's the content's great.
Yeah, it's absolutely relevant. I I think um one of the things I used to say in all of those conferences and keynotes and stuff like that, especially when I was talking with nontechnical people, part of the message is always you've got to start now because you've got to begin to get your understanding. You can't go from zero to 100.
And um you know I I think it's the same thing and that you're absolutely right that AI business school course uh that's um on the Microsoft site the learn it's on learn.microsoft.com
that course I know has had a bit of an update but really the fundamentals that that course are the same as they were in early 2019 which is
the potential to do things very differently.
Yeah. And you've had an opportunity now to look outside across multiple like organizations and industries over over your career and you you were able to kind of look across multiple technologies. What kind of other things are kind of appearing to help people learn I suppose about all of this stuff? Cuz cuz I I've got my my kind of biased lens on on where I'm coming from on the things I'm seeing. But I'm sure there's a lot out there, isn't it?
I think you're saying some of the best uh if I'm honest.
Yeah. Well, you know me, I'm I'm I'm the kind of epitome of lifelong learning. So, if I'm not if I'm not running a course and studying a course, then there's something wrong with me. So, I've done about nine AI courses since wow 2015. So, 2015 then there was a bit of a gap. Then 2019 uh onwards I I've done all kinds of courses. I've done uh data and ethics uh and AI which of course is fascate. Oh, I did a generative AI course of course. Um, the AI business school you mentioned. Uh, I did a AI for product managers. So, it was about how do you manage product?
I remember you saying that. Yeah. Yeah.
And and they're all they're all good. They're all really good. But I say I would say that AI business school is a really good structure if you're nontechnical. Actually, forget the nontechnical. If you're nontechnical, it's good. If you are technical, it's good too because it helps you to relate the wizzy fantastic things your technology can do to the business problems that everybody is trying to solve. And you know the way I talk about this Dan it's okay to say business problems in education because the educ the business of education
has you know a lot of things and processes and things it's trying to do and those are business problems. So that uh AI business school and then probably dive down I would say into the ethics and responsibility piece because that's really important. So I would jump from the Microsoft learn courses to Corsera and pick something that really interests you on Corsera around AI like the ethics piece uh like the privacy and and AI piece because all of those things are interrelated
because you mentioned that responsibility. What I'm seeing at the minute now is currently just from my zone of reference being my LinkedIn feeds and things I'm seeing a lot of the folks whether in departments of education or um universities things start to sort of settle but people just creating responsible AI standards or principles or or that kind of element and I know we've spoken to people and we've spoken about responsible AI in the past for quite a lot of time but it seems to be like I don't know like a cycle the things have happened the cats out of the bag or whatever and things are happening and now people are trying to come to the table with policy we've seen something from the EU I think this week or one of the EU countries or maybe in the UK you know there's been big catchups with Richie Sunna there's been a lot of this policy happening now you know and I suppose we in June and when I when I sent a message out to all the CIOS that I manage in my particular role in December you know happy Christmas blah blah blah I sent out an email uh and said by the way you might want to try this chat GPT thing you might want to read Brad Smith's book about tools and weapon funds and you might want to read um a couple of other things and you know lo and behold we come back in January February time and then the GPT tools have just gone through the roof. What are your thoughts first of all on what's happened in the last 6 months and then where we are now.
Yeah. I mean I mean there's only one thing that history will remember has happened in the last 6 months which is the it's the a magical creation of generative AI which of course It's been around forever, but it made the leap, didn't it, from a technical thing to something that every consumer could get their head around because you went to this website and said, you know, write my homework essay in some cases or tell me about write 300 words about this and suddenly it became really relatable to people. And so I think that's the thing that is going to unlock a lot of the conversations and a lot of the potential in the future. And Most people can relate to it. And most people got that magic smile when they went and asked a question and saw this computer typing something for them and went, "How does it do that? How how does it do that?" And and so that that's probably the biggest thing if I'm honest that has happened not in the technology sphere because there'll be a whole bunch of things and people will be going, "Yeah, but GPT 3.5 wasn't as good as GPT 4.0 and 4.0 can't pass this medical test." Like, forget all of the detail.
Yeah.
The big thing is suddenly all this AI is meaningful to people because they can get it to do a task that they can truly understand.
But then you've had all of these applications which have been developed since then based on that technology you know any games platform or whatever somebody create something some bit of hardware and it's the the brains that build the things on top of that and the people coming from industry and going oh we could use that for this application and we see we've seen a lot of that in the last six months I I had not an interview meeting this morning at 9:00 with a company from Ireland um which had developed generative AI into an assessment tool you know and and it was great and they just developing on top of that those things. So we've seen a lot of that but then now we're at this position where I suppose people are looking at responsible ethics and really in a frenzy to work out what do we do about this?
Yeah. So let's let's draw a little bit of a picture of uh where we are, what's happening and then where where where that all comes in. So
yeah,
you know, first of all, we had generative AI, which is just awesome to say that phrase
because everyone understood the chatbot or the open AI and then the tech industry said, "Oh, no, that's too short a word. Let's call it generative AI."
So, so first of all, we had that and um of course the students got on it first and they went, "Oh, I could use that helping me with my essays."
Um and I know that because in some of the online courses I was doing, there's peer-to-peer assessment of uh some of the activities. So, it's like, yeah, think about the stuff you've learned, do 600 words about how you would apply it.
And, uh, three or four times I was asked to peer review somebody else's work that started with the sentence, I'm only a chatbased large language model. So, I can't really do this, but dot dot dot
and and so, you know, obviously another student on the course had just gone and put the question into chat GPT, got the answer, copy based not smart enough to rem remove the first paragraph that said I'm only a large language model. Um but there kind of is a question that if you can write it with an AI shouldn't you
like actually you should you know and so there's a whole question about the future of assessment based on consumer tools that students have access to
and and that point there as well the other thing that in terms of a perfect storm where you're saying that students went into this first part the other thing I'd add from an Australian perspective is all of the teachers are down tools at that time in January time from December to January doing planning and then suddenly people started to do their planning work use you know playing with chat GBT and then going oh this has just done my planning for me you know plan my lesson for for year seven chemistry but try to put a spin on it of Marvel characters or all of these kind of weird and wonderful things or write a rubric for me on this and suddenly like there was that that element as well from a teaching point of view So, so there was a I I I would say I would challenge you a little bit and say that there was a point in maybe for for three days in January where teachers were ahead of the students with CH GPT.
Yeah. But but I think individuals have been you know it's been a race for individuals and then it's the system problem isn't there is so so one of the things that I've been watching really interestingly over the last few months is this whole thing about how do we detect that students are using AI in their assignments and how do we stop it? And that's just a that's just an arms race because, you know, I I I don't honestly believe we're going to be able to solve that problem because somebody will build a uh an open AI model that uh generates things that cannot be detected by anybody else. And so there's the kind of race between the detectors and the non-detectors. And but there's also the thing of the AI system is blackbox and the text the detector is also blackbox. You don't know how it decided that that thing was or wasn't written by a robot. And so you can't do that in education if you are going to start making judgments on students saying well this was created by an AI if you cannot be absolutely sure you know um you know the easiest way to build a a absolutely 100% sure of finding AI detector is to say that every single thing is written by AI. So if In the world of AI thinking, and I think we've talked about this on the podcast enough, the true positives like finding AI written text and being sure and detecting it, the easiest way is to say everything was written by AI.
Yeah.
Um, now unfortunately, you then get a whole load of work that wasn't written by AI that gets labeled that way. And if you look at the tech detectors, they've got very high rates of finding things, but they've also got what I would say are high rates of falsely finding things. Something written by a human and it says it was written by AI and
even if it's only 10% that means three children in the class 30 students in a big engineering class whatever are going to be accused of writing something with
uh chat GPT and they've got no way of going no no no it was me I I sat down and wrote it I I read um a really good example where um international students where English is a second language um often They will write their essay in their home language
and then use a translation system and then polish it up
to put it into English.
Pretty much all the detectors say that those essays have been written by an AI system
because of the style that comes from the translator.
The assessment element to this I think if this disrupts enough of of assessment now saying that I thought COVID might have disrupted universities and things you know for more unis than I do. But I would have thought that CO would have disrupted hybrid learning more than it did. You know, everybody's clicked back into class face toface learning. Um, we can really think about different ways to assess now. What are your thoughts on that?
Yeah, we're going to have to the future of assessment I think is up in the air at the minute because the way that we've solved the problem you think about the way we handle it in COVID is more and more inspection of the learner when they're taking the assessment. You know, we all did a professional exam or something during COVID where we had to show the uh invigilator around the room with our laptop camera. You know, I remember my mom was like, "Make sure there's nothing printed on the ceiling. Now, show me the underside of your desk to show there's nothing there." Um, you know, so that for mumbling cuz I read questions out. So, I was like, you know, if I get a question on a proed test, I always go, you know, I'll read the question. I don't know why I do that. This guy said, "Please stop mumbling." Sorry.
Well, look, if we keep on that angle, the only person that will be able to pass the assessments are robots
because humans won't. So, we've got to work out the future of assessment. I think we talked about in the Christmas special things like, you know, set the homework to be go and get chap GPT to write your homework for you, then show me how you've improved that and corrected errors
because that's what people are going to do when they get into the world of work. Both of my children are out there doing professional marketing jobs and they're using ChatGpt to create content for them in ways that I wouldn't have imagined six months ago. So people are going to use it in the workplace. So we've got to think about how how we model for that. But on the other side,
on the positive side of it, isn't it great for busy work? And and by busy work, I mean uh you've got to have your lesson plan in this format. You've got to have your curriculum document done like this when when you know I mean you you would have had it when you were a teacher, Dan. You've got your favorite lessons that absolutely you just draw out the bag on a wet Friday afternoon because the kids are going to be b instead of knocking holes in the wall.
Now you can pull out that favorite lesson and say, "I need a lesson plan that's 422 words that fits in this box, which before you would have wasted time doing because it would be easier for you to describe it or show somebody than to write it down. So that kind of busy work,
that's great for that." The one in higher education is research publications. The amount of time that reaches researchers spend reformatting it according to the rules of each specific publication. I don't know if there's an AI out there for that, but they're blooming well to me.
Yeah,
that's a really good example of busy work.
I was just on the the research site. I remember when I was doing my masters and I I I forget the name of the first part of a dissertation out, but it's that where you do your research study and you do you go in and you do the qualitative research and you read the body of evidence and and and all of this kind of stuff. And you know, it it' be good to like just using things to summarize a lot of the the the pro was written, you know, and getting extracts of this stuff out and getting some what my my son did a good example of this the other day cuz I I asked him to use chat GPT for some of his English work cuz he was reading Handmaid's Tale and he was really struggling with this. So I got I got him on the computer and I said, "Look, ask the Bing chat, the Bing was using I said ask him to um ask her to summarize chapter one of the Handmaid's Tale and and he just did each chapter at the time, you know, in like three or four kind of sentences or bullet points and he just he could he got the gist of the book for his um he hasn't his results yet but it just gave him that and and I know it doesn't you know he's so he could be seen as cheating but he probably did more work on that than he would have I don't think he would have read the book I really don't think he would would have read the book at all he he started and he was finding it really hard
but I think you know the next stage is going to be like personalized models you know personalized trained with your own data whether it's an institution level you level, a national level, you a global level, you know, the the personalized learning, the personalized AI system for biologists or for people studying computer science or whatever it might be. And and I think we'll start at kind of individual institution level because it'll take a while to bring it together at state or national levels. Um, but it we'll we'll end up with some really personalized stuff rather than the general stuff that's available. And that's where you might start to get something where you start to build real student helpers
um that are designed to do that kind of thing. Uh we'll also get the return of the thesaurus. Dan, you're you remember when you were a kid, you would have had a thesaurus.
Um because it sounds like your son would have needed that because they would have taken the uh information from chat GPT and it's like I've got to rewrite it into my own words. Where's my thesaurus?
Yeah. Yeah. Yeah. Yeah. That's so true, isn't it? I know my daughter. So if thesaurus is huge. She takes a d she got every year got to buy the mccquory dictionary the mccquory thesaurus cuz we never find it seems to it goes into the school's library somewhere along the line I'm sure in lost property but it's huge but it's such a good point in terms of those tools you've seen that have come out personally you know I'm sure you've played with some of these as they've gone through what's really excited you which ones have excited you most have you seen any good applications you know generally it hasn't got to be an education context anything back back to where we started from the chat GPT the open AI stuff that's the most exciting thing because it's made it accessible to the everyday consumer
you know there are no barriers for people to start using that
the visual stuff you know midjourney you and I probably both follow on Twitter people that use midjourney a lot cleaves put some great stuff with midjourney you know I'm not so much of a visual designer as I am thinking about words and data and things like that so I'm loving watching that stuff.
And every now and again, one of the things I did over
over COVID, Dan, was I became an improviser. And I do um I've got a am I allowed to spru things? I'm I'm doing two man show in the Sydney Fringe Festival in September. All of the visuals for that were created by MidJourney.
Oh, wow. Really? That's fantastic.
You know, it's it's those kind of things. I'm also excited by a completely new professional that's going to come along which is prompting
which is how do you get AI to give you the results that are the best results. So that that whole art and skill about learning how to prompt AI to give you the results that you want and um that's going to get pretty interesting. It's quite a technical thing at the moment in the sense that you need to understand quite a lot to be able to be to do it well but I think that's going to become another skill alongside the other technical skills that we all need to do our jobs. You know, meaning you're a designer, you need to have visual communication skills. Well, probably most people are going to need AI prompting skills.
Yeah, I I agree. I I had this conversation yesterday um with with with my partner actually. She we were talking about um chat GBT and I think lots of people like sometimes you forget again when we're in technology that that everybody's using this, you know, and we get caught in our own like bubble again, don't we? And like I watching like you said, Pip doing her stuff and and Julian ridden. I know he's doing quite a lot in that area as well. And and Pip's doing something where she is doing prompt engineering each month to see how good the quality of the images are. So, she does the same prompt every month to see what what comes up with some Japanese art that she puts in there, which and the characterization in there is is phenomenal the way it's it's improving. Um, but I still think are are we going to start to miss out people again and and create digital divide? Because, you know, when I was talking to my partner yesterday, I said, "Oh, you've got to try to use chat GBT, you know, put some prompts in there or whatever, but I think people are not aware of the depth of that it's a conversation, that it's not search, you know, I don't think everybody's immersed in it like we are, you know, and and I think some people don't just assume it's like search, you know, please write me like this thing or work this formula out for me and then it comes back with a incorrect answer and they go, "Oh, well that's bloody rubbish." And then they move on, you know, and it's like, well, you got to keep you got to it's like a child. You got to keep persuading and go, No, that's not what I meant. You know, what I'm trying to say is this, and you got to keep rephrasing and referencing and building the the model.
But I think the speed of of that change is phenomenal. If you think about, let's go back to the other big inventions of the last h 100red years.
Yeah.
Or 200 years. You've got the invention of the telephone and the television and everything. And that was all going to ruin society. And
you know, the reason that kids were dropping out of school was because somebody invented the telephone. phone or the book or the bro. Um, and so I think this is a similar one of those things is it's going to be misunderstood uh to start with, but my goodness, the speed that things are changing compared to other technologies and the speed that it's being adopted by and understood by people is way faster than other technologies. I know that because of the people that I socialize with that are not in technology because I've got some friends that aren't involved in technology. at all. Amazing. I know.
Um, but it's really interesting having them tell me things about
how they can use chat GPT and the experiments they've done and it's just awesome. I get great ideas from them about things that you can do. So, I I think yes, there are there is a a phase in which it's misunderstood and the true potential isn't in isn't um brought out of it, but it's going to be pretty small compared to other new technology coming in in the past which has taken I mean we used to talk about decades for adoption of things and then we started talking about years for adoption of things while we know that you know we're in months now but you know you still need to get socializing it with people the smart people are doing their jobs faster and better using chat GPT and so the person sitting next to them is going to spot that they've done their work faster and it's going to say how do you do that
yeah I'm interested to see that because you mentioned about your daughters there and I'm I mean just to see how current generation who just might gone to university learn learned about writing literature you know copywriting whatever and then um how they embrace that change cuz we always think that younger people will go and pick up whatever app and do whatever but you know it'll be it be interesting to see have you have you had much feedback from your daughters about it do they just think it's amazing or are they a bit cynical?
No they know how to use it so they just use it for the things it's good for. you know, if they've got to write something in their corporate style, they won't use it. Uh, my daughter's office, they're having a day out of the office, and part of that is kind of um they need to fill some gosh, I hope nobody from my daughter's firm is listening to this in the future. They're doing a an architectural treasure hunt around the city. And so, she said I needed to to set some clues. So, I just got chat GPT to write riddles about the things that they're going to be going to. So,
yeah.
Oh gosh. One is going to get advantage of the listening Sydney Opera House. It wrote a riddle about Sydney Opera House as a clue. And it's I would never have thought of doing that. And so, you know, that's a really good example of, you know, the kids get this stuff. And and and they're not kids, they're fully grown adults, but true.
You know, yeah,
they they get it. And and I don't think it's an age thing as it often is with technology or often been accused of being with technology. It's actually are you willing to try doing something in a way you've never done? it before and learn along the journey. And I think there's a lot of people that are in that box. They're just happy to go and try something.
I agree. And and I think the one of the one of the interesting ones for me, you know, in in terms of what I I've seen out out of this was was something that one of uh my ex-colagues back in the UK, Chris Goodall, did and um he shared it with me and I've shared it with some teachers over here. What what he did, he asked he was in Bing AI again and he went in and he said um uh he showed me slides on his great so basically said, "Look, I want to run faster and jump higher than anybody else has ever done in history. What do I need?" And he kept prompting. He prompted it for three times. And then basically chatbot came back, you know, GPT came back and kind of said, "Look, you need a shoe that's got enough spring in it, but also it's got to be light enough to move forward." And he was asking questions, what material should he use, and then he said, "Could you design this for me using prompts?" So, uh, like which I could put in into Midjourney and basically he had another conversation with it and it prompted him to actually put into Midjourney which he then put into Midjourney and it created this phenomenal athletic shoe that had never been created. So, so from his idea of what he wanted to do, it had gone through this conversation then gone in and app smashed into a completely different app and then created this like shoe that had never been seen before. That that for me was a great moment of you know really originality coming through there and creativity. in the way that that was that was prompting it was it was fantastic.
But it did need the human vision at the beginning and during the process to go I've got this idea and and you know
legion
yeah I think people are worried that you know AI is going to rule the world and we're not going to have jobs and anything to do but actually many many many of the scenarios I see start with human creativity and they get guided by human creativity along the way and and so that's good for us all but it's also good for you know, remembering the innately human skills that we all bring to any situation that that that's going to get emphasized because that's the bit that you won't be able to replicate. Let's get rid of the boring stuff and give it to
Well, I give a good example of this. Um, there's a podcast I follow who's a guitar like um person called Rick Bato out to the US and there's a band called PI I think they called and there's a guitarist there called Tim Henson who and and he interviewed guy and basically he's a young guy who's who uses AI to create uh guitar music that he can play. So if the AI is making him like uber creative so like Rick Bat who's like a stunning musician and sort music for all conservariums and he's a producer and things and he's watching this guy playing the guitar and you can see his disbelief because it doesn't fit within the norms of how you learn. not teach or even play. You wouldn't play you'd never play this note with this note and you'd never play these things, but the AI is just basically mashing it together and and composing something that this guy can play and like which is is unreal. So, it's it's boosting the creativity and then allowing that that creative spark to kind of be developed even further.
So, let me tell you about three things I'm excited for for the future because it fits exactly into that. So, the first thing is I'm excited that we're going to get rid of the boring stuff. So, things like having to read emails, especially the the corporate emails that somebody has crafted very very carefully that you have to read five times to understand what they're saying. So, uh write writing admin documents, you know, we we all have that kind of thing. And do more fun stuff. So, get rid of the boring stuff and do more fun stuff, more of that human human interaction thing.
Second thing is I'm excited by the potential of nontechnical leaders to understand the potential of AI. You know, they're they're all hooked. They've all found a thing. And so that's going to uplevel everybody because um often those non-technical leaders have viewed technology with suspicion, but now they can it's much more relatable to them. They still have some suspicion about what's going on in the background, but you know, getting nontechnical people engaged, understanding, and um wanting to take advantage of the potential that that's the second exciting thing for the future. And then I think the third one coming right back to education is the potential for personalized learning at scale. Um and and what I don't mean is some of the examples I've seen at the moment where you get a computer voice to simulate uh to simulate you get a computerenerated video of a head and you put together a training video. I mean that kind of thing. But taking the learning resources and the learning journeys we have at the moment and personalizing them for the starting point of an individual and the end point they want to be at. Um, personalizing it for, you know, somebody with the reading age of 12 probably needs could get get the same resource that you have for a PhD student, but rewrite it
personalized down to somebody with a reading age of 12 or 10 or whatever it might be.
That's right. Yeah. Yeah.
As somebody who is an as English as a second language. Um, currently they have to a lot of learning is inaccessible to them because they, you know, that it's written in such a way that I barely understand some of the words and I have my dictionary tutorials next to me. You know, imagine somebody that's just arrived from a third country and they're having to do all of that, the barriers they've got.
Just think about what you could do with some of the generative AI stuff to say, um, you know, take this reading and make it access to people who don't have English as a first language and just watch the magic happen. That that kind of so many potential ideas for how we could personalize learning.
It's a great way to bring things together because like that that was my question where where things would be going in the future and I think you kind of encapsulated that really well. Is is there anywhere is anybody you you'd follow at the minute? I mentioned a couple of people and everybody's doing different things in these. Are there people that that you follow at the minute in the community and in education that um you share here. I I know for me, you know, PIP definitely around that mid midjourney stuff and I mentioned Julia, there's a guy down in South Australia who's amazing called Dr. Nick Jackson and he's he's really bringing on bringing on the student agency discussions around this and bringing students in on the conversation as well and he can see the impact and see the change. So, he's doing some amazing stuff and I mentioned earlier my ex-colagues, Chris Goodall in the UK, he's picked it up and really ran with it and he's really sharing some great practice this and and some great insights into that and and leading the way there. Any any people from your professional learning network that jump out?
Yeah, the top end there's people like Simon Buckingham Shan UTS you know he's been doing a lot on learning analytics and by extension uh machine learning and AI with data doing some great work around the the institutional policy side. The other is Matt Eastman's talking about this a lot.
Oh yes yes
you know and and the great benefit of Matt is, you know, came from the classroom. So, it's not one of those, oh, I know what you should all do things. It's I know what I would have done if I'd had this. He's great. Phil Dawson, Philip Dawson at Philip Dawson, he's doing some great stuff around assessment and some of the conversations around assessment, but as part of that, doing some really good insights into how this all works. Um, and so, yeah, I I'm definitely getting value from those three as well.
Yeah, that's brilliant. Well, well, thanks for coming back and and saying hi, Ray. I think we're going to be rebooting the podcast coming up. So, like I'm sure this is not the last time we're going to chat and we should keep this conversation going to kind of really support everybody out in this area because things are moving so quickly. Really, really are. So, thanks for jumping in today and sharing your insights. It's great to kind of uh chat again and um thanks so much.
Brilliant. Thanks, Dan. See you soon. Bye.
See you soon. Bye.