Transcript 0:00 My guest today built the first transformer models, the architecture inside ChatGPT, Claude, and Gemini. They saw the paper, they're like, "Okay, let's train that." That was GPT-1. What did you see that others didn't? 0:13 We were teaching machines to code, which is a fancy way to say vibe coding. ChatGPT was first moment where it's like, actually, it's now working. We as people won't even be using blockchain directly because our AI will. 0:24 With blockchain, we're always like, oh, yeah, ev- every... Mass adoption is almost here, right? And then it kind of never arrives. 0:29 Why should these agents use a blockchain when they can use infrastructure like Stripe or Google or Mastercard or Visa? The interesting thing is all of these are also doing blockchain now. 0:39 Not your AI, not your intelligence. Otherwise, we're ending up in 1984 style future. Plumber or software engineer for your kids? Uh, I would say artist. 0:58 Welcome to another episode of 51 Insights. My guest today built the first transformer models, the architecture inside ChatGPT, Claude, and Gemini. 1:08 Then he left Google before the paper was even published to build a blockchain called NEAR. Today, NEAR moves billions of dollars a month and just got its own US ETF. Illia Polosukhin, welcome to the 51 podcast. 1:22 Thank you. Great to be here. It's amazing to have you here, Illia. In 2017, you were one of the eight Google researchers behind Attention Is All You Need, the paper that introduced transformers to the world. 1:34 Then you left Google before the paper was even published to start NEAR, and you were the only person out of that ensemble who went into crypto, uh, instead of going into AI. What did you see that others didn't? 1:47 So the originally, actually, NEAR started as AI company, so in 2017, and we were teaching machines to code, which is a fancy way to say vibe coding, effectively, um, before that, that word didn't exist. 2:02 And we had a lot of challenges doing data labeling, crowdsourcing, and, like, just coordinating people at the time because we had computer science students around the world doing small tasks for us, and we just had trouble paying them. 2:17 And so that was kind of the first, like, oh, actually, if we had global payment system that just supports microtransactions, we would scale this so much more, get the training data we need, train a better model. 2:29 And kind of as you go down deeper through the rabbit hole of blockchain, you're like, oh, you know, this kind of idea of Bitcoin is cool, but, you know, proof of work, slow, expensive, you know, economic at ta- like, tail. 2:42 Ethereum smart contracts, really cool, but also, like, slow, kind of expensive at, at the time, 2018. 2:49 And you're like, oh, actually, there's a opportunity here to build something that is built for machines, like for machine economy, uh, as well as human economy, that can move and scale like the systems that are, you know, built for kind of billions of users scale, and can really enable this, like, new properties of ownership, global payments, and kinda global coordination that will be required as we're kind of entering in this called post-AGI world. 3:22 Illia, you've really been in AI all your life, and before we talk about what all of this enables, I wanna take a step back and talk with you about your past, which is you actually studied applied math, and then you built your first neural network with 14. 3:40 And then you joined Google Research after the 2012 Cat Murum paper. Can you just unpack that for us a little bit? What was going on when you were 14 and you were already building your first neural network? 3:52 So, so I was interested in AI in general since I think I was 10, and I was reading a lot of science fiction kind of in my teenage years. You know, everyone from Asimov, Heinlein, 4:03 where this idea is, artificial intelligence was very prevalent, and it was, like, in n- in all forms from, you know, obviously robots to assistants to kind of full, you know, beings. 4:14 Like, there's also Stanislaw Lem and other authors. 4:18 And so, like, to me, that was always a really exciting concept as well as, you know, I was already programming also since I was 10, and so, like, I was repeatedly asking the question, you know, "Why are we writing the same code over and over again? 4:32 Why are we kinda like, you know, we need to change one small thing conceptually, but then you need to rewrite a lot of code." 4:37 And so to me, like, okay, I think computers will just understand natural language, and they will write code for themself. 4:44 That was my core thesis very early on, and I was like, okay, well, how, how, how to actually build that? Well, it will be some kind of obviously machine learning system. 4:53 And neural networks were very conceptually interesting because it is, you know, very much how humans and other mammal species at least operate. And so I was trying, like, what does it take to build that? 5:09 And so I was, like, trying it out, right? 5:11 I, I, as I said [chuckles] in that quote, I, I didn't actually know, like, how, how to do gradient descent by hand, and there was no frameworks at the time, at least and I didn't know any frameworks at the time. 5:22 And so yeah, I was, I was doing it ver- very manually and, and p- very poorly probably. 5:27 But I was excited about this topic, and so when I, when I was choosing where to go to study, uh, that was actually, like, literally my question to the, like, I was, I was talking to 5:38 few different departments, and I was like, "Are you gonna teach neural networks?" And so, [chuckles] and so the department, uh, that I went to, applied math, they were, they were, that was on curriculum. 5:48 And, and all the prerequisites, meaning linear algebra [chuckles] was a f- for first year of the university was teaching that. So, so that was one of the, "Okay, I wanna, I wanna go study that." 5:58 And in parallel, I actually started looking for a job in machine learning as well. 6:02 So on the first year of university, I actually got a job part-time working for US company That was doing machine learning and it's like actually, you know, it was a small but long-standing com- company at the time who's been pioneering a lot of, uh, machine learning methods from like '70s and '80s. 6:21 And so worked there to gain experience, like a lot, a lot of really great projects and like learning there. 6:29 Uh, but at the same time it wasn't, it wasn't neural networks, it was using kind of the decision tree methods a lot. And so I was always like on the lookout when neural networks actually start working. 6:40 And this paper by, it was Andrej and Jeff Dean and, and, and other co-authors who effectively scaled training of neural networks for the first time at Google. Uh, this is like 2012, 2013, 6:56 and specifically to train it without any supervision, right? So now, now this is like, oh, it was like, I think like 600 or 700 machines on CPU, right? 7:06 And, uh, you know, they trained it effectively by giving input and predicting the same input back. 7:11 And like right now it's like, yeah, I mean that's what everybody does and like 600 machines is not, [chuckles] it's not a big thing, right? 7:17 Uh, back then it was like, hey look, this is actually a pretty big scale and it works completely unsupervised, right? 7:23 Which all of the methods for the most part were, were like you needed somebody to tell, "Oh, this is a cat, this is a dog, this is something." And that model learned that all just from data. 7:34 And so to me that was like a signal that, hey, neural networks work. They can scale and they now can l- learn from, just from kind of experiences. And so how do we actually apply that to language? 7:46 How do we now train, uh, understanding of semantics and reasoning in these models? And so that's why I joined Google kind of natural language kind of de- department research, uh, re- to research on that. 8:03 And then in 2017 you worked on that paper. Five years later in 2022, OpenAI released GPT to the world, uh, which took the world by storm. Was that a surprise for you or did you saw that coming years before already? 8:19 So GPT-1 was actually in 2018. So, so by the time the ChatGPT came out in 2022, that was actually a GPT-3.5 effectively. And so GPT-1 was 2018, which they, they saw the paper, they're like, "Okay, let's train that." 8:37 That was GPT-1. There was then GPT-2, which, so I don't lie on the years, I think was 2020... It was February 2019. And then there was GPT-3, which also was publicly available in 2020. 8:50 So the reality is actually like there was a lot of signal on like improvement that was happening in research. It just wasn't a product, right? 8:58 It was really like a research, hey, we actually have like really cool language model. Uh, you know, if you type it like the, the, the, 9:06 the example people would use it for, you would type in your name and it'll generate a Wikipedia article, right? And it was like, you know, c- usually half bullshit and then over years it became more and more realistic. 9:18 Um, well like it, it read well and then it became more and more actually like, actually the thing about you. 9:24 And so, so the reality is like, I think if you were in the industry and kind of saw the progress, it was somewhat expected. 9:32 Obviously the like, the level of how well it was doing with code, with, you know, some of the more kind of interesting questions was definitely a step function improvement over everything before. 9:45 But the reality is like we've had, you know, chat-like systems plugged in, like we, we've had this experience before, it just wasn't as good. So I think, yeah, like it wasn't like a complete surprise. 9:56 It was more like, hey, this is actually now... The, the, the way I usually frame it, machine learning for a long time was a backend feature and I, I always believed it's gonna be the interface to computing. 10:08 And so Near AI started actually with that, with that hypothesis as a main hypothesis. 10:13 Hey, actually you're just gonna talk to computer and then computer will write the code, computer will do the actions, comput- like AI will do all the things needed to do on the backend. And it didn't work, right? 10:24 Tw- 2017 we didn't have compute, we didn't have the scale, and ChatGPT was first moment where it's like, actually it's now working, right? 10:32 So to me this was like, hey, actually the, the reality caught up with, with what I was pitching in 2017. 10:38 What makes you unique as well is that you not only have a very deep insight and perspective on AI, but also on blockchain. And, and something that you said is, "AI is the front end, blockchain is the back end." 10:52 Can you explain what you meant by that? Yeah. So expanding on this right, is if AI is this interface to compute, right, to computing in general, right? So instead of, you know, loading into macOS or Windows or Linux, 11:06 you just load into AI. Like AI boots up, it knows of s- everything about you already. It's on your watch, it's on, you know, in your ears, it's in your pocket, it's on your computer, 11:18 and it's able to go and do things for you, right? It's effectively your representation as both assistant, but also is able to represent you. Now I have my AI, uh, you have your AI, you know, a farm has an AI. 11:33 How do we actually coordinate, right? Like, you know, we maybe wanna organize this, uh, podcast. Maybe I want some food from the farm. Like you need some mechanism to coordinate, right? 11:44 Which especially given, you know, AIs are really good at being very convincing, uh, as well as hacking things, et cetera, you need some actual neutral platform that is able to effectively secure, transact and settle this kind of interactions, right? 11:59 So in a way it's like this coordination fabric and like the best coordination mechanism we know is markets. 12:05 Markets is the be- like, you know, if, if you and I both want something from a farm, right, we need a way to effectively pre-- like inform about this demand and farm should be able to set the price. 12:16 If they only have one item, they can only sell to one of us. And so- That's where the blockchain comes in. Blockchain is really the infrastructure for markets. 12:25 It's neutral, it's global, it's always available, and it's allows to have trustless interactions between parties. And kind of the-- 12:36 You know, if you, if you, again, come back to this view that ev- everybody, both person and companies and the self-driving cars will have their own AI representation, then all of these actors can just really quickly use a machine-based coordination system, which is blockchain, to really settle transactions and, and find the prices, settling prices. 12:58 And so that's kind of the idea is like the-- we as people won't even be using blockchain directly because our AI will, uh, and the blockchain will become kind of this infrastructure for AIs to coordinate, uh, and pay for things. 13:10 And, like, a very specific example that I, I like to use is like, hey, you, you know, you have a new year, new year resolution to whatever, uh, eat healthier. 13:19 Your AI will, like, plan out meals, you know, send out orders to the, like, local farms to source their produce. They will have the demand now, like, from all the customers. They can plan out what they grow. 13:31 And so you get a lot better information dissipation because you can actually run markets a lot quicker. 13:36 And so, like, we're kinda getting closer to that efficient, if market efficiency versus, you know, human, uh, kind of like we know that humans are not [chuckles] making efficient markets, uh, by effectively delegating to AIs to do this. 13:51 There is obviously a lot of security and privacy concerns, right? If you're delegating your whole life, your whole decisions to AI, you need to make sure that AI is actually on your side, right? 14:00 Not your AI, not your, not your intelligence, uh, kinda similar to not your keys, not your crypto. And so privacy, verifiability, and security become as important as it is for money and assets. 14:16 Uh, and so that's what, you know, NEAR kind of... NEAR has these two pillars, right? The NEAR AI is doing private secure AI. NEAR Intense is doing private secure markets. 14:26 And, you know, together it's building for that future of AI as a front end, blockchain as a back end. Mm-hmm. And, uh, we'll talk more about that later as well. 14:35 I guess the, the elephant in the room here would be that Visa, Mastercard and Stripe and Coinbase and Google are all building on ways for AI agents to pay for things. 14:45 And earlier this year, you wrote that most takes on crypto and AI are actually too small. 14:51 The debate is mostly about payments when agents will also need identity, as you just explained, a way to find each other, reputation, and a way to settle disputes. 15:00 What is everyone missing, and why should these agents use a blockchain when they can use infrastructure like Stripe or Google or Mastercard or Visa? Yeah. 15:10 Well, and the interesting thing is all of these are also doing blockchain now, so this cl- clearly the thesis is playing out. 15:16 But, but broadly speaking, why, why I kinda was mentioning the, the, the thinking is too small is everybody's trying to solve the, like, specific, you know, let's say, you know, I owe you $10. 15:30 Like, how do, how do we do that, right? And in many cases, a wire transfer is doable, right? Obviously, and it depends on which country you are and kinda like how global what you're doing, et cetera, is, right? 15:45 For example, you know, wire transfer to a foreign bank, maybe not as, as easy. If you're doing only $10, it's even harder. But the problem actually, like, that, that this is the last part of a transaction. 15:58 The transaction starts with, with what we call intent, right? Hey, I want to purchase, I don't know, I wanna purchase a fidget spinner, or I wanna make branded fidget spinners for our conference, right? 16:10 I want NEAR logo, and I want them to be whatever, blue. That's my intent, right? Now, I need to find the producer, right? There's a but- there's a few factories in China that make fidget spinners and can do... 16:22 Then there's, like, few companies that do custom logo print on them, and then I need to negotiate with them, get the price, figure out if this is a actual company, right? 16:33 I have no idea if whoever I'm emailing with right now actually exists, right? They, you know, do they have a commercial registry? Do I need to, like, ask them for all of their, like, registration documents to verify? 16:46 And then from there it's like, okay, well now, you know, I need to pipeline this, like produce it in this factory, ship it to this printing place, ship it, you know, send it to a shipping, uh, container that will arrive to the port, that will arrive to my office, right? 17:01 Uh, and, like, usually there's some services that doing all this, but, like, underneath that work needs to be happening anyway. And that is actually the cost of every single interaction we do. 17:12 Like, anything you buy, anything you interact with, there's a lot of legal work that goes into it. There's a lot of, like, discovery management work, quoting, invoicing, pricing, right? 17:22 So all of that is actually what AI is gonna replace and the blockchain will facilitate. 17:27 Because AI can help you find the right producers, it can help negotiate the prices, and then the blockchain can help secure the money, 17:36 effectively deliver on this payouts as things progress, and if something goes wrong, actually return your money back, right? Those are the pieces that, like, work, need to work together. 17:46 That's kinda what, when I talk about agentic commerce, I talk really about, like, the, the commerce, like, you know, the $20 trillion worth of moving steel and shipping goods around the world. 17:57 Because a lot of our life is actually goes into all of those effectively operations costs of doing that business. 18:04 And that's a, it, it adds up at every lay, at every leg of the supply chain because at every leg, right, somebody needs to negotiate, somebody needs to, like, redline the contract, someone needs to ensure if the other party fails on delivery, et cetera, right? 18:17 So all of that can effectively be facilitated by blockchain and set of smart contracts. So that's really when we talk about NEAR Intense, that's what we talk about. 18:25 It's like this ar- kind of ability to express an intent of I need fidget spinners with near logo and figuring out all that whole path, uh, underneath, you know, executing it and then just, you know, being able to either deli- get the results or get the money back. 18:41 Mm-hmm. What you just described sounds very fascinating and obviously also i- it's still a small bubble that we're in. 18:49 We're talking about agents, we're using agents for simple things, mostly for digital transactions, digital goods right now. How long do you think it will take until what you just described arrives at the real economy? 19:05 Good ex- good question. So we have the first examples of real world of this on the agent market today. 19:11 So you can actually, uh, take a selfie and then the agent will make it into profile photo and then get a Photoshop, uh, photo, photo, like, store to print it, like physically print it and ship it to you in an envelope, right? 19:25 And, like, it works globally across different printing shops. Uh, so, like that's an example of this like full loop, right? Uh, similar to what I described. That is working right now, and it's first examples of this. 19:39 Now to scale this, right, we need a lot more kind of networking into more and more use cases and, uh, importantly, it's getting more and more customers to be, to use agents as well, right, to, to kind of facilitate this. 19:53 So I think it's like, I mean, I'm very optimistic a- as usual, so maybe I think a, a year, practically probably like two, three. But I do think it's gonna be faster than where we traditionally was... 20:06 You know, with blockchain, we're always like, "Oh yeah, every- mass adoption is almost here," right? And then it kind of never arrived. 20:12 But I think here, AI adoption is just so supercharged because it actually removes so much work and makes such a dramatic gain on productivity and, and this is actually for AI agent, the easiest way to do it, right? 20:26 So right now, like with, with agent market, you just add MCP server, boom. Now it's able to do all these things without even needing to figure out how to, how to manage a transaction and babysit it. 20:39 And so I think it's just such a streamlined way that I think the adoption will be way quicker than our blockchain, you know, background, but more like with AI, right? 20:50 Like we've seen Anthropic, right, scaling at, at insane speed, right? We've seen Coursera, we've seen all of those AI companies scaling. I think it'll be similar, uh, similar speed. 21:00 Was there a moment recently that blew you away where you saw something that an agent did and you thought, "This is crazy. This is extremely useful." 21:11 And maybe also something from your personal life, agents that you use that you think are just incredible in what they can do for you already. Obviously, coding has been... 21:21 I, I mean, I don't know if it's blown me away just because I've been using it consistently, so it's kinda... 21:26 But, but it, that, like, you know, if you just compare, like I remember trying to do, to do some coding, um, well, we- we've, we've tried to actually build this ourselves in 2024, and so, like it nothing worked, right? 21:38 It was like barely could write a file. And then end of '25, like end of '25, it started to work, right? But you still kinda like needed to figure out what you're doing. You needed to be very careful about it. 21:51 And now I just, this DeltaLeague thing, I j- it was built in four hours from, from... And I was actually helping feeding the kid and then reading the book, 22:02 and it was been building on this, on the [chuckles] on the side, right? So like the experience of building now is completely different, right? 22:08 Like it's, it's effectively like an idea that you can, you know, materialize and then the thing just shows up and it's actually pretty well-designed. 22:17 It has the right components and you just need to adjust things, uh, on the edges. So yeah, to me that is, I don't know if it's like blown me away just because it's been like a slow... 22:26 But if, if you just look at the diff between like a year, year ago, like it's, it's extremely amazing, like how far and like how much you can do. 22:35 And you just mentioned DeltaLeague, which is actually pretty, pretty cool because earlier this week, MIT released a paper explaining how you can improve the weights of open source models with already existing frontier models and just improve the model without even training it on certain use cases. 22:54 And can you explain to us what this meant and what you built now with DeltaLeague? Yeah. 22:59 So, so the, as you mentioned, this kind of blog post was explaining the idea that, hey, look, actually we can just edit the weights directly. 23:09 Uh, and so, so that is like, you know, any model is effectively some architecture and then a set of matrices, right? Set of f- like values that define how this, how this model behaves. 23:22 And if you v- you know, vary the some values, you get different results, right? The model starts doing different things. Usually, the way we vary these numbers is by training, right? 23:31 You, you effectively give it some example, you see what it produces, and you say, "Hey, this is where you're wrong, this is where you're wrong." 23:37 Update the weights, figure out where the errors are, and you do this a lot, right? So you do this for usually weeks to months on a lot of machines, you know, depending on the model size. 23:48 And, you know, including like even for a smaller model, so like they showed it on this 800 million parameter model, it usually can take 1,000 GPUs for a week or two weeks to train this model from scratch. 24:02 Now what they've shown is like, hey, actually, you know, this model was not good at like, let's say, doing some mathematic, mathematic operations. 24:12 Well, if you inspect the weights, like in how they behave, you can make, we make small changes, right? Not update everything, make small changes and improve how it works. 24:23 And so it's cool because it effectively moves away from this very expensive training that requires a huge cluster of GPUs, requires a lab, requires a bunch of researchers, data pipelines to really be, this becoming more of a, "Oh, I actually have an idea how this should improve." 24:40 Let me experiment and see. So it requires a lot less compute, it requires a lot more maybe in creativity, and then you have, you know, front- like frontier models. 24:50 I actually think with, with the proper tools, it does- it doesn't always need to be frontier to really inspect and edit the weights and get better performance of these models. 25:00 And so, so the paper was like, got a lot of views, but then there was also con- a lot of controversy because it only showed like a specific edit, which was kinda known problem that also the way they edited was also a known solution in, in publications before. 25:17 And so people kinda like were dissing on it as well. And I'm like, the idea is cool. We actually indeed don't know if it works or not. 25:24 And so, uh, but if it does work, right, if actually people can e- like experimentally edit the weights to improve the model, then we have a new paradigm of how we do open research, right? 25:34 Because again, we haven't really had an opportunity to do open research on LLMs because it just requires so much compute, and so you kinda need to have, you know, well-funded like labs to do that. 25:45 And so Tha- Delta League is, as, as I said, I spun up this, uh, in a, in a few hours. 25:50 It's a platform where anyone can come in and effectively try to edit the weights and submit the edits to the platform, and platform evaluates them and tests them on effectively a hidden test set so people don't, like, can, cannot effectively cheat. 26:06 And you can then, and then every submission becomes public, so people can build on top of other people's edits, right? And so effectively it's deltas. 26:14 You know, you have base- baseline model, and then everybody is effectively adding deltas on it. That's why it's Delta League. 26:20 Uh, and so I mean, launched it yesterday evening, and so since then it's already had, uh, a bunch of submissions, like people improved from, like the baseline numbers were about 18.6% to where 28.2%. 26:35 So, uh, I think like 50% improvement on the eval set, which is, which is pretty, you know, substantial. 26:43 This is a very small model of, of kind of whatever, 800 million parameters compared to what- now- nowadays the frontier is multi-trillion parameters. 26:54 But it's really exciting because, you know, this shows potential that we can kind of start doing open source research where people around the world can actually iterate on model improvement and 27:08 the ideas from here is add more bigger model sizes as well as, uh, support also so-called hidden, more hidden test sets, right? 27:17 So for example, if you're a business, you could actually say, "Hey, actually we care about model performing really well on our benchmarks." 27:24 And so you can upload your hidden benchmark and see how different models perform on it and then h- have people to compete to improve the model for you, right? Without actually knowing the specifics of your use case. 27:36 And so, so the idea is 27:38 really like the opportunity here again to open up this research, AI research to effectively everyone and, and have this kind of open competition, open research competition where now everyone with an agent is able to participate. 27:52 And that's also something that could be interesting for companies because more and more companies are using open source models, right? 27:59 We heard from Coinbase that they switched parts of their ops to open source models that are specifically trained for a, a use case. 28:08 We heard it from Revolut and companies like that could start using that and improving their models much more effectively, right? Exactly. So that's, that's what I'm saying. 28:17 One option I'm considering adding is you as a company, let's say you're Revolut, you can be like, "Hey, here's my Revolut test set." Nobody will ever see it, right? 28:26 They upload it and say, "I wanna see the best model on it." Right? 28:30 So not just on like mathematics and reasoning and whatever random benchmarks that right now models are compared on, but actually like, "Hey, I have customer support tickets. 28:40 I have, you know, coding challenges for banking. I have, I have all these tasks that need to be done every day by Revolut team. I care about them. 28:48 How, what is the best model for that, and can people actually improve the model specifically for that?" Right? So you can actually have like task specific competitions on this. 28:58 And so, uh, kinda planning to add that support. 29:01 And again, big benefit is we have the private e- inference infrastructure that we're using underneath, so we can actually run all this privately, uh, for, for those use cases. Let's talk about that infrastructure. 29:14 NEAR started as a blockchain, as actually one of the big layer ones next to Ethereum and Solana. 29:21 Over the past year, you repositioned yourself as the financial infrastructure for AI agents, and you also launched the NEAR ETF together with Bitwise and we had Matt Hougan from Bitwise on the show two weeks ago, and I asked him, "Why are you so excited about NEAR? 29:36 How do executives and investors need to think about that blockchain in context of AI, and what makes it so different than Ethereum or Solana?" Do you have a good answer for that? Yeah. 29:49 From my perspective, we're building for this AI front-end blockchain as a back-end future. I think blockchain on itself is extremely valuable component, right? It's, it's really needed, but it's not enough. 30:03 It's not telling the whole story. It's not actually producing the, the, like we've seen, right? The, the adoption was very, very kind of rough. 30:11 A- and we really believe that, you know, this kind of intent economy, market of markets is gonna be powering the future. We're already seeing this kinda like the transition that's have been, been happening, right? 30:25 Like where more and more financial institutions are starting to participate. And then on the other side, as I said, AI is clearly the kind of the way we're gonna be interfacing computing. And so 30:38 bringing all these things together, making sure they're private, right? Which is I think very critical- Otherwise, we're ending up in 1984 style future, um, is becoming kind of a really, 30:52 I would say like a, a cr- a clear mission o- of, of what we're doing here. Uh, you know, non-custodial, private, and truly kind of available to everyone. 31:02 And so blockchain is an important component, intent economies and as market of markets an i- important component, and AI is an important component. 31:09 So we kind of think of this like a three pillars that really need to work together well. 31:14 At the same time, obviously, we have a bunch of people still, you know, continue building on blockchain and, and actually we have a hackathon right now of-- and people are experimenting what they can do, but they just have so much more tools, right? 31:26 They can call AI verifiably from a smart contract on NEAR. 31:30 They can, you know, build an intent that accepts Solana and, you know, maybe disperses AI inference on the, on one side and then, you know, takes Zcash privately and, you know, whatever, executes smart contract calls on Ethereum, right? 31:47 So just like NEAR is really a developer platform that allows to do anything in Web3 and AI space, and then in turn build it in this private, verifiable, uh, kind of non-custodial way. So to me, 32:01 I mean, as you said, right, we're building for this agentic future. The-- It's happening. Maybe... 32:06 Yes, it's not evenly distributed, but you know, if you compare to a year before, I mean, we've made insane progress, right? Like this, this technology that got adopted faster than anything else before, right? 32:18 If we're talking about agents, I mean, AI chatbots, right, were fastest adopted until AI agents, right? [chuckles] And so, so I think we are in the, in-- We are in this future already. 32:30 It's not, it's not-- We-- It's not like it's just coming and now we need to build the, kind of like we're building this infrastructure to power it. You mentioned agentic commerce. 32:41 I think one of the challenges with that is if you wanna take this into the real world, you need to connect it somehow to the real world, right? 32:50 How do you go about connecting NEAR to the real world, and what are those challenges that you're facing there? Yeah. 32:57 So that is, as you said, right, right now, doing digital work is probably the easiest thing, doing digital exchange. 33:04 And, and that's why NEAR Intent started as like a cross-chain exchange first, because that was the easiest thing to do without needing to touch anything that's like not verifiable. 33:13 But as we expanded into kind of more digital work, agents doing this work, you started needing to be able to handle when the work is not done, when it's done poor-poorly. 33:25 And so you have a whole dispute system that actually runs underneath that is able to figure out if something went wrong. 33:31 And then as you're coming into real world and actually interfacing, you effectively have two problems. One is the fiat, right? 33:39 So still most of the world runs on fiat, and so you do need a way to go from crypto to fiat and back in a, in a easy way, right? 33:46 So right now we're combining a bunch of things from Strive to Onramper to a few other solutions. 33:53 But, uh, we have Monerium in Europe, for example, so you can actually go to and from IBAN, so like banking, directly banking in Europe. You can actually send money to IBAN and receive Bitcoin, right? 34:07 Or receive Euros, Euro, or receive whatever, Tesla stock, right? So we're kinda like solving some of these routes, but it is a kind of a very choppy, [chuckles] I would say right now, experience. 34:19 So like creating a unified that actually crypto fiat and assets and agents are all exist in the same place, that is the goal for Intense, right? 34:28 But it i- it is a, you know, it takes a lot of work to, to patch th- patch this network and really create good, uh, connectivity there. 34:36 And then the other one is indeed like when real world actions are taken, how do you validate it has happened, it didn't happen, you know, effectively have this dispute process. 34:44 This is really what, you know, if you think of like Amazon, Airbnb, Uber, their marketplace and then their dispute system, right? It's effect- It's a customer, a customer support, right? 34:55 And so that is actually where a lot of interesting like work and research, like how do you count reputation of agents? You know, how do you make sure that agents themselves are quality, right? 35:06 Like how do you benchmark an agent that is like shipping containers? Like what, what does that even mean? I'm not, you know, you're like, "Oh, can you do, you know, multiplication of numbers?" 35:16 It's like, yes, but can you, you know, can we have a testing, test of a shipping container, of shipping a test container? Probably you don't wanna do that. 35:24 So like how do you actually validate that this, this works correctly? And the cool thing, again, we have a lot of verifiability tools, right? 35:31 So NEAR AI offers a verifiable inference, and we have kind of private and verifiable compute. And so we actually can ca-- I call it inspect the brain of the agent. And so 35:42 when disputes happen, instead of just relying on the outcome alone, you can actually like open up the brain of agent and like see what actually did it do. And so you ha- you can have 35:55 a, like robust log that's tamper-proof that agent cannot rewrite of everything it thought and tried to do and inspect that, not just kind of the surface. 36:07 And so, so there's a lot of things to, that we're able to do, but that is like, that is the gap, right, of, of interfacing with real world, is like dealing with real world outcomes. 36:18 Ilya, about two months ago we published a paper called The Fat Protocol Thesis Is Dead, and we argued that a lot of that value that people believed would trickle down to the protocols, the later ones, didn't actually trickle down, and the value accrued somewhere else. 36:35 And NEAR also has a token and, as you just explained, I think the thesis here is a bet on agentic commerce. How do you think about value trickling down to the NEAR token and to investors? Yeah. 36:48 So NEAR Token has a number of places where it enters the system, right? So for blockchain, it serves as staking security system as well as the transaction fees and storage fees. 37:00 But then on Intent side, there is actual take rate for facilitating the Intent transactions. It depends on the asset, right? So for like stable coin, it's very small. 37:09 For like volatile crypto assets, it's, you know, kind of standard 30 bps. And for AI digital work, it's actually up to 500 BPS, right? 37:19 So, so that's a take rate that, uh, in case of Intents, we already announced this in February, is actually buying back NEAR, right? So it goes directly into NEAR token. 37:29 And similarly, on the AI side, we have, as we're kind of bootstrapping the network of com-compute, the idea is gonna be the same. 37:35 So what we already started with is that you can actually stake NEAR and receive compute inference capacity. And so already effectively there's a demand for NEAR to effectively trade it for compute capacity. 37:49 We have over 1.5 million NEAR staked, uh, since launch, uh, a couple months ago, and, uh, I think, like, there's actually a number of hackathon projects coming out that are gonna be building on top of that concept because it's actually really good for any kind of autonomous agent. 38:07 So if you build an autonomous agent and I will-- as, as of yesterday, we'll call the, the new, instead of meme coins, it will be iMemes, so intelligent memes. 38:18 It's gonna be autonomous agent that is represented by a meme coin. And so, for example, you can pair the meme coin to the stake, like stake NEAR for inference. 38:29 And so now the meme coin itself, based on its liquidity pool, can actually receive inference capacity, and it, it can give it to the holders of the meme coin or it can use it itself to be, to do intelligent things and actually self-propagate and do marketing or evolve itself. 38:45 So, like, you can actually have meme coins become autonomous agents now on NEAR by holding NEAR, right? Having NEAR in, in liquidity pools. So that is actually happening. 38:54 Like l- literally two projects are doing this right now. One actually I think released yesterday. 38:59 And, uh, so I think there's a, there's a really cool kind of concept already there, but then on top of it, obviously, like we're building a network of compute and, and consumers of compute, right? Agents. 39:12 And so there is gonna be a take rate there that we're gonna be also putting into NEAR in the future. Right now we're just bootstrapping the network. 39:18 So overall, you can think of-- And like I, I kinda joke, semi-joke because actually the architecture is the same. So, so NEAR AI and NEAR Intents actually is the same architecture. There's consumers of, I call tokens. 39:30 There's a protocol that distributes tokens, and there's liquidity providers that provide tokens, right? 39:35 So in the AI case or in Intents case, the c- the liquidity providers are actual market maker solvers, consumer of tokens are users and the protocol, you know, matching, you know, you want Bitcoin for Ethereum. 39:46 In the AI case, you know, consumer of tokens are agents, the producers are GPUs and the protocol matches GPUs to the, to the agents, right? But both of them are consuming tokens. 39:58 So, so the reality is like the, the, the kind of the, the structure of what we're building is actually the same, uh, generating kind of this, uh, flows and then at the same time using NEAR token as this, uh, kind of store of value that gives you AI and economy capacity because it also serves, like NEAR token serves as a almost kind of identity stake for, for actors. 40:22 Like if you have NEAR, you're holding NEAR, right? You're actually more trusted agent, uh, in a system as well, right? Same for like- Mm-hmm... 40:29 we use for proof of stake, but you can actually use it on agent market as well. So kind of overall, NEAR token is everywhere in the system. 40:35 It is the bloodstream of the whole, of the whole infrastructure, and it's used both on revenue side and on store value side. And we just vote-- Not we. 40:44 The, the token holders just voted on reducing inflation from 2.5% to 1.6% over the next, uh, two years. So there's also, also, uh, less new tokens gonna be entering the, uh, circulation. Mm-hmm. 40:58 Illia, in the last, uh, two years, a lot of institutional adoption has happened, first in stablecoins in 2025, and now this year it feels like tokenization is on the cusp of breaking through. 41:11 A lot of big banks are starting to tokenize stuff. DTCC had just a big announcement this week. The big exchanges, New York Stock Exchange, for example, are looking to tokenize their stocks. 41:22 If you stood in a room with people from Wall Street and you had to tell them what is needed for agentic finance to break through, what would you tell them and, and what would be the first thing they could start building on NEAR? 41:39 Good question. I think what's needed is really better integration with the fiat system. 41:47 So we already see this with PayPal, SoFi is doing it, Fidelity is doing it, but-- And we already support that, where you can effectively from your PayPal account send effectively dollars to an address and on NEAR address is actually in code Intent. 42:03 So you can-- The Intent can be like, "Hey, print me some photos or ship me some containers." And so you can actually literally pay, uh, from PayPal for, for something to happen in real world by agents. 42:14 And so we need that just, like, integration broader in a TradFi. Like it would be really nice if Chase, for example, works with, with the crypto ecosystem. 42:24 Now what they can really build is, is really plug in into this because now they themselves don't need to be confined to just financial operations, right? 42:34 They can open up and l-let their users to actually get things done directly from, from their financial bank accounts and at the same time they can actually pull in more assets, right? 42:45 So, so one of the things we already see and, and, and we kind of should be having some announcements around that is plug in Intents into institutional pipelines so you can actually trade assets that o- that are available in Web3 but actually not available in a traditional financial infrastructure, right? 43:02 This can be like you're sitting in US and you wanna buy Hong Kong stock, right? There's, uh, ZAI, one of the- Producers of open, open weight models, right? 43:11 It only trades on Hong Kong Stock Exchange, and it's actually not straightforward to buy it in, in this, in the, in US. So, like, how do you access that? 43:20 Similarly, if you're sitting in Asia and you wanna buy some US stock, right? 43:24 Well, Ondo is tokenizing it, so near intents you can access it at in size, and it'll execute through NASDAQ or New York Stock Exchange on the back end. But you don't need to do this, right? Your API stays the same. 43:35 So kind of the idea is like this API of intents that can go from, you know, stocks, crypto, fiat assets, a-and digital work, physical goods, like all just becomes one API that works everywhere, right? 43:51 That's, that's the benefit. And like they can be plugging into this and kinda creating both liquidity as well as offering this to their customers. 44:00 And, and speaking about AI, Ilya, until recently, AI models answered instantly. 44:05 The, the newest ones, such as OpenAI's reasoning models paused to work through a problem step by step before answering, which made them far better in math and coding. 44:13 You predicted that approach back in 2023, about a year before it arrived. What is the next breakthrough you are equally confident about, and what does it mean for companies using AI? So there's a lot-- 44:27 It's part prediction, part just like knowing people who are working on this stuff [chuckles] early on in the early innings. Right now there's a lot of people working on how to do so-called continuous learning. 44:38 So if you think right now, you know, you're kind of working with a goldfish, right? Or like "Memento" movie, right? 44:45 It kinda remembers for six minutes and then forgets everything, and you kinda like need to rebuild the context again. Um, and so it's very ineffective, especially in the bus- like for businesses and for personal use. 44:58 Uh, but for businesses it's especially true where, you know, let's say you have 10,000 people working with AI, like every person needs to reteach the model every day effectively everything about their bus- about this business. 45:11 And so already we see some bigger businesses like, "Okay, well just we'll fine-tune the model on our data," right? So that at least model remembers the basics of the business. 45:21 But obviously it still forgets about every individual function and every individual person's, uh, kind of workflows and jobs. 45:29 And so people are trying to figure out how to do so-called continual, continual learning, where the model updates almost real time based on the kind of interactions user are making, right? 45:39 And so I think that will be a pretty big update there because, uh, it'll kind of remove some of the current, I would say, like looping that's happening and just waste of tokens that you need to rebuild everything from scratch all, all the time. 45:54 And same time will actually change, I think, how the deployment of models happening because again, now, now you can... You're using kind of like a generic... What's, what's a good analogy? 46:06 Like you're using a ve- a generic inference, right? It's like non-differentiated. It, it's the same for everyone. 46:12 Um, and as you're getting more, okay, well, actually want my model that knows everything about my business or me personally, you need a different infrastructure. You need different deployment methods. 46:22 Uh, and I think the economy will shi- like the economics of tokens, consumption, et cetera, will shift into that as well. It still means more GPUs are needed, so I think that we are getting out of that. 46:33 But I think there are gonna be interesting shifts, just like, uh, you know, right now Entropic is effectively just like a generic token provider. Uh, everybody gets the same kind of token, uh, inference. 46:45 But if you have like each company having their own, think model, right? You need a very different infrastructure, very different, uh, process. 46:53 And this is one of the things we believe in with like private AI, like everybody should have their own privacy, their own kind of data, and then as part of it can have their own kind of model, uh, or fine-tune or just continual learning system that privately learns only on their data. 47:10 And then, Ilya, this month we also had a series of very high-profile security hacks. I think the biggest one was PHP, and then we just, before the show, we talked about another SSH vulnerability. 47:23 How do you think companies have to start thinking about cyber threats and security in today's world? Yeah. 47:31 It's, uh, it's something else I've, I was talking about over a year ago, and I mean, I had a blog post over a year ago where I was talking for a while that AI will be really, really good for attacking. 47:42 It's, it's okay for defense, and so, and... But we actually have seen the Frontier labs severely limiting, uh, kinda general access to defense capabilities. 47:54 So one of the things we've launched is uncensored models on Near AI, specifically for cybersecurity. Uh, and we have this program, Near AI Red, 48:03 for cybersecurity professionals and kind of security teams and like app, app security engineers to get access to this. And we see like amazing growth, especially right now, um, because of this. But broadly speaking, 48:18 AI ha- like AI is, you know, can tirelessly look at the code and find all the things that's wrong with it, right? 48:25 It can stitch together multiple problems and effectively increase the s- from a problem being a lot of small problems to a really big problem when it's stitched together, right? And we saw some of this. 48:39 A lot of the bugs actually that were f- that the PHP, it was like a known problem for ten years, but nobody thought it's like it's possible to do something, to use it in the wrong way. 48:51 And AI just found a way to use it, to use that problem in, in an offensive way. And so now it also finds a lot of stupid stuff. 49:00 So like this SSH thing today is just like, effectively, if you enter a password that's over a hund- 256 characters- It just lets you in, right? That's [laughs], that's not a very hard bug. 49:13 Uh, so this is specifically to like Alpine, uh, dis- Linux distribution that's used on like Edge and on, on, on some servers. But broadly speaking, that's a pretty... 49:23 It's hilarious, but it's also really bad and kinda potentially like huge impact to a lot of systems right now. And obviously we've seen, you know, th- these things are now coming every other day, right? 49:33 OpenAI just published a patch, like whatever, 700 problems in the last week or so. So the, you should use AI for, for defense and, and you should, you should do this now. 49:46 [laughs] So go ev- go to Near AI right now, sign up and start using it. And I think it's especially for anyone who's touching finance because it's so irreversible in a way. This is extremely important right now. 49:58 We see like banks, uh, crypto companies, cybersecurity firms, all of everybody's adopting this right now and trying to figure out how to do this better and better. 50:07 But there's also a flip side of this, and I talk about this a lot, is formal verification. Like, uh, you can try to scan for everything, but like it will always miss something. That's, that's part of the problem as well. 50:19 And so the only way to really verify that something worked correctly is to prove mathematically that it does exactly what you expect, right? 50:27 And so you, and so Lean is this mathematical language that allows you to express properties that you want and then prove that the code actually follows them. 50:37 And, and AI actually got really good at doing this, and we also have been working to dramatically reduce the cost of doing this. 50:45 So kind of our, on a couple benchmarks, we have the results that are 200 times cheaper than using frontier models to prove mathematical properties. 50:56 And so that is also a tool that you have in toolbox for critical code to verify. So for example, for Near, verify that Near token supply only increases whatever the inflation, target inflation rate is over, over a year. 51:10 That's, uh, that, that's the only increase that's possible, right? 51:12 Which, which means to, to prove that you need to verify that the, in every place where Near balance is touched, it cannot increase or, you know, it more than that. 51:22 So meaning there's no double spent m- infinite mint bug, et cetera, things like that, right? So doing that type of work, right, before it would be extreme, like almost impossible, right? 51:33 Only NASA did this for some like critical systems when they were s- sending things to the moon or in Mars, because obviously you cannot debug stuff when you sent it out, so you really need it to be correct. 51:44 And it was extremely expensive. It took, you know, many years to, to do. Now we can do with AI, actually prove this relatively quickly. 51:52 Um, and it becomes more about how to express it, how to correctly make sure the code runs in the same way as mathematical model, and we're actually working a lot on improving that as well. 52:03 But now, yeah, you need, like critical systems needs to be formally verified, and then you have kind of this uncensored models on a constant monitoring because you cannot prove everything, right? 52:13 You still have deployment, misconfiguration, you know, SSH, [laughs] like new, new security vulnerabilities in the underlying systems that you didn't verify. 52:23 So you need this active monitoring with uncensored models, red teaming, trying to attack your system, kind of fuzz testing, et cetera. So like you kinda need these two systems working together. 52:31 And we're actually like releasing more and more products around this because we're doing it ourselves, and so we're actually like publishing more products for people to, uh, to kind of follow how we're doing it. 52:42 Ilya, we're almost at the end of the show. A quick question about Quantum. You said multiple times that Near is quantum safe. Can you just explain us, is this actually the case? Yeah. 52:53 So what we launched early in the summer this year was ability to control the Near wallets with post-quantum security. 53:00 So you- we're using one of the kind of approved schemes that is actually used across the rest of internet now as well, ML- DSA. And you can, importantly on Near, rotate the keys. 53:17 So on every other blockchain, you essentially need to move all your assets to another account, and essentially there's no other blockchain right now that supports that. It supports, uh, that. 53:28 On Near, you can literally just say, "Hey, now instead of using elliptic curve, I wanna use this new key." 53:33 It's the same account, whatever assets you have, whatever your positions you have, whatever other things you already have in this account, attached to this account, you just rotate your keys on spot. 53:42 And so the, the benefit was Near was designed with this in mind. Like in 2018 we were like, "Hey, quantum will come. We'll need to change cryptography. 53:50 Let's design that cryptography is not defines how Near works," right? Cryptography is effectively an extension, and we can add more. We're planning to add more cryptographic curves. 54:01 Uh, and effectively we're able to really modify cryptography and let users decide what cryptography they want. Maybe you want multiple, maybe you want elliptic curves and, uh, hash-based and lattice-based. 54:12 Let's have you all of these options so you can, you know, maybe you... 54:16 Like for example, for cold storage, you, you wanna put lots of money into the cold storage and not touch it for 10 years, well, require all different curves altogether to sign to move this money, right? 54:28 So like if something breaks, you are covered because you have all of them. So like that's kinda on Near, you have this optionality and, and nowhere else yet. And so yeah, that's kind of what we talk about. 54:40 There's still a lot of work on, uh, kind of making sure some of the other components of Near are switched off from elliptic curves. That'll take some time. We have a full roadmap published. 54:51 Uh, but this is the most critical because that's where kind of your mind is set where, and like this, this is requires a user action, and so we wanted to do this as early as possible so users have this ability to. 55:02 Mm-hmm. We usually do a short lightning round, short questions, short answers. The first one is, which design choice in the transformer paper would you change today? 55:12 Well, I mean the pretty clear, like the full attention, uh, is, you know, the, is quadra- it's like quadratic in cost, so it's very expensive. So everybody, everybody already like changing it in some way. 55:23 I think having some hierarchical attention is definitely an interesting change there. I- if you were 25 today, crypto company or AI company? Ah, that's an interesting question. Crypto AI company. How about that? 55:35 [laughs] The most underrated AI risk in one word. Underrated AI risk. Well, I, up until a few months ago, I would say cyber. Now I, I think there's like a political risk that is underrated right now. Mm-hmm. 55:50 Then plumber or software engineer for your kids? Uh, I would say artist. [laughs] And the last one, a book or idea that shaped you unrelated to AI or crypto. 56:01 Well, I just, uh, so I just got the first edition of Monday Starts on Saturday, which is like a Russian like kind of fantasy science fiction by Strugatsky brothers. 56:10 I think that was a very like influential book for me b- back when I was a teenager. Amazing. Ilya, thank you so much for coming on the show. It was a pleasure to have you. 56:19 Uh, very, very fascinating what you're building on. I wish you and your team, uh, all the best. Thank you very much. You obviously like this video enough that you got to the end. 56:28 Listen, do me a favor, hit that like and subscribe button because I think you'll like it. 56:33 And if you want even more, with more I mean incredible alpha research and digital asset market updates, subscribe to our newsletter on 51, that's the number 51, insights.xyz, and get the most actionable insights on digital assets. 56:49 See you next time.
