This forum will explore how artificial intelligence is reshaping global decision making and diplomatic strategy. What are the opportunities—and risks—AI presents as governments increasingly rely on data-driven tools?
KRISTIN CAULFIELD: And I now turn this event over to Professor Erez Manela, acting director of the Weatherhead Center.
EREZ MANELA: Thank you very much, Kristin. Welcome to the Weatherhead Forum, our platform to address pressing topics of the day. My name is Erez Manela. I'm the acting director of the Weatherhead Center for International Affairs. Our pressing topic of today is the role of AI in foreign policy.
What we want to do is explore how artificial intelligence is reshaping global decision making and diplomatic strategy. Our experts will discuss the opportunities and the risks-- and I guess putting opportunities and risks in that order is the optimistic way of looking at it, which is what we're hoping to do-- the opportunities and risks AI presents as governments increasingly rely on data-driven tools.
A quick note with regard to our format-- each speaker will present briefly. Then I may pose a question or two to the group. And then we will take your questions, the audience, from the Q&A feature in Zoom. Please draft your questions there. I will try and get to as many as I can, and I will combine questions that are similar if necessary.
So today, we are honored to be joined by four esteemed guests. I will introduce all of them now, and then they will speak in the order in which they're introduced. Our first guest and speaker is Marc Aidinoff. He is an assistant professor in the Department of History of Science at Harvard University. He researches the interplay between digital technologies and public policy in the United States.
He served previously as chief of staff and senior advisor in the Biden-Harris White House Office of Science and Technology Policy, where he helped lead a team of 150 policymakers on key initiatives, including the blueprint for an AI bill of rights and guidance to ensure federally funded research is publicly accessible. Before that, Aidinoff was a domestic policy advisor in the Obama administration and a strategic consultant with Blue Rose Analytics.
Second speaker is Carmem Domingues. She is an AI specialist and communicator. She served as a senior AI technical advisor in the White House, where she advised on AI policy and led the AI training for the federal workforce, a program to educate civil servants about the powers and limitations of AI. and how they could leverage AI for public benefit. She is a distinguished fellow at the Future Government Institute and a member of the Council on Foreign Relations.
Our third panelist is Ofrit Liviatan. She's a lecturer on law and politics at Harvard University's Department of Government. Her publications, research, and teaching interests focus on the intersection of law and politics, including the function of legal systems in divided societies, the role of legal mechanisms in the accommodation of diversity, socio-legal dilemmas around artificial intelligence, and the legal dynamics between religion and state.
And our final speaker is Bruce Schneier. Is it "Sch-nayer" or "Sch-nyer"? Did I get that right?
BRUCE SCHNEIER: It is "Sch-nyer."
EREZ MANELA: Schneier. Bruce Schneier. Apologies. He is a New York Times bestselling author of 14 books, including Rewiring Democracy and A Hacker's Mind, as well as many articles, essays, and academic papers. He also writes a popular and long-running newsletter and blog on cybersecurity, Schneier on Security. Schneier is a fellow and lecturer in public policy at the Harvard Kennedy School and the Munk School at the University of Toronto and a fellow at the Berkman Klein Center for Internet and Society at Harvard University. Thank you all for being with us today and contributing your insights. Marc.
MARC AIDINOFF: Thank you. Thank you so much for having me. And thank you to this panel. These are really hard and essential issues to think through, and I think through them as a historian of science, which means I think a lot about the history of expertise. And as you said, I've been a applied historian of science. So I was in the Obama and Biden administrations, where my last role was really thinking about what kind of expertise do we need to make public policy. How do you inform a decision?
And right now, I see AI policy as facing these two giant epistemic challenges, sort of known unknowns in the language of foreign policy folks. And I want to raise them and then talk through where I think some historical parallels might be helpful. The first is this giant question about who is an expert about AI.
And it comes to practical questions, as you put together even a panel like this, that this has been a long-standing problem in tech policy more generally, where tech policy is fundamentally distinct from something like economic policy, where economics is very codified. You have modes of thinking. You have macro and micro.
We know what an HKS degree means. We know what a business interest represent. We know how they're organized. And there are long-standing modes of making analysis commensurable, whether it's micro analysis. So cost-benefit analysis is designed to think across a whole set of domains. And then there are macro analysis, where people are going to fight about how much inflation exists. But they're all in generally the same conversation.
Tech policy is not like that at all. The domains are completely disparate. And so literally, yesterday, in my class on digital democracy, I asked what the goals of tech policy are, and the students gave answers that really showed the gamut of this problem, questions ranging from cybersecurity to mental health, environmental impacts to antitrust. People talk very conceptually about shared truth claims and the problems of living together, and they talked very concretely about intellectual property and what it means to own ideas.
So how do we think across domains? How do we think about global competitiveness at the same time we're thinking about mental health harms? How do we compare apples to oranges? And that is the key question for tech policy right now, generally for AI policy, particularly in foreign policy. And I would argue that the answer repeatedly comes down to questions of political judgment, political judgment as a tool for moving among apples and oranges, and strategies for thinking across domains.
And often, this gets shorthanded in practices like we need technologists at the table. And this is a phrase of, like, we need more technical expertise. And that's absolutely true. But what kind of technical expertise, and how are they moving among domains? And in my first tour of duty in government, the phrase was used a lot, "We need technologists at the table." And I asked, who do you mean by these technologists?
And they identified three HLS grads who were really good at coding. And this question of what it means to move between policy, between different languages really complicates how we think about expertise in this space. Technology policy pushed out certain other types of orthodoxy in terms of the policy-making space. And so maybe in the Q&A, we can talk a little bit about the rise of something like industrial policy, where tech policy has carved out a new space and a new type of expert.
But the second question that I want to spend some time with is, what does AI actually do to expertise? What does it mean to use AI tools to think, to make judgment, to know more about the world. And this is a question that I think universities are really struggling with because for us, this is a very pragmatic problem. We are in the business of making knowledge, and suddenly, we have this very strange and evolving tool that's supposed to be a partner that we do and don't own, that is changing.
And so I think there are a lot of lessons where we can really apply where research has fundamentally changed as it's become computerized, how computers can be-- AI systems can be sources of surprise, delight, discovery, but they can also really narrow our research questions. So as we think about those two sides of expertise, who's an expert and how expertise is fundamentally changing, I just wanted to raise a few areas where I think history can be a guide in some of those known unknowns.
And they come back to the way we've developed, I think in tech policy, particularly a blind spot created by 1990s triumphalism. And I'm very much, at the end of the day, a historian of the '90s. And I think that the '90s have led us to have some major misunderstandings about what tech is and what foreign policy with tech ought to be.
And I'll just tell one little anecdote, which is having conversation with a tech policy friend who I deeply respect thinking about how to regulate AI. How should we think about this? And she said, well, we need to learn the Cold War was a success. We figured out how not to annihilate the world. That's the model we should work from.
And as a card-carrying historian, the idea that the Cold War was successful is patently ridiculous and one we should aggressively push back against. The Cold War was horrific. It was horrific from a global perspective. And we both have to interrogate how the narrative that the Cold War went well became solidified in foreign policy circles, this notion that things went well just because global annihilation was prevented, but how we can push back specifically on those notions that we are building a system based on clever stability.
And so I think a lot about two colleagues in this space, my colleague in history of science, Ben Wilson, who's talked about how those very experts' selling stability, in fact, promoted the rise of the military industrial complex and certain kinds of global violence. And on what kind of expertise? I think my colleague Joy Rohde, who talks about the early crisis warning systems and these efforts to take knowledge, to put all judgment on index cards, to automate it.
And Joy is much more receptive to the idea that these folks were trying to create a more stable and more peaceful world order. And it just doesn't work. There's a rise in violence. There's a rise in militarism that comes time and time again. And what she teaches us is that these digital systems turn our attention away from the political issues that really matter.
My work looks a lot at government systems as they get automated and says, how does the power system within those organizations change? And what we see is a move from lower order technocrats being able to make decisions moving higher up the administrative hierarchy. We see legislators becoming increasingly disempowered as unelected folks take more power. We see the bureaucracy as a whole create its own sense of momentum. And we see old-style political judgment repeatedly devalued.
So my call today in the few minutes I have remaining is to remind us that in the context of AI policy, we should not lose sight of the fundamentals. And, in fact, focusing on the tools always risks us losing the big picture, us paying attention to the way money matters, the way nation states are composed and are competing. The sort of crude stories can get lost in these models that sometimes are designed to point us away from the stories we know.
I think AI has led policymakers to forget key sources of national strength. So especially in the US, you see a defunding of things like the US research university's efforts to defund our entire R&D apparatus and replace it with a doubling down on AI. That's not to say AI isn't going to do extraordinary things, but it's quite concerning if that's allowed policymakers to forget core national strengths in this hope of performing it to be true.
Finally, I want to end with a note that sort of takes a different tone, but I think we have to ask how conversations about AI policy can turn our attention towards or away from questions about the rule of law. And we're existing at this moment where you can't have a policy conversation without talking about what happened last weekend, without talking about Minnesota and its global ramifications, and the way AI companies are fundamentally entwined with the creation of databases, with the creation of narratives about where we're going in a foreign policy direction that may not turn our attention to the futures that solve the problems we most want to solve.
I find that the AI futurism is both exciting and missing. It's setting the bar at not having a Cold War and not saying what does it mean to live collectively in a democratic system. Thank you.
EREZ MANELA: Thank you, Marc. Carmem, you're next.
CARMEM DOMINGUES: Hi, everyone. Thanks. I'm going to be sharing my screen, so just give me one second. I have a couple slides that I'd like to share with you guys. In the meantime, as I'm pulling that up, thanks a lot, Marc. I feel like we have a lot to talk about, a lot of issues. I also study a lot of issues of this epistemic questions around AI narratives and so on. So can everyone see my screen? The slide looks OK?
EREZ MANELA: Yeah, looks good.
CARMEM DOMINGUES: Perfect. All right, so I'm going to go ahead and get started. So thanks, everyone. Just a little bit more color, I come from a technical background, but I also study the social and ethical implications and impacts of AI today. So I'm going to be focusing a little bit on that but from a more technical standpoint.
So I'm going to just define what I mean by AI because I feel like that's not always aligned among audiences. I'm going to talk about the AI pipeline and geopolitics, and then I'm going to focus a little bit on LLMs and their influence on foreign policy. So definition-- what do I mean by AI? AI, broadly speaking, is the ability of computer systems to perform tasks that typically require human intelligence.
Nowadays, people tend to think a lot of AI as ChatGPT or large language models or generative AI, ChatGPT and similar tools. You have Midjourney for image generation. You have Google Gemini, Claude, all these other chatbots and LLMs that we're using more and more in our day to day. They're being sold as sort of AI agents and assistants and even AI companions, which I'll talk a bit more about later.
But when I talk about AI here, yes, I refer to those, but I also refer to what's traditionally called, if you will, traditional machine learning. So think of spam detection on your email, recommendation algorithms for e-commerce, so what product you should buy next or for your social media, what posts you're shown, what content you're shown, recommendation algorithms, for example, in streaming services, so when you're browsing Netflix and it tells you what you might like and things like that. So we're kind of surrounded by different types of AI. And I want to just make sure that we're kind of encompassing all of that and not only talking about LLMs and AI.
So I think it's always interesting to think about the AI pipeline as a whole. So when we think about AI, we think of, again, of these models or these tools that we use on a day to day, but we seldom only look back as to how do we get there. How are these tools developed and built from the material products that are necessary and inputs they're necessary to the development of these models?
So we start with minerals that are literally mined from the ground in different countries across the globe, mostly in what's usually talked about as the developing world or the Global South and so on. And they're traded internationally. Then we have chips, which are essentially what makes the devices able to function. So we have the lithium from the lithium batteries. They're mined and so on.
The AI chips that allow building of GPUs and so on and processing of all this data and these models, et cetera, in devices like our laptops, like our phones, the servers that are then placed into data centers that host and, again, process all of these models and train these models and where the models are trained. And then you have the AI models, which use all this data. And they're developed by AI developers but also rely heavily on data that has been labeled for training.
And that data labeling is often done outsourced, again, to countries like Kenya and so on. And then you have the AI tools or apps that use these AI models. So when you use ChatGPT, for example, that has the GPT model in the back end, the different types of LLMs and so on. And then you have the end user, which can be any one of us that either has AI being used on us or that uses it directly.
And I say used on us because all of this data that we are-- the digital crumbs that we are leaving across the internet as we use all these apps is being collected, is being sold, is being traded. And that is information from us that's being used, whether we know it or not or consent or not really consent to it.
So as we look at this pipeline, we can see that there's issues around policies and regulations and legal matters, which I'm sure Professor Ofrit will talk more about, that encompass both the trade areas, the environmental issues, because we have data centers that consume energy and water. And where is that coming from? We have export controls on chips.
So until recently, for example, NVIDIA, which is the main producer of chips, was prohibited from selling H20s, which is a particularly advanced type of chip used for AI development, through something that was called AI Diffusion Rule. But that has now since been rescinded, and they can now sell H1 chips to China, even more powerful ones. We have issues around the geopolitics around Taiwan.
So Taiwan is the main producer of AI chips. So even though NVIDIA is the main designer, it's an American company, the machines that are used to build these chips are developed in Europe, but then the chips are actually produced in Taiwan. So when we talk about, oh, Taiwan and China and so on, Taiwan belongs to China, does not belong to China, what's going on there politically has deep impacts for all of us globally because we're all users of AI.
And that kind of dictates a lot of foreign policy from the different countries that are involved. And similarly, with data centers, these data centers host information from citizens of different countries. And as, for example, Europeans have certain regulations around AI, GDPR, and so on, and data protections, what happens if their data is transported to other countries where they don't have those protections? What happens?
So I'm just going to skip ahead because of time. I wanted to show a little bit of the distribution of the compute divide between Global North and Global South from this paper, but I won't have time, so I'll just skip ahead. So how are LLMs and foreign policy related?
So research shows that LLMs influence how we think and act. So think of LLMs. Think of, again, ChatGPT, Claude, Gemini, AI chatbots. It could be, for example, character.ai. It could be something that's being sold as a companion AI. And they influence how we think and act in a variety of different ways and through different techniques. And I'm going to talk about a couple of them because, again, time is limited.
So personalization-- So one of the main selling points for AI chatbots is how personalized they can be as AI assistants and as AI agents. It's essentially build your own AI assistant to better help with the tasks that you want to optimize. And what research shows is that AI, the LLMs, can infer psychological dispositions of the users from user-generated text, needing surprisingly little data.
So Peters and Matz here, the paper quoted here-- and this is just, again, a sampling of papers that can show these different phenomena that I'm talking about-- show that, with just some Facebook status updates, LLMs can infer people's personality traits. And at the same time, research also shows that LLMs can mimic personality traits. And so basically, they can be built and shaped along desired dimensions to mimic specific human personality traits.
So for example, now, if you download ChatGPT 5, you can look in their product description. It talks about how you can pick from four different personalities, like nerd, robotic, listener, and I forget what the fourth one is. But then you can customize on top of that for it to have and react and respond to you in ways that better match your preferences. And then so this personality matching, or matching the content to a user's personality, has been found to be highly effective at enhancing levels of persuasiveness and influence.
And all of these research that talks about how these phenomena happen, they always highlight ethical concerns around individual autonomy, self-determination, or one's ability to make decisions about one's own life and manipulation. Yeah, OK, we're back to the slides. Oh, wait. Sorry, I skipped a slide.
OK, elicitation of data-- another thing that LLMs are also very good at is collecting way more data about us than was previously possible with other platforms. So they can infer personal attributes through simple conversation. So even if I don't say, hey, my name is such and such, I am X years old, this is my general income level, they can infer that from just benign conversational exchange, quite accurately.
And so the more we use these chatbots and the more time spent and more conversation we have with them, the more data we end up sharing with them indirectly, but also directly. More and more, especially as these tools are used as AI assistants, we're feeding, actively feeding, and providing access to more and more sorts of information. So it can access your calendar so that they can set up meetings and things like that.
They help people with writing tasks, so they know what topics you're writing about or how you write. They also learn from that. Maybe you research health issues or use it as your therapist, which more and more people are doing, as well. You're sharing a lot of information with these things. And when we have-- and when people get involved with AI, like in emotional relationships-- and research showing that 20% to 30% of adults are reporting having had some sort of intimate encounter with AI-- you're sharing a lot of that information.
And this paper here on the right from the MIT lab shows that almost 70% of AI companion relationships actually started unintentionally, with 40% having come through productivity uses, so people that started using AI chatbots as AI assistants to just help optimize and expedite a lot of the tasks that they do on a day to day. But through these conversations, again, with these tools, they ended up developing emotional attachments and connections. So essentially, we're sharing our deepest thoughts and fears and so on with these technologies.
So there are different mechanisms, also, through which the LLMs can shape our thoughts and behaviors. Work by Jakesch-- I think that's how you pronounce their name-- show that writing assistants that express certain opinions can influence how people write and think through a phenomenon that they call latent persuasion. It's essentially like when you're, for example, in a buffet line, and they change the choice architecture they have around what choices that you pick from, the food you put on a plate, and you put the healthy options first, it's more likely that you will load up your plate with a healthier options and have less space for the unhealthy ones. But you can still see that they are there, right? You can make the active decision.
So what they say is that these nudges in behavior end up making some choices more convenient than others in this way. But in language models and technologies, they may operate or cause shifts in opinion by making it easier to express certain views, but not others. And the issue here is that just choice architectures on a buffet line, for example, are visible. Opinion preferences, they are built into language models, may be opaque to users.
And then there's this other phenomenon also that the Cambridge Analytica whistleblower-- if you remember the Cambridge Analytica scandal from a few years ago-- Christopher Wylie, refers to in his book as a scaled perspecticide, which essentially kind of shaping the information environment around people so that they can essentially be manipulated. And if we have LLMs there interacting with us both in feeding us information and helping us produce and create information, we're essentially, potentially, immersing people in an information environment that we are shaping for them.
So if you look at what happened with Cambridge Analytica, that was basically-- it was based on Facebook over 10 years ago, Facebook-- these personality tests and likes and so on and Facebook, being a social network-driven algorithm, where you're mostly seeing information that's shared by your friends and people in your social networks. And they have essentially, in the back end, what I referred to earlier as traditional machine learning, these recommendation systems. And ads there are quite clearly marked, generally speaking. You see that something is an ad for a product or even a political campaign or something like that.
And then you have TikTok has also been researched quite a bit and found to have had influence in the US elections, generally speaking, or at least on the political discourse happening in the country, or countries. It's algorithm-driven. Like you're not relying on social network. It's just what is the algorithm feeding you based on what you engage with. And it's still more traditional machine learning. And the ads here may be more or less embedded into the content. It may be less clear, if an influencer is sharing an idea or recommending a product, whether or not they were paid by anyone to do that.
And then with chat bots, as we evolve that, they're very user-driven. You're telling it what you need, what you want, what you like, what you don't like actively, and directly and indirectly by what I discussed in the previous slides. So we're sharing a wider amount of information and sharing much deeper types of information, as well. And, they're powered by generative AI, LLMs, and things like that.
And as companies start to show and portray and accept ads in their platforms-- OpenAI recently announced that they will start showing ads-- it's less clear how they're going to do that. And the AI-- and I'm not singling out OpenAI here specifically, but I'm saying, in general, the possibilities that exist. The ads here can be embedded into the personalized text that the GenAI, the chatbots, are generating as people interact with them and ask for research or recommendations, share thoughts, and so on.
So what are some of the implications for foreign policy? If countries can use these technologies to influence how people think not just within their country, but across the globe, they can potentially single out, oh, we want users in this particular country to be shaped in this particular way. Or companies from-- domestic companies, so let's say in the US, an American company, could be potentially paid by a foreign government or power or company to advertise products and ideas that go against national interests of the country where it's based.
So we have, essentially, foreign companies that can potentially operate in and exercise influence over domestic and global contexts. And because of the opaqueness of the algorithms and the ads that I referred to earlier, this becomes much harder for us to identify what's happening and for researchers to measure the impact that this is having, for policymakers to craft policy that makes sense because researchers and policymakers and so on, we need access to open data, to APIs and so on. And it is hard to do that research when this data access, this API access can be shut down by countries, companies in other countries, or that can be limited access, and so on, both domestically and internationally. And then we have issues of data transfers across foreign borders and so on and legal landscape and enforcement.
So takeaways, AI influences and is influenced by foreign policy issues throughout its whole pipeline, all the way from the mining of minerals to the use of the AI models, by or on all of us, including issues of international trade, labor laws and dynamics, legal issues around intellectual property, copyright, data rights, and so on. And AI is also designed to influence behavior. And the more advanced it becomes, the better it gets at it, from the business intelligence from back in the day to traditional machine learning and LLMs and GenAI.
And what are the consequences of this? Behavior influence across boundaries has significant impacts to democracy, to global relations between countries and so on. And the asymmetry of power that the AI creates, the increased use of these technologies creates, between companies and nations that hold it and users and companies and, therefore, nations as well, can shift the balance of power on the global stage, with, again, deep implications for foreign policy. And we can talk more about this in the discussion. Thank you.
EREZ MANELA: Thank you very much, Carmem. Ofrit?
OFRIT LIVIATAN: Thank you for convening this, Erez, and the Weatherhead Center. And thank you, of course, for this generous invitation. So I'll talk a little bit about AI governance, which is in its infancy.
On the global level, we find a patchwork of non-binding instruments, declarations, standards, principles, et cetera. They are still a very good way from any coherent framework. On the national level, we find that regulation is dawning, but implementation is still very much in its early days. And similarly, on the regional level, we have the much-celebrated example or the unrenown, depending on one's perspective, of the EU AI Act that is also yet to become fully applicable.
Now, these very early, very infant-style regulatory attempts are couched in an intense debate on our future with AI that swings between utopia and dystopia. We heard about that a little bit from my previous people on the panel. With all their differences, though, AI optimists and AI pessimists seem to agree that we are in an Earth-shaking moment.
So for someone like me who thinks about law and policy, it's quite intriguing to consider, what is this AI moment? What does it mean for the international legal system? After all, history is paved with examples where turbulent technological periods actually served as catalysts for the development of international law. Think, for example, about the age of steam, or the discovery of electricity, the invention of the telephone.
All of them kind of ushered for us a global norms on monetary issues, on intellectual property issues, protection for workers intended to navigate the effects of industrialization. And of course, the atomic bomb was a driver for global agreements to avert the catastrophes of that human ingenuity. So this AI moment isn't only offering us new ways to make international law. It is also an opportunity to reflect on the consequences of our innovativeness and potentially introduce adjustments.
So let's first discuss how can international tools enhance the design of international law. And we deploying these tools, which we do increasingly, as we heard from Carmem, will have a dramatic effect on the design of international law. Large language models, LLMs, our new oracles, can drastically speed up law formation. They can optimize data analyses. They can expose weaknesses in current laws. They can identify compliance gaps, and so forth.
So in short, AI tools have the capacity to really revolutionize the production of regulatory instruments. At the same time, the same tools can be equally effective in evading international obligations. Their capacities are ideal to identify path for forum shopping, say, or log-rolling. And they can help very much with regulatory fragmentation. So in a word, AI tools can fuel the downsides that international law is predisposed to-- inconsistencies, overlapping rules, and non-compliance.
And the challenges don't end there. Our AI moment is a significant stress test to the ability of international law to shape the direction of technological development. And in the interest of time, I'll just mention three such examples of the challenges that AI poses for international law.
The first challenge hinges on the nature of AI. And we heard about that previously, as well. The nature of AI is a rapidly changing technology, and in that, it demands an anticipatory approach. And that is a very rare undertaking for the international legal system that has long followed a reactive approach, remedying defects after they occurred, not before. But governing ever-accelerating tools requires forward-looking rules, which means a conceptual shift for the legal system that has traditionally looked backward on state practice and other matters when it produced regulatory solutions. So that's the first challenge.
The second challenge for the development of global AI governments is that it will have to overcome a misguided, in my view, but a very dominant presumption, that regulating AI will hurt competitive advantage and constrain innovation. I personally find this innovation regulation standoff misguided, given that innovation isn't always progress and because the function of law is precisely to steer behavior and shape preferred directions. We also have in place extensive regulatory infrastructure for pharmaceuticals, for food, for transportation, you name it. And in all these areas, innovation seems to be plentiful.
So any serious attempt to harness AI for the betterment of humanity must include social safeguards and must define legal accountability. Yet we keep hearing about the threat to innovation in the attacks, for example, on the EU AI Act, in Trump's AI action plan, and from people opposing regulation, both based on what has come to be known as the Collingridge dilemma, or the pacing problem. In a nutshell, the Collingridge dilemma contends that, at the early stages of development, it's very easy to regulate an invention, but the harms are yet unforeseeable, that the risks are unknown yet. The longer you wait to regulate, the harms are clarified, but the cost of regulation rises, or the regulation becomes ineffective or inappropriate.
The most common example given to illustrate this Collingridge dilemma is social media. So back in 2004, we could have potentially regulated social media, but the harms were still unknown. Today, we know the harms, but the business model under which it is operating, what Shoshana Zuboff coined as surveillance capitalism, makes regulation challenging.
Now, as a recovering lawyer, I find this representation a misreading of the art of lawyering and a misreading of lawmaking's capacity to innovate. Yet it is quite a dominant perspective in the discourse today and heretofore stifles AI governance. So that's the second challenge.
The third challenge to global AI governance arises from the fact that the digital ecosystem clashes with the classic categories of public and private law. As non-state actors, big tech aren't part of the classic state-centric international law. Here, too, developing AI will require a paradigm shift that treats these corporations as subject of international legal concern.
Now, before closing, I want to contextualize this AI moment within this competing narrative of utopia and dystopia. The utopian version of the current AI moment rests on the idea of superintelligence, rests on the idea of singularity. And that is that, soon, very soon, machine will surpass human intelligence. And now, if that scenario materializes, we can potentially find ourselves under a legal regime designed by a superintelligence.
Predictably, the loudest overhypers of superintelligence being "just around the corner" are those tending to profit from AI adoption the most. And this is troubling not only because they have the least incentive to place any guardrails on this process, but also, from what we heard from Mark in his opening statement about the history of technology, which teaches us, A, that the impact of new technologies has been unpredictable, and B, that technology in general and particularly AI have boomed in the past and then busted.
Yet, given the singularity hype that there they'll take over and become and do everything better than us, we should be asking ourselves whether a legal regime designed by a superintelligence, superintelligent AI, should be the object of our desires. Living under an AI-produced legal regime that we cannot comprehend because it was designed by a much smarter tool than us isn't only senseless, it is also quite inconsistent with foundational legal principles. What is the rule of law without accountability? What is the rule of law without explainability? So it seems like careful what you wish for does come to mind, even when considering the utopian version of AI.
But the dystopian view is also unhelpful. Interestingly, superintelligence, or singularity, or losing control of our inventions, made its debut not in techie sandboxes in Silicon Valley, but actually in literature. There are wonderful literary narratives of humans creating the systems that turn on them. To my knowledge, it started with the golem of Prague in the 16th century, onward to Frankenstein, into the 20th century with EM Forster, Huxley, Asimov, so many more, and we are still counting.
But our current reality illustrates that we don't need to wait for superintelligence or singularity to suffer the havoc of algorithms. Just look at the impact of pre-singularity algorithms on our current global environment-- the negative effects of social media on our democracies, our social fabric, the rising disinformation, the ever-expanding cybersecurity risks. Hyperfocus on this dystopia of superintelligence deems what we should be urgently reversing now.
The EU retreat from regulation-- the only supranational entity today to regulate, is currently retreating from its regulation; the hypocrisy of tech giants who make promises on AI safety while downsizing their safety and security teams; the digital carbon of data-hungry machines. Reversing those dangerous trends of algorithms will require collaboration, cooperation, and buy-ins. That is, it will require traits that machines lack but, luckily, we humans still have. Thank you.
EREZ MANELA: Thank you very much, Ofrit. Bruce, you're up next.
BRUCE SCHNEIER: Yeah, I've got to get my camera to work. OK, hi. Thanks for having me. I didn't think, actually, I would be left to talk about some of the optimistic things that AI can do for international relationships. But I think, since we haven't, I want to talk about it.
I just wrote a book called Rewiring Democracy that looks at how AI, very broadly, will affect democracy-- very broadly. And it is largely optimistic. I mean, certainly there are risks. Certainly there are challenges. But I think the notion that these technologies can make democracy more responsive, more fair, more equitable, more just-- there are lots of possibilities.
Fundamentally, AI is a power-enhancing technology. In the hands of people, humans who want more democracy, AI will help that. In the hands of humans who want less democracy, AI will help that, as well. It will magnify the power of the people who use it.
Something I want to talk about, there was a talk, I think, by the second speaker about some of the resources that AI consumes. This is all changing. Right now, the large tech monopolies are hell bent on these very massive, very expensive in several dimensions models. But it's not necessarily true. We're seeing diminishing returns from these ever-enhanced models. And there's an alternative version for AI, which I want to talk a bit.
This comes from the separate technology of from the business processes that are producing it. The fact that AI today is overconfident, is obsequious, makes mistakes in a certain way, has biases, these are not the technology. These are the companies that are making the technology. And the companies can choose different paths.
So there's been a growing movement for something called public AI, which is AI not created by the large tech monopolies, AI created by a non-government organization, by a government, by a university, not beholden to the profit motive. Last year, there's any number of supports and ways to think about this. When we wrote this book, it was largely something we hoped for.
But in October of last year, this changed. Switzerland produced a model called Apertus. Apertus is the first fully formed public AI model. It uses no rare Earth minerals. It used chips that the computing center had lying around from a few years ago. It used renewable power, hydroelectric. Its water is renewable. It used no, zero, poorly paid Global South labor. It used no, zero, illegally taken copyrighted material. It is an example of how we can build an alternative vision of AI.
And my guess is, as the cost of making these models continues to decline, we will see more of this. And right now, the performance of Apertus is about a year behind the current frontier. The Swiss are working on a second model. This is a consortium of universities with some funding from the government, much, much less than the American AI companies demand. So that's one of the many things that are happening to change the business dynamics of this technology.
I think it is really interesting to watch these technologies being used in an advisory capability. And I think there's a lot of value that we're going to see of these models in international relations directly as advisors. I mean, we can imagine a human having-- going into a negotiation with human advisors and AI advisors. And that human will use both of those. And the AIs will have insights, as they do. I mean, they're becoming incredibly capable.
Certainly there is worry about whether you trust the AI. But that's going to be whether you trust the corporation. And my guess is we're going to see a government AI that the government will use for these more sensitive negotiations, that won't upload your data to who knows what company, that won't unduly manipulate you, that will be designed under different principles, not under profit. So that's the good side.
The bad side is that I'm not sure we'll be able to trust these AIs And by that, I don't mean that they're going to make mistakes. I don't mean hallucinations. We're getting much better at that. Things you read a year ago aren't true anymore.
Sure, they're not perfect, but humans are not perfect. And in any of these systems, always look at compared to what. Compared to your human advisor, who will also make mistakes. That's why they're advising and not in the lead.
But I worry about these as computer programs. At the end of the day, AI systems are written in software. They're running our hardware. They're attached to networks. They're run by users. And then all of this is vulnerable to hacking. I come from computer security. I know about vulnerabilities in computers and networks and software and systems.
So as soon as you put any AI system in a position of power, where it is making a recommendation to a government or even to a corporation, you're in some environment where someone else has a vested interest in what the AI produces or the data it uses. That means it will be hacked. And that means, just as governments around the world, we hack each other's power grids, we hack each other's government databases, we're going to hack each other's AIs.
When you think of AI hacks, there are three different levels you want to think about it. First, an adversary is going to want to manipulate the AIs output, maybe by biasing the training data, maybe by going in and manipulating its process, maybe by directly manipulating the output. That's the most important thing you can do.
Failing that, you're going to want to eavesdrop on it. If I can't affect what it says, I want to know what it says. I want to know what sort of advice that AI is giving to a negotiator. And failing that, I want to be able to disrupt it. I want to, at some important moment, have the thing not work or not work well. And all of those are possible.
So I think about an AI as an advisor in an international trade negotiation or as a political strategist or a legal researcher. Someone will want to hack that AI, maybe a criminal, maybe a government. And it doesn't matter how accurate, capable, powerful, hallucination-free that AI system is if you can't guarantee it hasn't been hacked. It just won't be trusted.
We are already seeing Russian attacks deliberately manipulate AI training data. We are already seeing attempts to influence what the AI does. And that's not going to change. So everything I know about computer security flows onto AI, along with everything else about AI vulnerabilities-- prompt injection, training data manipulation, all of these vulnerabilities that we don't know how to fix. I can go on on prompt injection. It's actually a problem we don't know how to solve.
So I see an enormous promise for these systems in international relations because of their power, because of their capabilities. And those will continue to get better. I think their environmental concerns will be lessened as they become easier to run. I know someone who's running the DeepSeek-- that's a Chinese model-- on their phone. It's not being run fast, but they're running it on their phone.
This is going to change. And don't think about-- don't think the limitations of today, whether they be corporate or even tech, will be limitations of tomorrow or next year. And I want us to think about the possibilities of using AI in international relations, assuming these things will get better.
But also, as a security person, I know I cannot solve the security problems in a way that will make these trustworthy for some of these applications. And that's going to be the challenge that I see of AI in these sorts of systems.
EREZ MANELA: Thank you. Thank you very much. Thank you very much, Bruce, and thank you to all of the panelists. If everyone can turn off-- or rather, turn on their camera again. This has been absolutely fascinating and very stimulating for me and, I'm sure, for our audience, which is more than 150 participants.
Let me just start, if I may, the conversation with two quick questions that have been on my mind and have been touched on in various comments but haven't been fully answered, I don't think, not that we can fully answer them here. But I'd like to hear more about them.
One is-- and Bruce and others were referring to this-- the emerging role of AI in the policymaking process. Now, I'm a diplomatic historian. I know that these bureaucracies, the State Department bureaucracy, the international relations and the security bureaucracies, produce mountains of documents. And I'd be shocked if AI has not been used in the last few years, at least, to produce some of these documents, starting with public-facing documents like the National Security Strategy and including memos to principals and all sorts of things that will be then closed for the next 30 or 40 or 50 years, but future historians will see, one hopes.
So particularly for those of you who've either looked at this from the outside or have seen this from the inside, being in government, what do you think-- what are you seeing in terms of using AI both as an advisor and as a producer of documents? Is this already a common practice, and how, if at all, is it shaping the policymaking process and the decision-making process. So that's one question that was on my mind.
The other question was on my mind, which is an obvious one, and I was surprised it didn't really come up more, is the use of AI in military conflict. As Marc reminded us, the Cold War was wasn't exactly a garden of roses. And of course, the post-Cold War period hasn't exactly been conflict-free either, as we know.
And everybody, of course, is expecting-- AI is already used in military conflict. We heard reports, for example, that Israel was using it to select targets in the conflict in Gaza. And I'm sure it's been used-- I haven't read anything specific about the war in Ukraine, but I'm sure it's been used there, as well. I'm wondering what that conversation looks like, the use of the various types of AI. As Carmem reminded us, it's not just LLMs, the various types of AI are in military conflict, and how is that shaping. Both the prognosis of policymakers, the decisions of policymakers, and your own sense of where we're going?
CARMEM DOMINGUES: So I might jump in here, especially on the first one. So when I was in government, we were doing oversight over AI policies across and what the agencies can and cannot do with AI. And agencies, at the time, were not allowed to use-- US agencies-- not allowed to use AI to create policy. So they were not allowed to use it to draft memos and things like that, for a number of reasons.
But there were a range of other uses of AI. And you can check out the AI use case inventory. It's actually a public inventory of AI use cases across government. And of course, there are some that were not disclosed on there due to security restrictions and so on. So those, of course, I wouldn't be able to talk about either.
But one-- you talked about the State Department bureaucracy and so on. And I think that they had a very interesting case. One of the use cases that they had was they developed an application called StateChat, which essentially was an LLM machine or system that allowed foreign officers and diplomats and so on to expedite the writing of cables and other documents, leveraging other documentations that they had that were private to the State Department with the goal of expediting communications across consulates and things like that. So that was an example of it being used for bureaucratic reasons or to reduce bureaucracy or expedite services.
BRUCE SCHNEIER: I can touch on something real--
EREZ MANELA: Sorry, just to say, Carmem, you mentioned the fact that there was policy against using AI. There's policy against using AI in my classroom, too. And yet I assume that a certain non-trivial percentage of students are going to use it anyway. Bruce, apologies.
BRUCE SCHNEIER: Certainly true. I can talk about a related area, AI in lawmaking. So the first example of an AI-written law came from Brazil, Porto Allegre. A city councilman wanted a law about water meters, fired up a chatbot, got text, submitted it to the legislator.
It was debated. It was voted on unchanged. It's the first example of AI-written law. This seems like a phenomenal development. A human has a need, goes to an assistive tech for some assistance, gets words, which go into the human process of debate and vote.
Presumably, as Erez said, this is probably happening all over the world. France has an AI model for legislators designed to write French law. Chile has an AI model designed to look at interactions between laws. They have rules about making interactions known about new laws and previous laws.
So I think we're seeing all of this assistive tech. We know judges are using, or attorneys are using it. What's really interesting is how bias fits in because my guess is that these AIs are going to be biased, and that's a good thing. Just like a judge chooses law clerks who share their biases, they're going to pick an AI that shares their biases. So one person can use it as a draft. It goes into the human process, and I think we're good.
There was a discussion just yesterday about US government Transportation Bureau using AI to draft regulations. I mean, we use humans to draft regulations. And always compare to what. I mean, sure, we have to deal with mistakes, correcting. They're both processes.
I want to quickly mention about AI and targeting. I mean, computers have produced targeting decisions for decades. And this is just slightly different math. I'm less concerned about the technology doing it and more about, again, the human processes behind it when we let autonomy make decisions of who to target. Think of a landmine is the ultimate example of an autonomous weapon that just makes absolutely no distinction between who it blows up on. So these are not new problems. I mean, they're different, but they're not new.
MARC AIDINOFF: I just wanted to say something on the sort of off-brand and sort of under-the-desk use of AI at the State Department or in your classroom, which is, the way you understand these tools, is by playing with them and playing with them a lot. There's just no way to understand certain technical systems without extensive tinkering. If we all-- just to use the canonical ChatGPT examples, I feel like everyone goes through this curve of it's so amazing, oh, my god, it's so bad, oh, no, I actually understand how to use it.
What's happening is, when you disallow this, or when you require only in a sandboxed environment, or you actually have a FOIA requirement that says the play is limited to create documents, it creates these real problems of people not using the tools, not developing the certain kinds of comfort with how to use the tools that really worries me. I think of Harvard set up a sandbox. It sounded great. We were going to have a sandbox, and the students were going to use it.
And actually, the students don't use it, because they assume that they're being spied on. And it's not such a dissimilar story with some of the sandboxes in the federal government that they haven't proved as successful at helping people play. And then a lot of the use is very vulnerable to the very attacks that Bruce was labeling, where you really, really don't want federal employees using their personal phone under their desk to access these tools. So actually figuring out what the environment is for legal and appropriate, not only use, but I'd say play of getting to tinker, to try it out, to really understand and get an intuition is a central part of how we make these successful.
OFRIT LIVIATAN: I wanted to touch on what you asked about the military, the connection between the military and technology. And of course, the marriage between the military and big tech is one of the strongest there is. It's kind of nice to see a good, strong marriage with really no problems whatsoever.
And Bruce mentioned that the problems are not new and maybe different, but I'm not even sure they're different, that very much. So you have the secretary general starting two years ago-- he wants to produce a treaty to ban lethal autonomous weapon laws-- he calls it "laws, which is kind of funny-- lethal autonomous weapons by the end of 2026. I think he finishes before that even is hitting any ground.
So the same problems in international law, the same type of tensions between soft law, hard law, between places we want to conquer versus places we don't want to conquer, digital cold war, all these things. We've seen it before in many ways. And I think it's just shaping again under the AI umbrella.
EREZ MANELA: OK, well, thank you. Thank you all. I'm going to go now to the audience questions. And I'm not going to be able to just take each of them separately. There's too many for the time we have. And I think our discussion so far has already covered at least a bit of what's been asked. But let me ask-- let me reformulate a couple of ideas here and put them and put them to you.
One is a question from Gurol Tuncman. Forgive me if I'm mispronouncing this. I think the question is about AI models talking to each other and whether that could sort of have a-- let me see-- whether that-- if I'm understanding the question correctly, whether that could be a substitute or a supplement for elite conversation and bring us some new ideas. And again, maybe I'm taking this question in my own direction, but I think that's an interesting aspect that we haven't talked about yet, AI models talking to each other and coming up with new ideas on their own in a kind of conversational way among themselves.
The other question that I'm going to put before you, from Stepan Khzrtian, it has to do with global governance, AI global governance, and what happens if there is a framework that's being developed for AI global governance? Will that not disadvantage the players that adhere to it, as opposed to the players that choose to not adhere to it? Which I suppose is the perennial question about global governance in general.
BRUCE SCHNEIER: Laws always unfairly hinder those who agree to them, right? If I just would ignore laws and take all the money out of the bank, I would do so much better than everyone else who follows the law and doesn't. This is why we have enforcement. But yes, if there's no enforcement--
EREZ MANELA: Right, but internationally--
BRUCE SCHNEIER: --then of course.
EREZ MANELA: --one might argue that we don't-- we may not have enforcement or [INAUDIBLE].
BRUCE SCHNEIER: Indeed. I mean, I think the arms race metaphor is largely a bullshit metaphor. And there really isn't this nation-state-- this is not the 1960s. All the research happens in public. All the interventions happen in public. All the papers are public. We all learn from Chinese doing DeepSeek. We've learned from Apertus. Companies learn from each other. It's just a very different world.
I do want to talk about the notion of groups of AIs. A lot of work on that in concerts of multiple AIs working together or debating or discussing. And yes, there's interesting things come out of that. Will it change things? Probably not. But it is how the big systems are working today, multiple different AIs talking to each other.
Of interest to me is whether we could unlock modes of problem solving that are unavailable. There's a whole lot of problems where, if I told you, I would have to kill you. But I can't tell you the secrets, because now you know them, and I don't want you to.
But there are ways that you can build AI models of people or personas or countries, put them in a box, tell them all the secrets, have them do the thing, and then erase them when we're done. And people are starting to think about that. Does that unlock new ways of problem solving when sides can put all their cards on the table, knowing that they won't get out of that box. And I mean, I don't have an answer, but that's an interesting way people are thinking about.
CARMEM DOMINGUES: Unless the box is hacked? To your point earlier. [LAUGHS]
EREZ MANELA: We have about five minutes. So if Carmen, Marc, and/or Ofrit, you want to have a minute or two to contribute your thoughts on any of the things that have come up.
MARC AIDINOFF: I want to hear what Ofrit has to say on the global governance question. But I'll say that, from my perspective, it's important to remember that Europe's-- the EU AI Act being on hold in certain ways, and the British slow-walking AI regulation is not out of fear of being left behind. It's fear of retribution from the United States. And the AI Action Plan is very clear that it is US policy to-- I don't think this is overstating-- to aggressively promote US companies and interests.
And so it is not, when you talk to EU regulators, they're not weighing this sort of innovation regulation binary. But they're thinking specifically in a bilateral perspective of what is the US-- how is the US going to respond. And so that's a very different kind of foreign policy question.
OFRIT LIVIATAN: And again, I think that the issues of governance are the same with AI as they were throughout history. Are you a realist? Are you an optimist? This is how you define an international law. I also want to say something a little bit about developing things that, what Bruce was suggesting, that we can put it on the table and then take it off.
A lot of people with now being faked videos and other things, so they'll always be that close. And we are in Boston. One of the good examples is that the IRA once thought they can tape themselves at Boston College. And then suddenly the judge opened it up for everybody to see. And nothing will come out before everyone dies.
So I wouldn't suggest put your cards on the table in this kind of a model, where surveillance capitalism is the dominant is the dominant version. I wouldn't be, as a lawyer, I suggest to you, wait this one out a little bit. But I think the modes of AI governance are the same. I really don't see anything different. And what we've had, hopefully, we won't have a later. But that's probably one of the versions.
CARMEM DOMINGUES: Erez, I feel like my co-panelists covered quite a bit on this. Do we have more questions that we could get to in the next couple of minutes that we have?
EREZ MANELA: We have about three more minutes. Let's take a look at-- let's take a question from Tomoko Takahashi regarding potential biases of AI systems when used as sources of foreign policymaking. "Would be interested in your views on how this relates to the presence of specific lobbying groups within the AI industry-- for example, tech rights groups." I've never heard of tech rights groups, so maybe you can tell us what that is. Is it the technology itself has rights?
CARMEM DOMINGUES: So interesting question, actually. Yes and no. So usually it refers to advocates for data rights, and people should have rights over the AI and so on. But more and more, yes, there is work being done by philosophers around whether AI itself should have rights. Actually, somewhat recently, maybe in the last year or so, Anthropic hired a philosopher to study that. Interestingly, I'm not sure if they also hired somebody to study the impacts on human rights caused by AI. But that is up for debate and question.
So yes, there's work being done into that. And I think, look, these models reflect the data that they're trained on. So if they're mostly trained on data that is techno-optimist and texts that highlight only the positive impacts of technology, it's likely that the models will reflect that. And similarly, if they're trained mostly on the dystopic version of it, of how AI is going to blow up the world and screw everything, so that is going to be what they reflect, as well.
So I guess it's going to become a matter of how much power and space each of those groups manage get and how much visibility they get, and also how the models are tuned because going back to what I was saying, we have seen examples as well of models exhibiting certain biases. And then a certain policy in house changes, and then the models start exhibiting a different type of bias, answering questions in ways that are inaccurate one way or the other. So if that happens, then it kind of matters less what data they were trained on, but how they were tuned to respond to certain types of queries.
EREZ MANELA: Well, I think we've come to the end of our time. Apologies. There's so much more we could have talked about. And apologies to those of you in the audience whose questions didn't get to be discussed. But for me, this was just incredibly eye-opening.
So thank you to Marc, to Carmem, to Ofrit, and to Bruce for joining us. Thank you for all of you in the audience. And come see us back at the next Weatherhead Forum.
CARMEM DOMINGUES: Thank you so much for the invitation for coming, everyone.
EREZ MANELA: Bye-bye.