Security: Datacenter & Model Weight

I'm really honored to get to speak to you guys today on the topic of the security of data centers and model weights, a topic that I think is already very much in the active discussion at this conference. So I already was very kindly introduced and I'd love to mention that my co-lead on the Security Level 5 Task Force is Phil Reiner, the CEO of IST. He's fantastic and together we are quietly running an effort called the SL5 Task Force which is around a 40-person cross-industry group with the mission statement of creating the optionality for US AI labs to deploy security level 5 within the coming three years.

Ideally shorter, but you know. So today I'm going to start by introducing some key terms. Then I'm going to talk about the threat models, trying to argue for why is it critical to secure Frontier AI and why does this continue to matter? If China is catching up, I'll take a brief look at situational awareness.

Where are we at today with regards to security at Frontier AI Labs and what would it actually take to reach security level five? Where do we already know how to do some things for example from the government side and where might those things not translate one to one to the needs of the AI industry?

I'll close with some actionable takeaways and then leave a bit of time for Q and A at the end. First off, just to make sure we're all on the same page, weights are massive matrices which tend to be stored within data centers which are specialized facilities for AI training and inference workloads and specifically they have very high performance AI chips in them. Sometimes nowadays the weights are also still stored on AI lab developer like their researcher hardware and laptops.

But increasingly that is something that will probably be scaled back to be only inside the data centers. And I want to clarify that this presentation is only going to be focusing on closed model weights. Open-weights is a whole other situation. And also most people have probably read this report at this point, but the AI security levels were categories created in the RAND Securing AI Model Weights report.

Fantastic report if you haven't read it yet. And basically SL1 refers to systems that are vulnerable or can just defend themselves against amateur attempts. SL3 is a little bit more advanced cybercrime syndicates or insider threats. SL5 is you are the priority target of a nation state cyber effort.

So for some context this is of course not publicly stated anywhere. My estimation is that most of the Frontier labs are currently somewhere around just finishing SL2, getting solidly to SL3, maybe some really advanced ones are starting to be doing some SL4.

Unfortunately, some Frontier Labs still seem to be at a much lower security point. So where are we at today with all of this? Most of us have probably seen AI 2027. It's possible we may be under two years away from AGI human-level intelligence.

I think it's relatively likely that once we get to that point, we should be prepared for ASI to follow soon after. Could be in one year or a small number of years, we don't really know. Against that backdrop, just last week, I think Thursday last week there was a new milestone in threats from agents with regards to cyber capabilities. Somebody used o3 so one of OpenAI's reasoning models to find a zero-day vulnerability in the Linux kernel.

This was hereto considered a relatively high bar for a model to achieve and it has now reached that bar. At the same time, we are not on track to have the optionality to deploy SL5 at the current time. We are not in a place where the Frontier Labs are in fact protected from being a priority target from nation states. Last year Google, which most people look to as one of the leaders in security, had a Chinese engineer walk out the front door with over 500 documents relating to classified chip design that they were working on.

So, all right, the state of things isn't that great. But why is it so critical to secure Frontier AI? There are a number of reasons. The one that's often talked about is theft of model weights.

What's so bad about it if these weights get stolen? Well, for one, it's IP leakage. We're giving away very, very expensive IP for free. But much more costly than that is that once these weights have been stolen, all the kind of safety precautions, such as safety fine-tuning that were moved in or that were put in before deployment can be removed.

Also, we no longer have control over who can wield these very mighty models and to what end they are using them. This will become increasingly irrelevant as we get to a place where these models are human-level and are able to replace human workers are able to operate at the capability level of a highly, highly competent human.

The other big component here is theft of algorithmic secrets. This has not been part of the discussion as much yet, but this is basically even a much harder payload to secure because these secrets can be even just a couple lines of code and these can often lead towards accelerating adversary R and D efforts.

It also can expose the company that has had their secrets stolen to more informed sabotage attacks. Third, loss of control risks, a term that is probably not new to most people in this room. I want to specialize this down a little bit more with regards to loss of control when it comes to securing front AI inside a data center. We need to be treating these AIs as novel types of insider threats.

They may pose types of threat vectors that we have not made any experiences yet with so far. They may be operating at a scale of autonomy that we have not had to deal with yet in the way we've done insider security thus far. We also need to be prepared for the fact that the AI or AGI that we're creating may be misaligned. It may seek to take control over resources that we do not want it to be controlling.

We need to be prepared to put in place the kind of strong containment necessary to avert this kind of outcome. I also want to be clear. I think we cannot contain ASI. There exists no system specification that can successfully do this indefinitely.

However, we can buy time for us to find better solutions if in the meantime we have put in place strong containment such that the model cannot immediately self-exfiltrate or cannot immediately deploy itself somewhere or take control of the data center. Finally, one threat model that has recently entered the conversation more and more is sabotage of the data that is used to train the models on, but also the model itself and its integrity.

Why does all of this continue to matter if China is catching up and is maybe having open-weights for most of the models that it's developing so far? The answer to that, I think, is sabotage. As we near what I'm calling the snowball point of automated R and D. So when we are using the AI models themselves to do a lot of the critical components of the AI R&D research cycle, we get a form of acceleration that can become quickly, very, very rapid and snowballing.

At this point, let's say China has caught up, or we're neck and neck, or maybe even they're slightly ahead, the incentives to sabotage each other's deployments of these models steeply increase. And we don't know at this point. If you are the leading US effort, say, and you get sabotaged, can you still catch up?

How long does it take you to get back to the point where you're competitive with the adversary? It is entirely possible that if, at the critical time point of when automated AI, R&D takes off, if you get sabotaged, you may have permanently lost. It might be days, weeks, months. We don't really know what that time is, that you would have to still recover and still be able to be competitive. So I think this is something we can't afford to be in that position that we are vulnerable to that degree of sabotage.

I think it's also worth mentioning that at this point, time point, it would be very likely that the AI models that are that capable are one of the most valuable resources on earth. And the incentives to attack are correspondingly high when it comes to sabotage. Even covert sabotage that is very, very hard to detect and maybe only has a gradual effect initially can lead to the type of underperformance and using up of resources that can set us substantially behind.

Also, not to forget, even if China has caught up, there exist other highly cyber-powerful nations out there who might still continue to have a whole range of incentives to attack these systems. So that's all very grave. But what does it actually take to secure data centers and weights at the SL5 level? The truth is this is really, really hard.

It can involve steps that go towards, for example, air-gapping. And an important principle to consider here is that the more complex your system is, the more attack surface you are opening up. So what we really need to do to reach SL5 is think about what are all the ways that we can radically reduce complexity in our systems. This is a massive infrastructural effort and I think it needs to happen in one to two years.

Unlike security level four and below, security level five will require coordination across the industry and government to work together to build a robust solution. All the usual suspects and further ones need to be collaborating to get this right. The AI labs themselves, the hyperscalers, the US government, et cetera. Beyond that, there are a lot of components that need to go right within SL5.

We're dealing with types of personnel security which need to also consider the AI models themselves as insider threats. Beyond that, we need to engineer components of supply chain, physical security and info and cybersecurity to have the kind of compensating controls in place that can account for the fact that we don't have time, we don't have time to onshore the supply chain.

We don't have time to work entirely with a US citizen staff. A couple more examples of the kind of constraints the AI industry has for which I believe the typical government processes may not work. For one, there is an extremely high proportion of non-US citizen technical staff, maybe also in critical positions that cannot be replaced. I want to note I'm naming informal numbers here that are based on my own assessment from engaging with these companies.

For example, xAI I think appears to have maybe even around 70% East Asian technical staff and I've heard a rumor that apparently there are so many that there are entire teams within these companies that don't bother to speak English because it's none of their first language. If we are trying to compare China and the US from a perspective of who has the more talent, China just has a much larger talent pool available. So I think we in the US we need to treat as assets anybody who wants to be on Team USA and can clear some bars of new systems that we have to come up with.

That brings me to the next point. US clearance processes are too slow to be workable for the AI industry. We need something else. Also, the types of extreme information compartmentalization that is typical for most insider programs in the US would incur prohibitive productivity costs.

The industry needs to be able to continue innovating and continue moving at a competitive pace. In addition, co-locating data centers and engineering staff getting all the frontier engineers to move to the desert, that would be extremely hard to pull off.

None of them want to do that. So I think what we need to do is reuse the existing capacities of the government and adapt them where we can, build novel solutions where we must. What are some actionable takeaways for policymakers? For one, I think there's a lot of room to create incentives for these companies to adequately and speedily invest towards reaching SL4 and 5 security.

For example, perhaps you can tie access to cheaper power towards reaching a certain security level, or you could enable building regulations if you reach a certain security level. Or sorry, exemptions from building regulations. In addition to that, I think there is also room to support efforts to get to SL4 or 5 optionality through funding, red teaming of efforts by the IC community and also other types of regulatory exemptions for AI security specialists and governance researchers.

I think there is also a really large scope of things that can be done. For one, if you can join a AI lab security team, I think they are the ones that in the end need to be deploying a bunch of this stuff. Also work for other external efforts that are trying to create the optionality for this type of security and make sure you're doing things that the AI lab security tech leads say they need.

There's also a ton of open research and engineering problems under SL5. Questions like can inference be continued at all at SL5 or is it all just closed at that point? What novel threat vectors will AI insiders pose? What do containment and monitoring systems look like at that point and can they be effective mitigations and also, what would a radically complexity-reduced AI training stack look like and how can we get there in one to two years?

It's hard. I think we're probably about three years late, but it's not over yet. I think we can still make it if we try really hard and for sure we can make it a lot more expensive for adversaries to attack us. The majority of gains from AI are in the future if we secure it.

Thank you for attention and I would love to take some questions.