Building safety tools no one can take away with ROOST’s Camille François

This transcript is generated with the help of AI and is lightly edited for clarity.

//
CAMILLE:
Today, 1 in 25 American teenagers says that they have had a deepfake nude produced of them. I’m not saying 1 in 25 tell you, I know somebody to whom this has happened. One in 25 tell you, yes, this has happened to me.
If you want fast collective innovation, if you want people to have access to the tools they deserve, if you want people to trust the things that they run their lives on, give them good shared open source foundations.
Policy without tools is basically poetry. It looks great, you can recite it, but you can’t really protect people with it.
//

ARIA:
In June 2018, Twitter bought a trust and safety startup called Smyte and abruptly cut off its other customers. Platforms like GoFundMe and TaskRabbit lost their anti-abuse tools overnight. It exposed a bigger problem. Safety infrastructure was concentrated inside a handful of companies, leaving smaller builders dependent on tools that they didn’t control.

REID:
Dr. Camille François has spent her career investigating online harm and building ways to combat it. In February 2025, she launched ROOST to make safety tools free, open source, and available to anyone building an online space.

ARIA:
As a kid in France, Camille wanted to be a space baker, baking croissants on Mars. She’s chalked that up to being French. Today, her ambition is to make building safely as accessible as building itself.

REID:
Her colleagues describe an anti-authoritarian streak, drawn to frontiers over fiefdoms, that may help explain ROOST’s approach: sharing technologies that others have kept proprietary and giving communities the tools to protect themselves.

ARIA:
Camille François is the founding president of ROOST and a professor at Columbia University’s School of International and Public Affairs.

REID:
Welcome, Camille. Your colleagues describe you as having an anti-authoritarian streak, more pirate than institution builder. Does that sound like you? And what were you like growing up?

CAMILLE:
I’ll talk to my team about that.

REID:
In an authoritarian way, I hope.

CAMILLE:
No, yes, of course I recognize that description. I do think that I am able to work with and within institutions. But yes, I like the energy of doing something together in a collective, maybe with a little bit of a pirate spirit. And when we were preparing for our very first ROOST team retreat, I actually found a book, which is great, called Be More Pirate, that looks back at the history of the glorious days of piracy and thinks through, okay, there are institutional lessons to take from those days, and there’s a way to rethink that moment in time as a moment in time that brought a lot of advances in equality and democracy. And so maybe there’s a different lens we can apply to our understanding of piracy.
But jokes aside, I think that I’ve always been part of the open source and free software movement. When I was a student, I founded Students for Free Software France and then sort of served on the board of the global organization. It’s actually how I met my husband. And that organization no longer exists, but back in the day, we referred to it as Baby Pirate Hacker Club. So yes, I can see where that comes from.

ARIA:
Yeah, exactly. I mean, anti-authoritarianism, I’m with you. That’s great. And it seems like everyone is talking about online safety, privacy. You were way ahead of the curve on this. You were building ROOST, you mentioned, which is a nonprofit that makes online safety tools free and open source. For someone who is hearing about this for the first time, who are these tools for, and what do they help people do?

CAMILLE:
Yeah, they’re for everybody. And ROOST is a nonprofit. It’s also a movement. And so we do this with a lot of people. We do this with large AI labs, but we do this with small builders. We do this with philanthropic organizations. We do this with many different types of institutions. And essentially, if you think about safety, we have at least three problems. The first one is, I think it’s fair to say that tech and AI have progressed faster than our ability to keep people who interface with these technologies secure, notably when it comes to kids and children. And so we have an innovation gap, right? We need to fast-forward our innovation and progress on safety tech. The second problem that we have is sort of a distribution gap, right?
There are a lot of safety tools that are okay, but people don’t have access to them. And there are many more builders who are building new applications, new websites, and who want access to the tools to do things right, to protect their users, to protect their communities, to do safety by design, right? That’s a real problem that we have. It’s nonetheless good news, right? That means there’s more appetite for safety. And then, of course, the third problem is there’s a little bit of a crisis of trust when it comes to safety technology. It’s because it has long been very opaque, right? So people tell you, like, hey, we’re using technology to keep you safe. And you’re like, okay, what is it? Can I see it? And it’s like, no.
And so of course we can change that model here. Open source really is a method, right, where we can accelerate innovation, make sure everybody who wants those technologies has access to them. And then, of course, where we can say, hey, here it is, here’s a safeguard. Go audit it, modify it. If you don’t think it works for your use case, go make it work in your language, your context. And then if you make a better version of it, just go distribute it or sell it, right?

REID:
You’ve spent a number of different years both as a pirate entrepreneur and professor, and then also years investigating online harm, from work on informing the Senate on disinformation. Was there a moment that convinced you to build ROOST? Was there a kind of an, ah, this would be a way of changing this ecosystem? What was that moment, and how did your experiences lead you there?

CAMILLE:
I don’t think there was a moment. I think it has been slowly building up, and frankly not just for me, but for the field. And if I think about my own trajectory in safety, of course I have taken from all those different experiences ingredients that you find in ROOST, right? So when I was studying information operations, one of the things that you very quickly find is information operations are cross-platform, right? And so you find them spreading on smaller bulletin boards, weird little forums, apps that nobody’s ever heard of. And when you sort of unravel this, you have a lot of people coming in saying, oh, we did not want to use the school board in this way. Help us, right?
And so you sort of understand that any sufficiently smart and motivated adversary is going to take advantage of the fact that there are many surfaces and smaller surfaces, and not everything is protected. Of course, that matters a lot for the theory of change at ROOST. When I was at Niantic, that was a really formative experience because I had a great CEO, John, who was very supportive of our safety efforts. And of course, Niantic had Pokémon GO. A lot of kids play Pokémon GO. And when we said, okay, we’re going to build a really good, robust infrastructure for safety, I went and I sort of knocked on doors and looked around at, like, what’s the state of the art? And I did not like what I found.
And the state of the art was like, yeah, what we do is everybody builds their own half-baked stuff, and then you buy a few good vendors, and then you duct-tape everything together, and that’s what everybody does everywhere. So I was like, okay, well, that sounds nuts. There has to be a better way. I was coming from more of a cybersecurity background where, of course, there’s a different approach, right? Like, there’s a shared infrastructure of open source tools across the field, so that you can focus your resources on building the piece that makes sense for you, but you don’t have to sort of fully reinvent everything from scratch all the time.
And so I started thinking about, okay, well, let’s try to figure out if those tools that everybody’s sort of cobbling together, are they the same for everybody? Like, are we doing this because people have such unique needs? And so I started interviewing a lot of safety engineers across the field, and they were like, absolutely, they’re the same. Most of the infrastructure is steady. So we did this project to sort of document what are the core components of safety infrastructure. We published a piece called the DIRE framework that stands for detection, investigation, review, and enforcement. But really what engineers were telling us is, yeah, we get staffed on something, we build 30% of a workable thing, and then we get assigned to another project. And so everybody’s sort of constantly rebuilding basic foundations, which is really frustrating.
And so we said, okay, great, now there’s an opportunity to change the way we do safety in tech, to sort of rewire the way we do safety in tech.

ARIA:
I feel like there are a lot of other issues and discussions around AI, but a lot of people are seeing AI as sort of social media 2.0. We got social media. We were so excited. There were these harms, some of them to children, some about misinformation, that we feel like we didn’t sort of attack soon enough. And people are worried that it’s going to happen with AI. Like, do you think that if ROOST had been around 10 years earlier, the landscape would be different? And, like, where do we go from here?

CAMILLE:
I actually think about social media as an exception to the rule of how we build tech, as a blip in the timeline. That’s just a bizarre happenstance, right? Because if you take the long arc of tech, the way we built the internet, the way we built the foundation of tech around us, is on open protocols and on open source software, right? TCP/IP was an open protocol. Tim Berners-Lee donated the web to humanity, right? And so the reason why we can innovate and then keep tech relevant and keep tech responsive to our needs as a species is because we do this in the open. And then there’s a sort of small era of social media where we kind of forget about all these lessons.
And it’s bizarre, but it’s a fact that we’ve inherited all of the safety tech that we have from that blip in the timeline, from that era. That’s why it’s brittle, that’s why it’s hyper-centralized, that’s why it’s opaque. And so what we’re doing with ROOST, frankly, is sort of bringing back all those normal principles of how we build good tech into the world of safety. We’re not really inventing something new, rather sort of, like, getting back to—

ARIA:
The roots of the internet.

CAMILLE:
Exactly, right. Changing the approach to something we know works, which is, okay, if you want fast collective innovation, if you want people to have access to the tools they deserve, if you want people to trust the things that they run their lives on, give them good shared open source foundations.

ARIA:
And do you think the leaders of the frontier labs are learning from any of those mistakes? Or, like, what mistakes are they still repeating?

CAMILLE:
I think some of those leaders are building these new open source foundations with us. So last year, for instance, we did a big effort with OpenAI. Our team worked closely with their scientific and engineering team to take one of the core pieces of their safety stack, a model that was doing a lot of moderation behind ChatGPT and then Sora, and to actually share it with everybody, to put it out in open weights. Now, that’s significant for a number of reasons. But the first one is, prior to that, it’s not like you didn’t see big companies share safety innovation, but a lot of that sharing looked like, oh, we’re going to make this available to everybody. Usually that lasts for the duration of a really good marketing campaign.
And then, of course, priorities change, and then people move on, and so things are not maintained. People are struggling to get access, things sort of fall behind, and then, of course, at any moment they can pull the rug, right? They can say, actually, we changed our mind. But open source isn’t that. Open source is a one-way door, right? So when OpenAI says we are actually going to share this model under an Apache 2.0 license and we’re going to put it on Hugging Face, that means if tomorrow they change their mind, they can’t get it back, and so people know they can build on it, right? So we did that, and they made that statement at the time saying that they believe everybody should be free to study, modify, and use critical safety tech.
Which is kind of fun, to see OpenAI quoting Richard Stallman, but I love that. And after that, we also did a really nice hackathon working with both OpenAI and Hugging Face. So we invited people, saying, okay, look, this model has been shared, come test it. And so we saw people who were really excited to immediately use the model. For instance, today this model is used by Wikimedia, right? We should also keep in mind that large, important tech nonprofits also deserve safety tools. But we also had a bunch of people come and break the model, which was really fun to see. I remember there’s a team who I think was building recipes for meth and ecstasy in Latin. So, yeah, that’s sort of like what we—you know, we see some of those large labs being like, okay, yes, we think we want to tackle safety a bit differently now.

REID:
I want to go into a specific area as an example in a second, but let’s first come back a little bit.

CAMILLE:
I’m not giving you the recipe of meth in Latin. Don’t, don’t insist.

REID:
I want it in French. One of the things, when you think about—I think there is a lot of fundamental importance about open source. As you know, I was on the board of Mozilla for, like, a decade, and LinkedIn actually open-sourced a whole number of projects. And I think it’s a generally good thing to do. I think it’s also good on certain patterns of safety and security, and I think the foundation of the internet is on it. I don’t think it’s actually only social media that creates the more turbulent stuff. I mean, we have the mobile internet with Apple. Now we have, like, kind of what’s going on behind YouTube, you know, within Google, and so forth.
So I want you to kind of elaborate a little bit on what does this weave look like, and how do we bring—because it’s not a disagreement; open source is an important thing here—to elaborate this point of view. But it’s like, given a complex environment, what is the role that you see open source doing? And then I’ll get to a specific commercial example question.

CAMILLE:
I think here open source has a unique ability to help us, as I said, reshape how we think about safety, really accelerate innovation, make sure that a lot more builders have access to the tools they want and need, and enable a system that can be scrutinized and be adapted to a lot of little communities. When you think back to the social media era, we talked about the sort of perils of hyper-concentration. There’s another peril, which is this idea that I think we held a little bit too long, that we were going to have big global public squares with rules that work for everybody. Well, I don’t think this panned out exactly the way we wanted.
I think the reality is you’re going to have a lot of smaller communities that have rules that make sense for them, and that’s important. That’s pluralism, right? That’s the world that we want to live in. If I think back about the difference between today, for instance, and, say, 10 years ago—as you said, I’m a professor. Ten years ago, if I looked at my students’ online lives, they were quite concentrated on a few platforms. So for instance, they used, back in the day, what’s called Facebook, you know, Facebook for schoolwork and also communicating with family and all that. But today I think what we see is people use a lot of different, smaller platforms for a number of things, right? They may actually have a WhatsApp group for something, a Signal group for something else. They might use a Discord server to coordinate about something, and then, you know, whatever, right?
So this multiplicity, I think now, of online spaces, both in social and in AI, also means that we need to be able to give all of these communities the tools that they deserve to shape what safety means for them, right? It’s also bringing back pluralism, which is something that maybe we had lost sight of during the social media era.

REID:
Because one of the things I know you’ve tracked is, you know, Twitter bought Smyte in 2018, immediately cut off all the other customers, demonstrated how brittle this is when you have a commercial system. So ROOST is an offset to that, in part. What are the things in terms of—is ROOST the only one that’s doing this? Are there more? You know, is ROOST trying to encourage things like Smyte, or other things, to be open?

CAMILLE:
Yes.

REID:
Like, what’s happening? How does the dynamic play out? That’s part of, like, the specifics within a commercial environment.

CAMILLE:
Okay, so let’s talk specifics of it. I love the story of Smyte. So Smyte was a little startup that provided safety technologies to a bunch of companies. One of them was Discord. And Smyte, which in fact is a rules engine, got bought by Twitter, because Twitter was like, yeah, that’s very helpful for what we do. I get that. And then Twitter shut it down. And so a number of companies, including Discord, were like, oh, we can’t protect our users and community. We just got the rug pulled super, super brutally from under our feet. And so they did what they had to do, which is they did the very boring work of rebuilding everything from scratch. What they have rebuilt from scratch, they have donated to ROOST.
It is the foundation for one of our core software projects, called Osprey. Because Discord was like, nobody needs to go through that again. I am going to open-source that rules engine. And the second they did that—right, we worked with their engineering team, because actually it takes a minute to open-source an internal system—Bluesky came in and said, oh, that’s great, I want to use that. So they started using this, and as a result, they started improving this. And so Discord said, okay, well, we want to use the branch that everybody is improving. We want to benefit from the collective innovation of everybody who uses Osprey. And so Osprey is this amazing story of, like, this is what the industry learned from Smyte, and this is what Discord decided to donate.
And now this project, Osprey, is actually used by many different platforms and organizations at scale to investigate fraud and scams and coordinated harassment. And it’s great to see it take on a life of its own.

REID:
Yeah, yeah, it’s awesome.

CAMILLE:
Yeah, it’s also fun for Discord because, well, as I said, they benefit from the collective innovation, and then they were able to spot really good engineering talent too. And so it becomes fun because all that really good engineering safety work also becomes visible, right? So it’s morale-boosting for the people who work on this. And then also it’s helpful for careers. Yes, absolutely.

ARIA:
I mean, every engineer wants to work on something that millions and millions of people are using. That’s the benefit of being able to be part of that community.

CAMILLE:
Yeah.

ARIA:
And I think it’s also, you know, Discord—it’s really important to have all the different actors in the ecosystem. You need the researchers, you need the professors, you need the people who are in the companies. And, like, Discord was uniquely suited to be able to do something because they had had this rug pull. And we talked about you being a pirate. So you’ve mostly worked sort of from the outside, trying to keep it, you know—and around. We’re only talking about pirates here. But you also were inside of Niantic, as you mentioned, you know, accountable for outcomes in real time, you know, at an actual company that’s pushing deliverables. What did that teach you that sort of the researcher being outside of a company couldn’t have taught you about this space?

CAMILLE:
I think it taught me how much tools matter. And if you take a step back and sort of look at the overall conversation that we’re having on safety, there’s been a lot of discussion on rules, right? What should be acceptable and not acceptable. And I understand that focus, because I think rules are really the most visible part of the pyramid, right? It’s the thing that we can all look at and sort of negotiate together. I think I learned at Niantic that policy without tools is basically poetry, right? It looks great, you can recite it, but you can’t really protect people with it concretely.
And so, the importance of those systems, the importance of infrastructure, the importance of having multiple systems that you know will be interoperable and will work together, because a lot of important context that you need to protect people also gets sort of lost in translation when there’s a handoff between one process and another process, maybe one internal tool to another internal tool. I think that I got a unique sort of very up-close and personal perspective when I built a trust and safety function.

REID:
Yeah, actually, tooling is really important, and it’s one of the things that academia usually doesn’t get, because it’s not quite the way that academia operates. So that’s an excellent bring into the academic environment generally. Not just in safety, not just in open source, not just in—right.

CAMILLE:
And you know, ROOST was incubated at Columbia University, right? At the Institute of Global Politics. I think we forget that academia can also incubate amazing tech nonprofits, right? We look at big business schools and all the startups they incubate, but there’s actually a nice tradition of amazing tech nonprofits being incubated out of academia. I think of, like, Scratch—love Scratch—coming out of MIT, right? So, like, universities here have a role to play. And yes, if they focus on boring things like tooling, we can go really far.

REID:
Yes, it will help. So I saw that your ABC framework was cited in Anthropic’s report on AI misuse. So first, briefly tell us about the framework, and then the question is, does generative AI break the premise of the ABC framework? Because now that you can have a single actor, you know, synthesize actor, behavior, and content all at once, does it work, or does it need to be updated? So first framework, and then that question.

CAMILLE:
The ABC framework has had a fun life since it was initially published, I think in 2019. Initially, I was looking at information operations and then sort of more broadly at moderation. And ABC here stands for actors, behaviors, and content. And it simply says that, essentially, trying to figure out who are undesirable actors on a platform—for instance, convicted child sexual offenders—what are manipulative behaviors on a platform, and what is undesirable content, it’s actually three different exercises and requires three different approaches that will draw on different disciplines, right? One will be closer to borrowing from spam, and the other will be closer to borrowing from cybersecurity.
So it was just trying to sort of put some order around those three things that we tend to conflate and focus on content, where you can see problems where there is both a problematic actor and a manipulative behavior and undesirable content. But also, again, it’s helpful to sort of separate those three things and think about the tools and approaches that are appropriate for each. It was very helpful in social media days to help sort of reorganize moderation functions, where a little bit of the cybersecurity mindset would also come in. And it was interesting because OpenAI, I think, were the first ones in the sort of AI era to also cite the ABC framework. And my talented colleague Ben Nimmo, who’s at OpenAI, said, actually, I think we can do completions for C.
And so he said the ABC framework still works in the context of generative AI. It’s just, like, actors, behaviors, and completions. Sometimes the AI will complete something, and you’re like, that is undesirable. And so I think he’s the one who sort of, like, brought it into the AI context. And I was also very pleased to see it cited in the Anthropic report.

ARIA:
Yeah, well, so if we’re digging into sort of this safety work, you have detection, investigation, review, enforcement. So one, which of those is hardest to get right? And then we’re talking about, perhaps it’s most important for there to be safety among the big actors that are serving millions, tens of millions, billions of people, but there’s a long tail. And so that’s also the problem, that smaller actors, you know, they’re under-resourced. And so when the under-resourced folks come in, what do they skip first? Yes, four things.

CAMILLE:
If you think of detection, investigation, review, and enforcement as sort of a pyramid, right—or an iceberg, rather—the exposed part, the thing that everybody sees and likes to comment on, is detection, right? So the question is always like, oh, do we know how to detect this harm? It’s a helpful, fun conversation. I’ll get back to it in a second. But in fact, once you’ve detected a harm, there are a number of things you have to do to be able to take action. And at ROOST, we focused on sort of building those boring tooling infrastructure pieces so that people have the means to take action when something is detected. So we talked about Osprey. Osprey is an investigation tool.
It allows you to say, oh, I can describe an activity that I don’t want on my service by its pattern. Please, in real time, go find the pattern and help me take action. Super helpful. For instance, if you want to root out fraud, or if you’re dealing with sadistic online exploitation and you have a signal that you want to be able to detect at scale. So that’s Osprey. And as I was sharing, this is coming from Discord. The other big tool that we have and that people are using is called Coop. Coop tackles review. Now, Coop came into ROOST through also a bit of a bizarre story, which is we bought a startup, which is a funny thing to do when you’re a nonprofit.

REID:
I love it. Yeah, nonprofit and university.

CAMILLE:
Exactly. There were two super talented engineers from Meta who had built a super little startup, which was a really good console that you can use for review and some enforcement, right? Because, for instance, say you want to detect if child sexual abuse material is uploaded on your services. If you do detect some, you actually have to be able to review to ensure that you know what is going on. And so for that, you really want wellness features. And then if you do find that indeed this has happened, you have an obligation to report it, if you’re in the U.S., to the National Center for Missing & Exploited Children, NCMEC. And that report is actually quite hard to do.
So we bought this startup, and we spent a lot of time working to open-source the core technology and to really add on to it. It was quite lucky for us that some of the former customers, like Notion, decided that they were in it with us on the open source journey and really helped sort of, like, take that seed, that base, and sort of grow it into what it is. We re-released Coop a few months ago as the world’s first end-to-end tool that allows for the detection, review, and enforcement of child sexual abuse material, fully open source. That’s amazing, because it means that now people who’ve always wanted to do this work can do it without having to knock on 99 million doors and pay lots of budget.
And what we’re hearing is that it has sort of met its audience of, again, people who were willing to do that and didn’t have the tools. I can go into many sort of stories for what this looks like in practice, but the point is we often underestimate the tooling needs that are not just detection. On detection, there are still things that we are not very good at detecting. At ROOST, we have the ROOST Model Community. It’s a really nice community where people come and actually share their safety models. So we talked about OpenAI doing this with us a few weeks ago. Mistral said, hey, actually, we also have a model that we use for safety. We think it’s quite cool because it’s bring-your-own-policy, it’s multimodal. It’s called Shieldstral.
We’re bringing it to the ROOST Model Community. That means they have to do office hours, and the community comes and says, okay, I’m testing it in this way. I think this is working, I think this could be improved. Everybody shares the feedback, and then we sort of continue iterating, right? There are also areas where nobody has a good solution, and that’s where we as ROOST invest in scientific partnerships with independent, rigorous sort of scientific and academic organizations, where we say, okay, here we’re going to need a big collective effort and push to actually bring some new forms of detection into the world.

ARIA:
Can I just ask—you know, you said, like, you know, thankfully Notion came on the journey, and there are benefits. Like you said, Discord, you know, they did it, but then they get the best of what everyone is doing because everyone’s building off them. Like, that’s obviously the whole ethos of open source. But are we, like, relying on the kindness of strangers? I feel like people are so worried that companies are never going to police themselves, or they’re not going to do it. How do you make it so every company says, hey, I’m going to be a part of this ecosystem?

CAMILLE:
Yeah, I want to figure out the answer to that question. You know, I think that leading by example is cool. I do think that when we were early, the few companies who said, hey, we’re just—we’re going to do that, we’re going to invest in this, we’re going to put our own resources, we’re going to share our own systems, we’re going to maintain the commons—I think they started something that continues to grow. Right now, we have more and more contributions to ROOST. I want that sort of race to the top, right? I want people to look at exactly the benefits that everybody gets by sharing this technology, the benefits that they get back in super talented engineering, the benefits that they get back in trust, right?
And of course, I want more people everywhere across the industry to say, hey, actually, I also want to share my tools, and I can see that there’s something that’s slightly better than what I have, so I’m going to adopt that. The other thing is we’re very focused on who else should be part of this conversation. I think Silicon Valley, for the longest time, has developed safeguards and safety systems a little bit in isolation. And when we think about, for instance, youth mental health, which is a big topic for us, there’s a lot of really important expertise that hasn’t been at the table. Working in the open allows us to work hand in hand with clinicians, with sometimes victims and survivors, right?
And to bring a lot more different perspectives than to just do this in isolation in a building in Mountain View or Menlo Park or whatnot, right?

ARIA:
Yeah.

REID:
One of the things I think people easily understand, if they pay attention, is why a safety commons in open source is useful in a cost structure, and in a quality structure, because multiple people using it and fixing it is good. Say a little bit about how it might also help startups succeed better. Yeah, right. Because part of the thing is, like, having them grow with it and have it as part of it is actually, in fact, something that isn’t just a, oh, I’m checking the box to make sure I’m not doing something bad, but it’s also helping me succeed.

CAMILLE:
Yeah, I love this question. Let’s take two different types of startups as a thought exercise. The first one is you’re a small startup, and you’re going to build a social product, and you want to focus on making that experience amazing. Maybe part of your social product is people can connect, and maybe there’s a chatbot, right? What I see from the younger builders, right, the younger generation that’s building products every day, is they have seen firsthand the harms of what it looks like when it can go wrong. They sometimes have personal experience, and they want to build safety by design. They actually don’t need to be convinced, right? And they think that safety can be an accelerant for them.
So they think that, you know, if they can get this technology, then on top of it, they can build a better product. There are also markets in which this is going to be the cost of doing business, right? So, for instance, in certain regulations, you have to go and filter out child sexual abuse material from your platforms or your data sets that are going to be used in AI. And being able to access the technologies to do this in a way that you can trust, frankly, allows you to compete in the market, right?
So it’s a pro-competition thing. It generally helps people build better products. What about safety startups? That’s also really interesting, because it’s difficult to have a safety startup, because, as we said, most of the companies that you’re going to try to sell into, or organizations that you’re going to try to sell into, have to pick things up from scratch. But if everybody has a shared foundation of tools, then it’s much easier for your clients to plug into the amazing technology that you are building. So we see already at ROOST safety vendors engaging with our work. We have this amazing company called Zentropi, who has been contributing fantastic open weights safety models. You can bring your own policy and tune it. And then we have another amazing startup called Musubi, who’s actually selling a hosted version of Coop.
Right. I’m laughing because in my head, I think about this project as Musu-Coop, because soucoupe in French means flying saucer. I’m reminded that this is not official marketing for it. But the point is, you have startups who are like, okay, this is really cool. We can build amazing services and technology on top of the shared safety commons, right? So if we do this right, the shared safety commons is not just a hippie idea. It’s also something that’s an accelerant to good business, startups, and innovation everywhere.

ARIA:
So clearly it’s important for safety for us to get it right. So quality is really important. But you also did mention cost, that this is actually cheaper. And so one platform estimates switching to Coop cut its tooling costs by 95%. Can you dig in there? Like, what made the old approach so expensive?

CAMILLE:
You know, I don’t know what they were using, but I think what matters is we are in an environment in which, sadly, people are, you know, reducing the budgets that are dedicated to safety. And so we want these resources to be focused on problems that are interesting and unique to those platforms. You don’t need to use your resources to rebuild over and over basic foundations to build a safety infrastructure on. And we want those resources to be focused on the great people who are going to do this work, on retaining the people who are going to think about the policies, who are going to investigate the incidents.
And so it’s true that in this context, giving everybody a shared layer to start from doesn’t mean that it’s totally free, in the sense that then you have to host it, right? But it allows people to focus their budget where it will have the most real impact on protecting people on a day-to-day basis, and we’re happy to enable that.

REID:
So one of the things that folks get when they understand open source on the defense is lots of different people can look at it, can rattle it. Improvements are shared across. And so there’s a set of things that, you know, traditionally, cybersecurity, et cetera, you know, the open source protocols for the fundamentals of the internet, they get better. But one of the things that is also a trade-off is the attackers can then also look at it.

CAMILLE:
Yeah.

REID:
So say a little bit about the principles about this trade-off between getting stronger defense by having a commons, but also how do you deal with the fact that the attackers can also see the commons, and what that might mean for helping people navigate this.

CAMILLE:
Yeah. Thank God this is not a question that we have to take from scratch. It has been structuring the entire field of information security for over a decade. And so I think here we can take some inspiration, frankly, on how things work on the security side. So if you are doing reverse engineering of malware, for instance, more often than not, you end up using open source tools to do the analysis. But because everybody uses the same tools, you can share the rules, right, amongst analysts. And saying we open-source the infrastructure doesn’t mean saying you have to share all of the rules for all of the investigative work that you’re doing, but it does enable defenders to collaborate and then go faster together. And so we see this in Osprey, for instance, right?
Saying we have a rules engine that everybody can run for free, in a way that’s permissionless, and you can adapt it to your infrastructure, doesn’t mean you have to publish all the rules for how you define and catch spam and define and catch coordinated harassment. But if you do find something that you think is a particularly gnarly activity that unfolds on your platform, and you have good reason to think that it’s also affecting another organization, and if they’re also using Osprey, you can share that rule. And then you fast-forward the collective defense.
And so it’s always that sort of thinking and muscle to develop between what needs to be in the commons to really lift up and enable all the defenders and, frankly, widen the circle of what is protected online and across AI surfaces, versus what is sort of, like, organizations’ own things to keep and share. And, you know, different organizations will have different preferences on that too, which is totally fine.

ARIA:
Actually, I have two questions for you. One: Coop, Osprey, ROOST.

CAMILLE:
Yeah.

ARIA:
What’s the deal? What’s the deal with birds?

CAMILLE:
Yeah, we like a good pun, Aria. You would be surprised by how many chicken puns we’re able to produce a minute.

ARIA:
So just random? No? Okay. No rhyme or reason. We just like puns.

CAMILLE:
We just really like puns. And also, birds of a feather. Okay, exactly.

ARIA:
Fly together for safety.

CAMILLE:
Exactly. Sometimes we also repeatedly use the same pun. So now every time I say, well, you know, it’s a chicken-and-egg problem, I will remind everybody that our Discord, where all those great puns are flying around, is completely open. And if you want to engage in technical safety discussions while making very fun puns—if you want to be punny—you can join us on the Discord. Okay.

ARIA:
Rife with dad jokes. Great. Glad we solved that. Second, perhaps more importantly, the conversation is sort of premised—you were saying that the problem with safety startups is, you know, Smyte gets acquired and then it’s shut down. And so there’s a rug pull for people who are relying on this. But ROOST is a nonprofit, and so it is also predicated on the idea that people are going to fund it. So one, how do you ensure that funding continues? And what happens if ROOST’s own funding dries up?

CAMILLE:
I feel like you were inviting me to make a very public call for people to come and fund our work. Please, please come and fund this work. Come and fund this work because it has a very, very tangible impact. And fund this work too because it’s a path to resilience. I love your question, because we are building this commons in a way that it can outlast the organization. The idea of our work is to bring enough people on board, to bring enough of a community, that innovation happens, more people have access to these tools, trust gets built. And if tomorrow we disappear, all of this stays there. That’s the difference with an open source commons, which has this resilience, right?
If tomorrow there’s no ROOST, people can absolutely still download and maintain and handle Osprey and Coop. Now, there’s a lot of work that we do to, again, like, catalyze those commons. Make sure there’s a common roadmap. I’m particularly excited about some of the scientific efforts we’re also building too, because now that we have the commons roadmap, we know where there are real gaps. And we’re able to join forces with institutions like Mila in Quebec or with the UMass Rescue Lab to say, okay, let’s focus and sort of plug these holes and gaps. But we are building with resilience in mind. I think that’s extraordinarily important in the current times, right?
The idea is really not to replicate a hyper-centralized system. That’s why we chose the open source way.

REID:
Makes sense. So I think the resilience that you’ve described is absolutely fundamental. One of the great things around open source, which is because it’s a collective digital commons, is it gets all the features of that, which is it never dies, it can be resumed, it can be steered in different directions, multiple players can engage in it. It’s part of the thing that is magical about open source. But there’s also a lot of different constituents. So there are nonprofits, there are universities, there are companies, there are engineers, there are random people.

CAMILLE:
Yes.

REID:
Say a little bit about kind of how these different elements come together, because I think that’s important for people understanding the resilience and the vivaciousness and the liveliness. And then we’re going to start shifting into how does AI begin to change this.

CAMILLE:
The magic of having an open source safety commons is that once the wheel starts turning, people start seeing how they can plug in. And so you have people who, for instance, don’t have technical skills but are really, really good at building policy. And as new big labs are saying, oh, here’s a safety model you can use, but you need to bring your policy, a few people are saying, oh, I can write a very detailed, clinically informed description of what suicidal ideation looks like in the context of an AI chatbot. And that happens to be super helpful for people who are going to use AI models for this. And I’m going to contribute this policy in open source in a policy pack. You, of course, have great engineers who are coming and are saying, I see this tool.
I have an idea for how to improve it. I’m going to submit an improvement. You have scientists who are saying, oh, I’ve always wanted to go and actually test some of these tools, and I’m going to go and I’m going to do this, and then I’m going to contribute a few insights from fields that have been traditionally left out of how we build safeguards that actually could benefit from this. And so the work of an organization like ours is to architect and organize that big collective will around these tools, is to make sure that everything is interoperable, right? Otherwise, you have, like, an island of misfit toys with a bunch of things that don’t work together, right? What we want is a good, strong, interoperable commons.
We have people using all of our tools together, right? So Matrix, for instance, is using both Coop and Osprey. And then you have people who only use one part of your system, right? So the work is to, again, orchestrate and organize this big collective effort that brings a lot of very diverse stakeholders. But that diversity is the strength of an open source commons. And I will say, in the team, we have amazing expertise from the sort of traditional open source side of things. So I have a fantastic CEO who used to run open source at Google, and so she understands how you sort of bring all these pieces together. Cassidy is our absolutely fabulous community manager.
He’s so good at sort of, like, you know, seeing two people go in very different directions and saying, actually, I know how we bring this into a community through office hours, we further the project. And so, yes, there is definitely a muscle to orchestrating that big collective and very diverse community around it. But this is what gives it the strength, and this is what gives us tools that are actually robust and better than people doing this alone.

REID:
Before we continue down the AI thing, I think one of the things that also may be helpful for a lot of people to hear is how does being open source help engage the kind of intense intellectual and scholarly work, research work at the university? Because that’s something that most companies don’t have any real access to. And that’s one of the many reinforcement loops that actually provides an advantage here.

CAMILLE:
Yeah, it’s a language translation question, right? So if you take this work and publish it with the right colleagues and in the right journals, then people who come more from the scientific and scholarly practice see it and say, oh, I understand the language of working through a data set and a paper. I can add my piece to this. And so, again, it’s about sort of, like, translating that work across different audiences to engage everybody. Right now, our director of product, Juliet, is doing this on the difficult problem of grooming and thinking about, okay, what can we learn from law enforcement who’s working on this?
What can we learn from the different, really talented academic teams that have been tackling this problem, and from the many corporations in the ROOST community who say, we have tackled this, we have some context, we have some data? How do we pull all of these pieces together that are currently quite separate, to really radically accelerate innovation when it comes to grooming detection? We started this work. Roblox, for instance, shared a first packaging classifier that’s focused on grooming detection, got feedback from the ROOST community—both people who are trying to use it and people who were thinking about grooming—took that feedback, and re-released something that’s faster, easier to use, and load better.
So it’s sort of doing this orchestration of, okay, let’s go and grab these perspectives, and let’s organize some collective push towards very focused, discrete, well-described projects we can tackle.

ARIA:
So coming into the AI age, AI is making it just easier and cheaper for folks to launch apps, chatbots, everything. And sometimes it’s small teams, sometimes it’s a one-person team, and they could have no background in safety whatsoever. How can these small teams make things safe right from launch?

CAMILLE:
I have a lot of thoughts on this. I have personal thoughts, professional thoughts. I suppose I should start by acknowledging that I am myself a compulsive vibe coder. I’ve vibe-coded a number of apps, including family apps. And I am excited by the fact that people are able to use AI to bring their own vision of something to life faster and faster. Our big vision at ROOST is that we need to bridge the gap between how easy it is to come up with a new app, a new service, and how hard it is to actually do some of the important steps of making it secure. I think where we feel that the most right now is on child sexual exploitation.
We have a lot of builders who are coming in and saying, we want to do this, we want to do this right. And we’re very excited when that happens. We’re like, okay, we will take you through those steps. But the reality, for instance, of removing child sexual abuse material from a foundational data set that’s then going to be used to train an AI model is that right now it is a hero’s journey, and you have to go and register with the authorities on one hand, and then go ask for a hash bank, and then go and implement, for instance, HMA to use this hash bank, and then you have to go and ask for a novel CSAM classifier and then implement it into something else. And so a lot of our focus is, how can we make that process much easier?
The overall vision is that one day it’s going to be as easy to launch an amazing website and app as to just type ROOST at safety, and it will just do it. Yeah. Or even, by default, those tools will tell you, hey, so fun that you’ve added a field in your app that allows you to upload, say, kids’ books into an app. Also, you may want to run a moderation layer here, and here’s why, and here’s how, and we’ll make it so easy for you. So we want to make that much easier. There’s a lot of work to do to do this, and we’re excited to work, again, like, with others to go in and tackle that gap.

REID:
So you started with the partnership with OpenAI and the gpt-oss-safeguard. So what would encourage more frontier labs to contribute to ROOST? And are the various frontier labs also contributing a bunch of tokens relative to the vibe coding side, which is kind of the equivalent of Mythos analysis, other kinds of things? Because that would strike me as a great thing to be doing. And I’m hopeful that there’s a positive answer here, but I figured I’d ask.

CAMILLE:
Let’s make it happen. I think we’re also starting to see consumer demand and regulatory demand for that, which I’m excited about. And one way to think about it is that’s kind of what we expect from other industries. So, for instance, if you wanted to buy a car, and I was trying to sell you my car, and I would tell you, it has airbags, I don’t know if they work really well if you’re short. Also, it has a seat belt. I think it’s made of paper, but I’m not sure. But also, I don’t really know what other people use. And you’re going to have such a great time. You would be like, I want none of this, right?
Because in an industry like cars, we have a set of standards and shared references and shared technology for what we know keeps people safe. In, you know, most cars that you can buy these days, they will use a three-point seat belt that was developed by Volvo. And in 1959, they said, we think it’s a really good model. Actually, we’re going to give all the other manufacturers the right to do this for free, and then we’re going to tell people how we do it so that we can continue improving on it, right? And that’s sort of our expectation for how things work. That’s also what enables innovation, right?
The reason why we have super fun cars that go really fast is because we know how to make them secure. And so I’m also hopeful that there’s going to be increasing demand from consumers and regulators to sort of tell people, like, hey, if you’re going around town telling everybody that you feel super satisfied with the way you keep people safe online and on AI, maybe you should give everybody those tools. Maybe you can join me in having this public call, right, to all of those amazing AI labs. I know they have more to contribute, and I can tell you people are hungry for this tech. What’s really fun when you have an open source stack is you discover people using your tools in production through different and funny ways. So I’ll tell you two stories.
There was a small fandom app, two people. They found Coop through LinkedIn, actually, I promise. And they installed it totally on their own; we didn’t know. And they deployed it. And in two weeks, they found 19 incidents of child sexual exploitation that they were able to report to NCMEC. And then they reached out, saying, thank you for enabling this work. We really wanted to do that. We didn’t have the tools to do it. Then we worked with NCMEC to say, okay, well, what does it look like to do this experience from the perspective of somebody who has a team of two? And I’m not saying an engineering team of two, I’m not saying a safety team of two people. I’m saying a team of two people, everything, two people.
And to think about, okay, how can we make this as easy as possible for builders who want to do the right thing? That teaches us that the coalition of the willing, of people who want to get this right, is so much bigger than the people who are currently equipped with the tools to do this. Which, again, is good news for our ability to quickly make a dent, a real dent, in protecting people online. Other story, actually also LinkedIn-related. Quite fun. One day, the lead contributor on Coop, the formidable engineer called Juan, receives a message from Atlassian saying, hey, we’re going to submit a pull request. Can you review it? And we’re like, Atlassian’s using Coop? We didn’t know that. That’s cool. They’re big.
We’re excited that large companies are staking some of their safety infrastructure on these tools, because then it’s exactly what’s going on: then they contribute to it. So it’s absolutely awesome to discover the use of your tools by amazing engineering teams knocking on the door saying, hey, actually, yes, we’ve been using it. Actually, we think we can improve it. Today, if I just think about Coop and Osprey—I’m not thinking about the ROOST Model Community and the amazing AI safety models, just Coop and Osprey—the people whom we know are using them in production reach around 370 million monthly active users across platforms and organizations. That’s our conservative count.

ARIA:
Amazing.

CAMILLE:
I think we can do so much more. There are so many more organizations to protect. What’s fun about this number is that it’s both large platforms and small organizations, two-person teams, or, for the first time, we also had a journalism team say, hey, we’re building our own community, we actually want it protected, and we’re going to use Coop.

ARIA:
That’s awesome. Well, so we’ve talked a lot—there’s a lot more to online safety than keeping kids safe, but we talk a lot about keeping kids safe. And Reid and I also talk a lot about AI companions. And so getting a little specific, when you’re thinking about creating an AI companion for a 15-year-old, what would a genuinely good AI companion look like? And what are the open tools that the developers would need to build one?

CAMILLE:
This is a hard question, and this is an urgent question. What we know from the usage data of AI companions and kids is that kids are increasingly turning to AI companions. And when we look at the types of conversations they’re having, they’re also discussing feelings and their own problems and seeking support. And so there are a lot of really important, consequential decisions to make, science to develop, safeguards to build around. What does it look like to keep those interactions safe? I am not part of those who think that we should just unplug older kids. We talked about this. I grew up playing with technology. I think it gave me a lot of meaningful connections and friends, and I think it gave me a meaningful career.
And we want kids to be able to play with technology and to engage. We also want them to do so in a way that’s safe. I think safe in this context includes things like products should be honest about being AI. We haven’t really crossed that bridge just yet. For instance, I think a lot about AI and toys. Today, if you’re going to sell a toy that has AI embedded in it, you don’t actually need to disclose that. And so you may end up having an AI toy in your house without anybody sort of thinking twice about it there. I think consensus around companions respecting age-appropriate boundaries and about supporting real-world relationships, right?
Ultimately, we don’t want those companions to isolate young people from their support networks, their friends, their families, and their community. I don’t think any of that is solved. I think this is a very active area of focus for the field, for ROOST, of course. And because this is not solved, because this is extraordinarily urgent, and because this is hard, that is why we’re doing this work in the open with scientific institutions like Mila. Our suicide prevention safeguard, we released it in July. And what we’re saying is we’re going to do this work in public and incrementally so that everybody is able to come join in and say, okay, let’s get this right, and we can sort of divide and conquer together to make sure that, again, we accelerate the pace of innovation.
Because, yeah, those usages are widespread, and I think we’ve heard firsthand and in extraordinarily dramatic ways the cost of getting this wrong.

ARIA:
So it sounds like you don’t think we’re quite there yet, that there is more sort of research and tooling needed before we’re able to safely deploy.

CAMILLE:
Yes, I don’t think that we know enough to say we absolutely got this one solved. I am very grateful for the builders who are trying to get this right. I’m even more grateful for the builders and scientists who are trying to get this right in public and with us. But I do think there’s a lot of work to do here. And, you know, again, I think that the usage data shows that this work is really urgent.

REID:
One of the classic challenges is if someone’s building something trying to include kids, you kind of have two paths. One is we’re just not going to include kids, or we’re going to include kids and we’re going to learn as we go. Probably we can only get success by doing the second path. But what would be some of the, oh, make sure you’re doing this, make sure you’re engaging this, make sure that you’re asking these questions, so that as you’re learning, as you’re going, you’re contributing back to everyone else, and we’re making the environment the right kind of safe environment for kids and elevating environment for kids that they absolutely deserve?

CAMILLE:
Again, I think that there are sort of some of those big questions, right? Does everybody know that there is AI involved, right? Do you have a good sense of what AI is involved, which safeguards are running around it? Do you have protections around some areas like self-harm? Can you ensure that, again, you’re not building a system that is going to lead to more isolation? These will also depend on the specific type of usage, right? You’re going to have somewhat different conversations again if you think about, like, AI embedded in toys, than if you think about multimodal AI, where you have a whole lot of different issues around images, for instance, right?
And similarly, building for a 12-year-old is a very different story than building for a 17-year-old. So I think we need to sort of engage with all of that nuance and take it with the utmost seriousness, and do this work rigorously, scientifically, in public, and say, this is what we know, this is what we don’t know, and this is the reality of how those tools are being used and, frankly, of how kids are being harmed by it today in those very specific ways.

ARIA:
So if we broaden out from just kids, just sort of looking at the whole ecosystem, and thinking about that—chatbots, language models, agents, like, they’re delivering a lot of services that used to be delivered by humans, especially in the safety world. So is there a specific safety capability where you’re like, that would make the biggest difference if we could bring that online? And how would you show that that works in the real world?

CAMILLE:
Yeah, I mean, I have a long list of CCC I’ll need. And again, I think there’s a distinction between problems that we kind of have solved for, but they need to be more robust, better distributed, battle-tested, invested in, versus problems that are a little bit more new, that we’re trying to wrap our heads around and thinking, like, oh, that is really bad. How do we tackle this? Yeah, I’ll give you an example of the latter, please. Non-consensual intimate imagery. Today, 1 in 25 American teenagers—

ARIA:
Oh, God.

CAMILLE:
—says that they have had a deepfake nude produced of them. Oh, I’m not saying 1 in 25 tell you, I know somebody to whom this has happened. One in 25 tell you, yes, this has happened to me. This is an enormous crisis, because, of course, those deepfakes are then used for extortion, for different types of humiliation, harassment, and harm. And thinking systematically about, okay, how do we tackle that novel usage, with the widespread scale of that problem and the very real consequences—that requires new creative thinking and, frankly, a whole-of-industry and scientific approach on where do we focus technical defenses, and make sure that they’re distributed as widely as possible to get ahead of the fact that this is an enormous wave of harm coming for teenagers.

ARIA:
I wouldn’t have even thought of that one. Now it’s going to haunt my dreams, so thank you.

CAMILLE:
Well, that’s why we have all the puns—because let’s get it solved. Let’s get it solved. Yes.

REID:
So, I mean, as part of that, how are the collaborations with the frontier labs and other AI labs? What kinds of things need to improve? What are the ways that government, civil society can help this as well? How do we get that in a faster and stronger kind of thing? Because, for example, solving the, you know, whatever—I don’t know what the exact term for the sexual abuse from deepfakes is—but, like, solving that is critical. And the kid doesn’t even have to be a user of it.

CAMILLE:
That’s right.

REID:
For that, right? It’s not even, like, yes, how they use it. It’s other people using this. So what’s currently going on, and what should we be doing to make it better?

CAMILLE:
I think this is a call for everybody to roll up their sleeves and work together. We need more attention to this problem. There are a lot of really good civil society groups who are doing these surveys. That’s why we know that this is happening, and who give us a lot of context on, okay, well, what does that look like for a kid? How does that happen? There are a lot of players who have technology to contribute that can help mitigate this harm at sort of the different areas where it needs to be stopped, right? And we want that technology to be contributed in the open commons.
We also do need philanthropic organizations to say, we’re going to take a bet on the fact that we can accelerate this work, we can do this in public, we can do this with the rest of the scientific community. And as I said, I think that there is a lot of expertise that lives in civil society and in the scientific community that for too long we’ve said, I know you don’t belong in the safety and safeguards conversation. If we want your input, you will sign a stack of NDAs, you will come to some office in Silicon Valley, we will hear you, and then you will leave, and you will never know what we took from your remarks. That’s not a good model.
That is not going to give us the pace and scale and excellence that we need in the face of these problems. So I think this is one where everybody can really bring something. I’m somewhat encouraged and also somewhat disturbed by all the conversations that we’re having right now on pacing the frontier and how everybody needs to band together to do something. There are also sub-frontier problems that are very urgent, very catastrophic, and that we absolutely should band together to focus on and solve.

ARIA:
Right. And I mean, so much of the conversation is also about global cooperation and what is happening around the world. And so you were recently speaking at the G7 in France, and you were talking about the importance of open source, digital sovereignty, child safety. Can you talk about why this is so important in the context of European builders? And so, like, why is it helpful, especially in that context?

CAMILLE:
Yes, I was invited to contribute to the G7, notably because of some of my academic work with a broader coalition of colleagues on defining what we mean when we say open source AI, because, of course, open source has had a traditional definition that we worked through a lot for software, but when it comes to AI, it’s a little bit different. And in fact, there are different ways to do open source when it comes to AI, and open source AI can itself take different shapes. It can be just open weights, or it can be everything is open, including documentation and data sets and safeguards. Or it could be many different combinations of this is open, but not that.
And you sort of have to think about the entire stack, and for each component of the stack, you have to think about what would it mean to open that? And then you think about, like, what am I trying to achieve, and which shape of openness gets me there? That conversation is very relevant to sovereignty because, of course, you have the European Union, but also, most recently, Canada with their big AI strategy saying, well, we want to be able to own, not rent, the different components of the stack that matter to us. That is also extremely relevant for safeguards, right?
Because, similarly, if you were going to legislate on child protection, you also want to make sure that people, your own ecosystem of innovators, have the means to comply with your law. Otherwise, the more we call for regulation, more rules on this topic, without investing in open source tools, the more what you’re actually doing is further concentrating power in the industry and closing down the market around a handful of incumbents who have the safety engineering tools and teams that are able to comply with those laws, right? So it’s critical to your ability to protect your citizens and to make regulation in this area. And it’s also critical to your ability to have, you know, independent and strategic and sovereign versions of your stacks.

REID:
Well, going a little bit more global, one of the challenges when we deal with a lot of the prospective harms is that, for example, you know, abusive images can be created anywhere, and so can distribution networks and all the rest. So what needs to happen globally? Right, because, you know, one of the questions is there will be image generators, for example, running in the vast majority of countries. There are already open source versions of them, or open weights versions of them. And there are nuances you were gesturing at there. So what should be thought of as some good basics or some good starts for the G7 coming together, for the UN—we’re here in UN General Assembly week. What would be the kinds of things that would be good things to do globally?

CAMILLE:
I think my first ask is, never use the idea that you can’t root out all of the problem to justify not taking the extraordinarily important incremental steps that really are going to save lives and reduce harm. Yes, there will be people doing this with models that we don’t have control of, in places that we won’t see. But also, when you look at the way the majority of these harms are being perpetrated, it is happening with apps that are being advertised on social media, distributed through app stores, and in contexts where there are actually so many different levers that we can identify and say, here, this is a good chunk of the harm that, if we focus and join forces, we can remove. So I’ll say that’s my sort of first, most important goal, which is, like, let’s get progress done.
Then we’ll worry about the sort of remaining part that we can’t get to. But there are so many lives we can save, harm we can remove, good we can do by focusing on all these incremental steps that we’re right now not investing enough in.

ARIA:
You’ve described your generation as internet optimists who believed cyberspace would spread democracy and expose corruption by default. We all heard this. We were so excited, and it came true. There have been, you know, many good things that technology, social media, the internet, et cetera, have brought about. We didn’t, though, predict all the harms. And, you know, we watched some of these beliefs get weaponized. And so is the optimism you still carry a conclusion from the evidence that you’ve seen, or are you just an optimistic person? Like, how would you say your optimism has sort of waned and waxed over the years?

CAMILLE:
There is somebody somewhere who could probably answer the question of what is optimism? Where does it really come from? And I would like to know. I think maybe in my case, perhaps it’s a personality trait. I am part of this generation who is extraordinarily excited about the power of technology and the internet. I remain extraordinarily excited about it. Over the years, I’ve had to work on violent extremism, terrorist use of these systems, foreign interference, hate mobs, suicidal ideation, child sexual exploitation. And so my colleagues often say that I am maybe the most optimistic person looking at the darkest corners of our technological life, sure. But I always see those reasons for optimism. The first one is that there is so much we can do if we join forces. We know what to do.
We just need to focus. And we are still in an environment where we can make very tractable progress by doing things that we know how to do. And so that gives me a lot of optimism, because then I know where to focus my action, right? There’s this question of, like, does action come first, or does hope come first? I actually think that hope comes from the action. The other thing is, I am very excited by the next generation of builders. Sometimes when we think about child safety, we can describe the next generation as sort of a big unified mass of youngsters to which tech is happening, but that’s not the case. They are also the builders of tomorrow.
They’re also very creative and fun, and there are a bunch of maker pirates within them too. And it’s fun to see what they do. There’s a hashtag online that says cyberdeck, for instance, where you can see young girls take Polly Pockets, and inside, they build mini computers, and then they run local AI models. That’s really cool. The next generation is building systems that they want. They’re going to build a technological environment that they want to see. And I think we need to bring them to the table not just as victims, but also as builders and shapers of our technological futures. And everything that we do at ROOST, we also do for them. And it’s super optimism-producing to see them knock at the door and say, hey, I’m going to build this weird app.
I need all these safety tools to do it. And we’re like, yeah, go, go, go. You don’t need our permission. Go do your app. So, yeah, that’s exciting. And that produces optimism.

REID:
So one of the kind of phrases around this that I really like is cyberpeace, and it describes not just fewer bad outcomes, but actually a kind of a healthy and dynamic and growing system, kind of equilibrium. And you’ve talked about it some today, with pluralist auditable safety, you know, kind of ecosystems, communities getting more nuanced and detailed and specific to their community, kind of agency. What do you think are the most important things to help continue to accelerate on that path? Like, what should people understand about what kind of ecosystem we’re trying to get to? Why should it be kind of almost like a, what? You’re not using these tools, you’re not participating in this—why not? Right. You know, it’s kind of the why—the why not.
What do you think it takes to get there?

CAMILLE:
So I’ve done my academic research and, ultimately, my dissertation on this concept of cyberpeace, and that is because I was amongst a group who was studying cyber warfare. I had two absolutely fantastic advisors who encouraged me and supported me to sort of look at the problem maybe from a different angle. And I was taken by the fact that everybody was very focused on defining cyber war, defining the threshold of armed conflict. And it’s, as I said, not unlike the conversations we’re having right now with the hyper-focus on the frontier. But there is all this great theory of positive peace building. What happens below the threshold? If we’re so intent on defining cyber war, do we know how to define cyberpeace?
Do we know how to invest in the mechanisms that make our online lives livable, that make room for thriving, that make room for cooperation? I think that the disappointing answer to your question is that some of that is the power of our focus. We get to decide where we want to spend our energy, where we want to spend our efforts, where we want to build, where we want to spend. And saying we care about making the sub-frontier, sub-war space livable, we care about tackling these harms that are immediately in front of us, is a decision we can make with our focus. And making sure that we can keep centering these in the conversations we’re having around AI, around harms, around AI safety, I think, is part of the problem.
We don’t need to have this opposition between catastrophic risk and immediate risk. I don’t think that opposition is particularly—of course, I understand why it exists, but we don’t need that strong of an opposition. We can make sure that we focus on both sides of the line and that we do both at once.

ARIA:
This is just reminding me, it’s like government works well when you forget that it exists. Like, none of us know that we’re getting tons of spam text messages a day, because that system actually just works. And some of the spam texts are just filtered out. We still get some, but most are filtered out. Like, all of these systems, when they just work, you, in a good way, get to take them for granted. It’s sort of like Maslow’s hierarchy. It’s like, if safety is the floor—that’s right—like, we have to have it exist before we can get to flourishing and before all these great things can happen. And so what does that actually look like once we have safety as the floor?

CAMILLE:
Yes.

ARIA:
And then what does that look like when cyberpeace is real for a teenager, a platform, a government? Like, how do we paint that picture?

CAMILLE:
I love that description. Like, yes, we want to become ubiquitous, and you don’t think about it, totally. And things are just more safe. And that is just the way it happens to be. And frankly, we have that in some corners of our online lives. How often do you think about the amazing work that Let’s Encrypt is doing? I don’t know, maybe you do. But generally, when people start a new website, they’re not going to their office thinking, I need to figure out cryptography, right? Like, no, we don’t.

ARIA:
Right.

CAMILLE:
We’ve managed to sort of, like, have building blocks and institutions that raise the floor for everybody. I think we can do this for safety. I think it will look different in different products because, again, we don’t know what people are going to build. What we know is there are fundamental things that need to be done in order for those users to be safe. Some of this we know how to tackle, and we need to make those technologies available to everybody. Some of this we don’t know how to tackle yet. We need to focus, make that scientific effort, build those new safeguards. And then, again, will this look like new, very fun chatbot interactions? Maybe. Will this look like less spam, hopefully less fraud? Please. That would be great.
Less child sexual exploitation? Absolutely. Fewer interactions with chatbots that lead to violent extremism? Yes, please. All those things, right? Like, yes, the idea is exactly what you said, which is, let’s make it ubiquitous so that it’s much easier to build from a shared, safe foundation.

REID:
All right, so let’s go to rapid-fire, and you can answer at any length. These are just the same questions we ask all of our delightful guests. So is there a movie, song, or book that fills you with optimism for the future?

CAMILLE:
Many. I will pick one, because I think that’s the game.

REID:
Yes.

CAMILLE:
So—

REID:
Or if you want to do two, it’s fine.

CAMILLE:
I am a Trekkie. I love Star Trek. I am a Trekkie of the Deep Space Nine category. And I’m not sure why, but it actually turns out that a few people around ROOST and in the ROOST community also are Trekkies. So I think that there is something about Trek inciting optimism about the future, optimism about our ability to collectively govern it at interplanetary scale. But, yes, when I think about our board, for instance, we have amazing people like Audrey Tang or Megan Smith or Maria Ressa, and all those people are also Trekkies. And so it has happened that we have very serious ROOST discussions, and then the end is like, okay, okay, good. Live long and prosperity.

REID:
Awesome.

ARIA:
What is a question that you wish people would ask you more often?

CAMILLE:
I wish people would ask, who should be at the table when we make safeguards? Again, I think we’ve inherited bad habits of doing this a little bit on our own, in the dark, in a way that’s very opaque and sort of siloed. And there is another way to build. And thinking about, well, who else has an interesting expertise to contribute on a very difficult societal problem is a question I wish people would ask more. And I think we will bring more people and, yeah, more perspectives.

REID:
So where do you see progress or momentum outside of—call it, you know, all the things we’ve been talking about, your industry—but, like, another area that you would like to shine a light on that inspires you?

CAMILLE:
Well, it’s interesting. I know you said another area, but for people who’ve been working on child safety, we are going through a very interesting moment where that conversation is becoming a kitchen table conversation. And so I am quite inspired and energized by the fact that many people around us are starting to think about, okay, well, what type of relationship do I want between young people and tech, and what is my piece of it? I’ll give you a very bizarre example. I recently became a mother, and as a result, my daughter goes to daycare. And they had a conversation on, well, we have Amazon Alexa in the classroom, but the wake word that we use is Alexa.
And in an environment in which this generation is going to grow up with companions, is that what we want? Because that’s anthropomorphizing the relationship to the computer. Now, it turns out—here’s a Trekkie secret—that one of the few other available defaults that you can use for the wake word is computer, like in Star Trek. And they really know that that’s cool. You can go and change it tonight. And so they were thinking about how to do their part in finding the right relationship between youngsters and machines, as everybody sort of grows up, again, surrounded by AI, and changed the wake word from Alexa to the very Trekkie computer. And I’m excited by the fact that many people are having this conversation at so many different levels, right—in classrooms, in boardrooms.
And this is becoming much more of a mainstream conversation.

REID:
Awesome.

ARIA:
In my house, my kids just try to say, hey, Google, in the craziest accents, at the loudest volumes, at the strangest tenors. So we haven’t quite—

CAMILLE:
No, that’s great.

ARIA:
We’re not at that stage yet, but this is super.

CAMILLE:
I think you have red teamers on your hands. I think you should send them to ROOST to go and red-team some safeguards.

ARIA:
They’re very into it. All right, our final question. So can you leave us with a final thought on what you think is possible to achieve if everything breaks humanity’s way in the next 15 years? And what’s the first step to get there?

CAMILLE:
Ubiquitous safety for everybody. Come roll up your sleeves and build it with us. We’ll get there one step at a time, together. Hear, hear.

ARIA:
Camille, thank you so much.

REID:
Live long and prosper. 🖖

CAMILLE:
Thank you.

ARIA:
Beautiful.

REID:
Possible is produced by Palette Media. It’s hosted by Aria Finger and me, Reid Hoffman. Our showrunner is Shaun Young. Possible is produced by Thanasi Dilos, Katie Sanders, Spencer Strasmore, Yimu Xiu, Aman Suri, Danny Garrison, Trent Barboza, and Tafadzwa Nemarundwe.

ARIA:
And a big thanks to Jeremie Ponak, Anne Bertucio de Hollembeak, Jacqueline Ortiz Ramsay, Danielle Hogan, and the team at Lighthouse Brooklyn.