Why AI governance starts with model makers, demands government control, and still requires everyday people to defend their own homes, schools, and businesses. 

This week has been big news for the AI governance world. I must admit, watching one of the most influential people in AI admit we have a global governance problem was a breathtaking moment. 

After months of swirls and intrigue from industry insiders about leaks, misuse, and AI swarm risks now described by Amodei in his own letter, it all came to a head this week with a startling revelation not only confirming our worst fears but, in fact, acknowledging that the major creators of the world’s most powerful AI appear to be collectively questioning this technology arms race. 

My Disillusionment & Restoration of Hope

As I read Dario Amodei’s utterly candid and truthful remarks, I was struck by many thoughts. First of all, it acknowledged that the unrest and guilt I have felt about AI was in fact, validated.

The disillusionment of working in an industry that I believed in and dedicated my whole life to, thinking I was improving lives, was now riddled with technology platforms that seemed to be destroying the fabric of community.

But after months of working in AI, I was angry and disillusioned by the industry’s treatment of people and lack of concern of misuse. 

Then I had a moment. I remembered Game of Thrones’ wall and the Night’s Watch. The wall was created to block “the others” in the books and “the white walkers” on the show. This entity that they knew was a massive danger to all humanity, but they really did not know its full capability. They only knew they had to be a watcher on the wall, an alarm, and a first defense to protect humanity. 

That thought made me pause and think: I need to be a watcher on the wall. I need to focus on helping us preserve this very powerful tool as the ultimate tool of humanity, not its downfall. I suddenly thought I could help defend this instead of feeling powerless. 

A Red Flag Waved by Anthropic

Dario Amodei’s letter was clear: it’s not that he does not have the best team. He hired the smartest people on the planet, but it’s the scale that is the battle. He was saying, there is still too much work to do. The system has become too complex and we need to take a collective step back as an industry and put in some framework, or governance practices. 

What is interesting is: it’s not about talent but process. This is not unique to AI. When powerful tools of humanity become this vital and can be possibly destructive to humanity, the innovation races ahead, and then there is a pivotal moment where the innovation has to pause in order to think of the process. There are a couple of prime industry examples: the commercial flight industry and the nuclear arms industry. 

In both cases, rapid innovation and accelerated timelines created powerful tools that have transformed humanity, but they both had the capability to destroy human life, one at a small scale and the other at a global scale. In both times, it was not only the US government that stepped in to regulate, but globally we understood we needed to work together to prevent mass destruction across borders. 

I think both analogies are needed. The commercial flight example is the model for small and enterprise business, and the nuclear example is truly the example for global governance when working with AI. 

The Model for Protected Innovation is Already Here

When aviation was discovered by two brothers from my hometown, the Wright Brothers, they changed the world. Their technology unleashed an industry that became one of our greatest innovations for community and global experiences since the invention of the printing press. But the changes that evolved in this case were followed by innovation quickly to make regulated guidelines to protect life every day. 

The FAA was established when air travel was no longer just ad-hoc adventurist but needed to suddenly translate to the everyday person and whole businesses. Its role was to protect, control, and regulate so that flight could be used by private business and grow a whole industry. They created the benchmarks and responsibility charter for all air travel. 

No one today would say that the FAA is impeding innovation in the aviation industry. In fact, with recent Boeing safety and oversight controversies, confirm it’s actually a front-line defense system. 

If air travel after the Wright Brothers had been left only to commercial viability and capitalist incentive, what would that industry look like today? We would not have a trusted global aviation system. We would have a patchwork of companies deciding for themselves how much safety they could afford, how much risk passengers should unknowingly carry, and how much failure the market would tolerate. And even with regulation, we still see what happens when oversight weakens or companies place business pressure above safety, as recent Boeing safety and oversight controversies have shown. If that level of governance is still necessary for aviation after more than a century of safety systems, how much more necessary is it for AI, a technology moving faster, scaling wider, and touching more parts of human life at once? 

When industries reach beyond having control of day-to-day business and are now deterministically in charge in the life and death of people, with impact in the millions, we have reached past capitalist free industry into devastating harm. 

The model creators carry the greatest responsibility because they built and commercialized these systems. Governments carry the fiduciary responsibility to force proper controls, oversight, and consequences onto the companies creating them. And only after those responsibilities are clear can we ask everyday users to carry their fair share by becoming first-line defenders in their own homes, schools, and businesses. 

The largest danger of AI, unlike nuclear power, which can be sealed off and be truly controlled by a handful, is that AI can be licensed and used by the many. This is the shift: when a technology that is as powerful as nuclear can be handled by anyone, we are at a true inflection point. 

And this is where section three of Amodei’s plan gets real. The fact that our government hasn’t already done more is not just alarming; it is a failure at a fiduciary level. When leaders are entrusted with protecting the public from known, foreseeable harm and they choose delay, political theater, or indifference, that is not a simple policy gap. In my view, that begins to look like malfeasance. And the blasé reaction from leadership in Washington this week reflects a leadership class that appears disconnected from the long-term consequences of this technology. 

The order of responsibility

So what to do? Well, first of all, we need to be honest about the order of responsibility.

1.) Model creators carry the greatest responsibility because they build, commercialize, and profit from these systems at scale.

2.) Governments carry the fiduciary responsibility to ensure there are proper controls in place to force model makers to govern what they create.

3.) Everyone Else Using AI We should also look at making sure we are teaching ourselves to be first-line defenders in our homes, in our schools, and in our businesses. 

It is not the sole responsibility of the AI model creators to set the governance. To be clear, do they need to own a lion’s share of governance and absolution. Should they have the most strict consequences for letting their creations’ actions? Yes, 100%. That is the role of an FAA version of AI oversight that our government needs to create. Europe has already seen these issues and issued a very compelling set of guidelines, including harsh financial consequences, which seem to be the only language that corporate America understands today. The stakes are just too high for there not to be real-world consequences. 

That being said, every one of us using AI should be thinking and educating ourselves about basic AI governance in our own lives.  

We should be asking:  

• What personal information am I putting into this tool?  
• Who has access to that information?  
• Can this answer hurt someone if it is wrong?  
• Am I using AI to make decisions I should still own myself?  
• Am I checking the output before I share it, publish it, or use it to make a real-world choice? 
• Am I using it in a way that is fair, secure, and accountable?  

The Critical Question to ask:

If this AI causes harm, can I honestly say I understood the reputational, societal, legal, and personal consequences of using it — and am I willing to be accountable for them?

We Are the Watchers

Governance cannot only live in boardrooms, legal departments, or government hearings. It must be taught in homes, schools, and businesses as basic digital responsibility. The everyday user may not be building the model, but they are still part of the system that gives it power. If we don’t teach people quickly that they are using a tool that has the propensity to hurt human life, even with the most modest intentions, we cannot expect anything but failure. 

We need to be able to report issues, flag concerns, and collectively patrol AI usage as a global community. Being able to crowd swarm governance is a way to help with the problem. In every disaster movie ever made humanity wins by the very things that we taught AI, with community we can control the danger. The answer is education and individual actions, in addition to government oversight and model creator accountability as another level to help the scale problem.

We need to be the Jon Snows and the watchers on the walls ourselves, for our own homes. We cannot leave it to a kingdom in the far north, our governments, or the model creators. We need to be the watchers on every tower, in every business, every school, and every home. Otherwise, the outcomes will be disastrous, and this time, the failure isn’t just a few hundred; it could be the extinction of our entire culture and species. 

AI Summary: This article argues that AI governance must begin with the companies building and commercializing powerful AI systems, with governments holding fiduciary responsibility to force proper oversight, controls, and consequences. Using aviation and nuclear regulation as examples, it explains why technologies capable of large-scale human harm cannot be left to commercial incentive alone. The article also calls on everyday users to become first-line defenders by understanding what they put into AI, checking outputs, protecting personal information, and owning the consequences of how they use these tools.