The Frequency

What If AI Doesn't Destroy Us? What If It Changes How We Build?

I'm not an AI safety researcher, and I'm not trying to predict whether AI will or won't become an existential threat. I'm looking at the question from a different perspective: what happens to the systems we build if the capabilities people are warning about actually arrive?

The Prediction

There has been a lot of discussion lately about artificial intelligence and the possibility that increasingly capable AI could eventually become an existential threat to humanity. Some researchers are putting surprisingly short timelines on that possibility, with some suggesting that something approaching human-level or beyond-human intelligence could arrive within the next decade and potentially create circumstances we aren't prepared to handle.

I'm not going to sit here and tell you they're wrong. I don't know. I don't think anybody really knows.

There are certainly reasons to take the possibility seriously. AI systems are becoming more capable, they are getting better at writing code, working through complex problems and operating with less human intervention. If that progress continues, it isn't unreasonable to ask what happens when these systems become considerably more capable than the ones we're using today.

What I do think is missing from some of the conversation, though, is the other half of the problem.

We spend a lot of time talking about what AI might eventually be capable of doing. We don't spend nearly as much time talking about what we are going to allow it to have access to.

As someone who has spent a large part of my career working with computers and networks, that distinction matters.

Capability Isn't Access

A computer can theoretically do an enormous number of things. That doesn't mean it can actually do all of them.

The same is true of AI.

An AI system could become incredibly intelligent, capable of writing software, finding vulnerabilities, analyzing enormous amounts of information and figuring out solutions to problems faster than humans. But intelligence by itself doesn't give that system access to your bank account, your company's servers, a power plant, a military system or every computer connected to the Internet.

Someone still has to provide the connection.

Someone has to provide the credentials. Someone has to give the system permission to execute code. Someone has to connect it to another system, expose a service, install an agent, create an API connection or otherwise give it a path from one place to another.

That is where I think the conversation gets interesting.

If we're willing to imagine an AI that is capable of finding vulnerabilities faster than humans can patch them, why wouldn't we also imagine humans changing the way we build networks because of that capability?

We've Changed the Internet Before

I remember what the Internet looked like when it was becoming mainstream in the 1990s. We were connecting more and more computers together, and eventually we realized that simply connecting everything wasn't enough.

We needed firewalls. We needed better authentication. We needed intrusion detection. We needed encryption, segmentation, endpoint protection and eventually much more sophisticated ways of determining who or what should be trusted.

The answer wasn't to turn the Internet off.

We changed the architecture around it.

That's an important distinction because technology has always evolved in response to the problems technology creates. When the threat changes, the engineering changes with it.

So if AI really does become capable of doing things that today's systems can't do, I don't think the logical response is simply to throw our hands up and say, "Well, I guess we're all doomed."

The logical response is to ask what the next version of the architecture looks like.

Maybe Everything Doesn't Need to Be Connected

We've spent decades moving in the direction of convenience.

We put our files in the cloud. We synchronize computers automatically. We connect applications through APIs. We remotely manage servers, access systems from our phones and allow services to communicate with one another without anyone necessarily thinking about the individual connections anymore.

For the most part, that's a good thing. It makes technology incredibly useful.

But there is a tradeoff.

Every connection creates another possible path.

Take something as ordinary as synchronizing files through a service such as OneDrive. From the user's perspective, it is incredibly convenient. A file exists on one computer and is available somewhere else almost immediately. But behind that convenience is a whole series of permissions, authentication systems, network connections and automated processes.

Now imagine a future where autonomous AI systems are significantly better at discovering weaknesses in those systems than humans are.

Would we really continue designing every system so that everything can communicate with everything else?

I'm not sure we would.

In fact, I think we may eventually go in the opposite direction.

The Internet of Separation

Maybe the future Internet isn't going to be one giant, relatively interconnected ecosystem where everything can eventually talk to everything else. Maybe we're going to start seeing much more deliberate separation between systems based on how important they are and how much risk we're willing to accept by connecting them.

Your home computer and your refrigerator might have one level of connectivity.

A company's ordinary business systems might have another.

A hospital's critical infrastructure might have another.

And a system controlling something truly critical might have very little direct connectivity at all.

Instead of asking, "How do we securely connect this system to the Internet?" the question may become, "Does this system actually need to be connected to the Internet in the first place?"

That's a very different philosophy.

It doesn't mean technology goes backward. It means we become much more deliberate about where we allow technology to go.

Building Layers Instead of One Big Wall

As spending over 25+ years in technology, this is naturally where my mind naturally goes.

I don't think the answer is going to be one magical firewall that an AI can never get through. Security has never really worked that way.

The better approach has always been layers.

Imagine a critical server sitting behind one security boundary. Behind that is another segmented network. The server itself has its own protections and is only running the software and services it actually needs. Communication is restricted to specific destinations and specific purposes, and anything unusual becomes something that gets investigated rather than simply being allowed because the connection technically works.

Now take that idea further.

If an AI somehow compromises one system, that shouldn't automatically give it a path to the next system. If it compromises that system, there should be another boundary. If it attempts to communicate somewhere it has never communicated before, something should notice. If a file suddenly appears that doesn't match what the system is expecting, that should become an event rather than just another file.

The objective isn't necessarily to build an impenetrable wall.

The objective is to make the attacker keep running into walls.

That's a much more realistic way of thinking about security.

Some Things May Need a Human in the Middle

I can also see us becoming much more deliberate about how information crosses those boundaries.

A critical system might not simply accept a file because somebody has the correct password. A transfer could have to come from an approved source, match an expected type and size, pass scanning and verification, and potentially be approved before it ever reaches the protected system.

The same thing could happen with communication leaving the system.

Maybe the system is allowed to communicate with three specific services using specific protocols. If it suddenly starts trying to communicate with something completely different, the connection gets stopped and somebody gets an alert.

The technology to monitor this kind of behavior already exists. The difference in an AI-driven world would be the importance we place on it.

For truly critical systems, we may even decide that some actions require a physical human presence. Not another password. Not another checkbox. A person physically has to be there before something can happen.

That might sound excessive today.

It probably wouldn't sound excessive if we were dealing with a system capable of autonomously finding and exploiting vulnerabilities faster than humans could respond.

The Human Being Is Still Part of the System

There is another uncomfortable part of this conversation that I think gets overlooked.

Humans are part of the threat model.

Even if we build incredibly sophisticated AI safeguards, someone still has to develop the AI, configure it, connect it to other systems and decide what permissions it gets.

That's where I start thinking about something that sounds almost like science fiction but really isn't that strange when you think about it. If we're going to put increasingly powerful autonomous systems into the hands of people, should we eventually be thinking more seriously about the people who have access to them?

We already do this in many other areas where the consequences of misuse are significant. We have background checks, security clearances, separation of duties, auditing and restrictions on who can access certain systems.

AI may eventually add another layer to that conversation.

The old science-fiction idea of giving machines rules about not harming people was always really about the relationship between humans and machines. The machine itself wasn't the only problem. The question was what happens when a machine follows instructions that humans gave it, especially when those instructions interact with a complicated real-world system.

The more powerful the technology becomes, the more important the person holding the keys becomes.

What Happens If Something Gets Through?

This is probably the part of the AI discussion that interests me the most.

What if the people predicting an extremely capable AI are right?

Let's actually play pretend for a minute.

Let's say we eventually have an AI capable of discovering vulnerabilities faster than humans can, writing its own software, operating autonomously and attempting to move from one system to another.

Now what?

My first thought isn't that we should give up on computers.

My first thought is that we should make sure compromising one computer doesn't mean compromising everything.

That is the entire idea behind containment.

A bank doesn't protect its vault by putting one lock on the front door and hoping nobody gets inside. There are cameras, alarms, controlled entrances, restricted areas, vaults, time locks and people watching for unusual behavior. Getting through one layer doesn't automatically get you to the money.

We can build computer systems the same way.

If an AI gets into one network, there shouldn't be a straight line from that network to everything else. If it gets through one security boundary, it should encounter another. If it reaches something important, that system should have its own protections.

Eventually, an attacker may find a way through something.

The goal is to make sure that "something" isn't everything.

Maybe This Is Where AI Changes Computing

This is why I'm not convinced that the most interesting question is whether AI will eventually become powerful enough to threaten humanity.

It might.

Maybe some of the people making those predictions will eventually be proven right.

But if we're willing to imagine the most powerful AI we can imagine, then we should also be willing to imagine the infrastructure humans will build around it.

We may see a world where cloud computing becomes more carefully divided. Critical systems may move further away from general-purpose networks. Network segmentation may become even more important. Communication between systems may become much more controlled. File transfers may become deliberate events instead of invisible background processes. Monitoring may become increasingly intelligent, looking not just for known attacks but for behavior that doesn't fit what the system normally does.

And some systems may simply remain disconnected.

Not because we forgot how to connect them, but because we decided that connecting them wasn't worth the risk.

That would represent a pretty significant change from the direction we've been moving for the last few decades.

We've spent a long time asking how we can connect more things.

Maybe the next phase of computing will involve asking how much we really should connect.

If AI Changes the Threat, We Change the Architecture

I don't know whether AI will destroy humanity in ten years, fifty years or ever.

I don't know whether artificial general intelligence will arrive when some researchers predict it will. I don't know whether the systems we build will become uncontrollable, or whether we'll develop effective ways to keep them within boundaries.

What I do know is that technology changes when the threat changes.

We saw it with networking. We saw it with cybersecurity. We've seen it with physical security and virtually every other technology where convenience eventually collided with risk.

So if AI changes the threat, we'll have to change the architecture.

That doesn't mean becoming anti-technology. It doesn't mean disconnecting the world and going back to the 1990s. It means recognizing that connectivity is a tool, not a requirement.

Maybe the Internet we've spent decades building was designed around connection, and the Internet we need tomorrow will be designed around separation.

Maybe the future of AI safety won't be about building a smarter cage for the AI. Maybe it will be about building a world where there isn't one cage to escape from.

And if we're going to imagine a future where AI becomes extraordinarily capable, I think we owe ourselves the same amount of imagination when it comes to the systems humans build around it.

Author’s Note

Doug Fessler is a technology coordinator, IT consultant, and technology educator based in Pennsylvania. Through his work with organizations, schools, and community programs, he works directly with the technology infrastructure that increasingly connects our everyday lives... from networks and cloud services to emerging technologies, STEM education, and artificial intelligence.

Doug writes The Frequency to explore the intersection of technology, people, communities, and the world we are building around them. His goal isn't to tell readers what to think, but to ask better questions about where technology is taking us... and what responsibility we have for the future we are creating.

This article was written by Douglas E. Fessler. AI-assisted tools were used to structure and clarify complex concepts — a reflection, in itself, of the subject explored.

Page updated

Google Sites

Report abuse