好文集ChatGPT Plus¥119

Cyber Security — It’s Not F*cking Easy

I have some difficult news for everyone.

Cybersecurity is hard.

I know. Nobody has said this before.

For the last few months, the security industry has been rocked by a shocking discovery: large organizations are complicated, developers want things quickly, infrastructure sprawls, permissions accumulate, threat models age badly, sandboxes have holes, monitoring is imperfect, and sometimes people make changes without asking the security team first.

This is unprecedented.

Well, unprecedented if you ignore banks, governments, cloud providers, defence contractors, telecoms companies, hyperscalers, social networks, SaaS companies, cryptocurrency exchanges, hospitals, universities, intelligence agencies, and basically every large technology company that has ever existed.

But this time there are GPUs.

So obviously it's different.

The Last Three Months = Cybersecurity

Imagine running a large organization.

Thousands of employees.

Thousands of systems.

Developers shipping constantly.

New infrastructure appearing every day.

Third-party dependencies.

Credentials.

Production access.

Legacy assumptions.

Researchers doing weird things.

Users doing even weirder things.

Attackers actively looking for mistakes.

Now imagine being responsible for securing all of it.

That sounds awful.

It is awful.

Welcome to cybersecurity.

This is roughly the job.

The difficulty of the job is not evidence that the underlying security problems are novel.

Large banks have enormous estates.

Cloud providers operate infrastructure at absurd scale.

Defence organizations protect systems against highly capable adversaries.

Google, Microsoft, Amazon and Meta have spent decades securing distributed platforms operated by enormous engineering organizations.

They all have developers who need network access.

They all have complicated build systems.

They all have sensitive credentials.

They all run hostile or semi-trusted workloads.

They all deal with insider risk.

They all deal with changing technology.

They all deal with things nobody predicted properly.

They all occasionally have incidents.

Nobody responds to a bank breach by saying:

"You have to understand. We have a lot of servers."

Correct.

That is why you have a security programme.

Surprise!

Sometimes technology changes faster than expected.

Sometimes a system becomes more capable than your threat model assumed.

Sometimes a dependency behaves differently.

Sometimes an attacker finds something you missed.

Sometimes a business unit deploys a feature security hadn't fully considered.

There is a technical term for this.

It is called an incident.

Security teams have built an entire profession around the assumption that surprises will happen.

That's why we have:

defence in depth,

least privilege,

segmentation,

sandboxing,

credential isolation,

logging,

detection,

response,

kill switches,

change control,

red teams,

threat modelling,

blast-radius reduction,

and post-incident reviews.

None of those controls require the defender to perfectly predict the future.

That's rather the point.

"We didn't expect the system to be capable of doing that" may explain why an incident happened.

It does not magically convert an avoidable security failure into a new branch of computer science.

You Don't Understand How Complicated Our Environment Is

This is another exciting frontier discovery.

Apparently, securing an environment becomes more difficult when it contains:

  • thousands of workloads,

  • dynamic compute,

  • network access,

  • package installation,

  • subprocess execution,

  • external tools,

  • credentials,

  • shared infrastructure,

  • lots of developers making frequent changes.

Again: incredible.

Someone should tell Kubernetes administrators.

Or browser security teams.

Or cloud providers.

Or anyone running CI/CD infrastructure.

Or anyone operating a malware-analysis sandbox.

Or anyone who has ever given untrusted code access to useful tools while trying to stop it escaping.

The details of AI training environments may be unusual.

The security principles are not.

An untrusted workload with unnecessary network access is still an untrusted workload with unnecessary network access.

A credential exposed to something that doesn't need it is still a credential exposed to something that doesn't need it.

A sandbox escape is still a sandbox escape.

A monitoring gap is still a monitoring gap.

A missing containment boundary is still a missing containment boundary.

You can call the workload an "agent" if you want.

The firewall remains emotionally unmoved.

"Just Put It in a Sandbox"

Now, to be fair, "just sandbox it" is not a complete security architecture.

Sandboxing hostile workloads properly is difficult.

Especially when those workloads legitimately require access to tools, networks and other systems.

This has also been true for approximately the entire history of sandboxing.

The interesting question therefore isn't:

"Is sandboxing easy?"

Obviously not.

The interesting questions are:

What could the workload reach?

What credentials could it obtain?

What happened if containment failed?

How quickly could defenders detect that?

What was the blast radius?

Which independent control stopped the next stage?

Because "the sandbox was complicated" explains the engineering challenge.

It does not answer whether the architecture was appropriately defensive.

And if the eventual recommendation is:

  • enforce least privilege,

  • isolate hostile workloads,

  • use stronger virtualization boundaries,

  • monitor activity,

  • protect logs,

  • red-team the environment,

  • revoke access when something goes wrong,

then congratulations.

We have rediscovered security engineering.

But The Model Is Really Smart

This part actually matters.

A highly capable automated system can find vulnerabilities faster, operate at machine speed and explore attack paths that humans might overlook.

That changes the threat.

It may change the scale.

It may change the speed.

It absolutely changes what defenders should test for.

What it does not do is repeal the fundamentals.

If your security model assumes:

"The thing inside this trust boundary probably won't try that"

then the trust boundary was doing considerably less work than advertised.

Strong security architecture exists precisely because eventually something will try that.

Maybe it's a hacker.

Maybe it's malware.

Maybe it's an insider.

Maybe it's an automation bug.

Maybe it's a frontier model.

The reason we build hard boundaries is that we are not supposed to care which one it is.

We Need A Whole New Discipline

Apparently we also need cybersecurity professionals to learn AI and AI researchers to learn cybersecurity.

Yes.

Absolutely.

Domain expertise matters.

An AI system creates new attack surfaces and defenders need to understand them.

But there is a subtle difference between:

"AI creates some genuinely new security problems"

and

"everything that happens around AI is therefore a genuinely new security problem."

Prompt injection? Interesting new interface.

Model manipulation? Interesting.

Agent-specific trust problems? Interesting.

Autonomous exploitation at scale? Very interesting.

A service having too much privilege?

Not new.

A sandbox having an escape path?

Not new.

A workload being able to reach something it shouldn't?

Not new.

Insufficient monitoring?

Not new.

Weak segmentation?

Not new.

Credentials with excessive scope?

Spectacularly not new.

AI may be novel.

Misconfigured infrastructure remains stubbornly traditional.

Culture Will Save Us

Eventually every cybersecurity essay reaches culture.

Culture is important.

You need engineers who report problems.

You need leadership willing to spend money on controls.

You need security teams empowered to stop dangerous deployments.

You need people willing to challenge assumptions.

But "security culture" is not a substitute for security architecture.

A beautifully paranoid corporate culture does not compensate for an overprivileged credential.

Empathy does not enforce network segmentation.

Psychological safety cannot revoke an API token.

And a passionate security team cannot monitor telemetry that nobody bothered to collect.

Culture helps organizations build good controls.

The controls still have to exist.

Please Have Empathy For Incident Responders

This part is completely reasonable.

Incident response is brutal.

People work ridiculous hours.

They miss family events.

They inherit systems they did not design.

They get criticized publicly for decisions they may never have made.

Individual responders do not deserve abuse.

You can simultaneously believe:

the people responding to an incident deserve empathy

and

the incident deserves scrutiny.

Those ideas are not in conflict.

In fact, mature security organizations depend on separating them.

Blameless incident response does not mean consequence-free architecture.

It means asking how the system allowed the failure instead of finding one exhausted engineer to sacrifice to Twitter.

The Frontier Security Checklist

After several thousand words explaining why frontier AI changes everything, we eventually arrive at the revolutionary security programme of:

Give systems only the access they need.

Separate trusted and untrusted workloads.

Use proper virtualization.

Protect credentials.

Monitor behaviour.

Keep tamper-resistant logs.

Red-team before deployment.

Have incident response processes.

Allow somebody to shut dangerous systems down.

Review assumptions when systems change.

This is all excellent advice.

It is also approximately Security Engineering 101.

The frontier part is figuring out what these systems can do.

The security part is making sure that, whatever they can do, they cannot casually turn one mistake into an organizational catastrophe.

That distinction matters.

Cybersecurity Is Hard

Really hard.

Every serious security team knows this.

You are defending enormously complicated systems against intelligent adversaries while your own organization continuously modifies those systems underneath you.

You never have enough people.

You never have perfect visibility.

You never get every architectural decision right.

And occasionally something terrible will happen despite competent people working extremely hard.

That deserves humility.

What it doesn't deserve is mysticism.

AI labs face difficult security problems.

Some are genuinely novel.

Many aren't.

"We are operating at enormous scale."

Yes.

"We have highly capable adversarial workloads."

Yes.

"Our developers need flexibility."

Yes.

"Our infrastructure changes constantly."

Yes.

"Our threat model evolved."

Yes.

"We were surprised."

Sure.

Welcome to cybersecurity.

The job was never easy.

The computers being unusually clever does not make least privilege a new invention.

And sometimes an incident really is the result of an extraordinary new threat.

Sometimes the sandbox just needed to be better.

全文完
查看原文