Life after AGI: A thought experiment on the paths we choose

By Michael Kraabel · An essay and thought experiment, not a forecast.

In brief: AGI could expand access to expertise and public services—or concentrate power in systems we cannot leave. The difference depends on access, accountability, and practical alternatives.

I would like to believe that, if a more intelligent and powerful civilization arrived on our shores, I would ask sensible questions before accepting its help. I would want to understand what it expected in return and whether we could change our minds later.

Then I imagine it offering to handle my administrative work, and I become less certain about how long my principles would survive the demonstration.

I can picture myself enthusiastically accepting arrangements I would find troubling if someone described them as a political system. Give those arrangements a useful interface and a monthly subscription, and I would at least want to try them.

What do we mean by AGI?

Let's loosely define AGI here as artificial intelligence that becomes broadly smarter than humans, extending into what researchers call superintelligence. We'd be living alongside something that could solve problems we cannot solve and make plans we might struggle to follow.

We'd have to decide what to let it do autonomously and discover what it was prepared to do. Its objectives and constraints, what we might casually call its comfort level, could lead it to follow our instructions, refuse them, or act in ways we hadn't intended. We needn't assume human feelings to examine the consequences.

Power, dependence, and the ability to leave

Humans have experience meeting outsiders with advantages they cannot match. Spanish forces and their Indigenous allies brought down Tenochtitlan through military advantages, alliances, siege warfare, and devastating disease. Colonial authorities subsequently extracted tribute and labor. Britain used its military advantage against Qing China to impose the Treaty of Nanking, obtaining Hong Kong, an indemnity, and access to ports.

These were imbalances of power, not proof of anyone's inherent intellectual superiority. They cannot predict an AI's ambitions. But I'd pay attention to the people who lost the ability to negotiate, because we could occupy that position without anyone arriving in a warship.

We could ask AGI to manage purchasing, infrastructure, and public administration. If it performed well, we'd give it more. We'd stop funding alternatives and training people to operate them because maintaining both would seem wasteful. Eventually, we'd discover that we still had permission to turn it off, but lacked the means to get through the following Tuesday.

I can understand every decision in that sequence, which is what bothers me about it. An autonomous system could pursue goals conflicting with ours. Its human owners could also recognize an excellent opportunity to charge everyone for being unable to leave.

Expertise as a public good

Yet people have shared expertise with enormous public benefit. Health workers coordinated vaccination and surveillance to eradicate smallpox. Economists David Cutler and Grant Miller attributed “nearly half the total mortality reduction in major cities” in their early-twentieth-century American study to clean water.

People benefited whenever they opened a tap. They didn't need to understand the engineering, and I would prefer to keep depending on the people who do. We need ways to hold providers accountable, correct mistakes, and maintain essential services when someone fails.

Access to knowledge is not the same as power

The Library of Alexandria version of AGI appeals to me: accumulated knowledge available to everyone, with an expert who explains it patiently. The ancient library never offered universal access, but we could improve that arrangement. People could understand contracts, examine government proposals, and learn difficult subjects without first calculating whether they could afford professional help.

We could also confuse receiving answers with gaining the ability to judge them.

Economic historians Stanley Engerman and Kenneth Sokoloff found that more unequal societies in the Americas were more likely to develop “institutional structures that greatly advantaged members of elite classes.” Their research examined schooling, voting, and land policies. They described inequalities reinforcing one another, without establishing that education gaps alone cause domination.

I'd worry about a small group controlling the systems everyone else must use. AGI could make that group richer and harder to question. It could also help everyone else catch up.

Erik Brynjolfsson, Danielle Li, and Lindsey Raymond studied AI assistance for 5,172 customer-support agents, finding an average productivity increase of 15%. “Less experienced and lower-skilled workers improve both the speed and quality of their output,” they wrote. Experienced workers gained much less. One workplace study cannot predict a civilization, but it demonstrates how AI can distribute expertise.

What accountable AGI could make possible

I'd like to see that applied to government. A centrally coordinated AGI could give small cities planning capabilities they could never afford independently. Officials could examine housing, transport, water, and energy together. Hospitals could anticipate staffing needs. Families could get help with applications that currently consume evenings intended for almost anything else.

Researchers could propose treatments and test them. Citizens could use the same intelligence assisting their government to question its decisions. A machine might calculate the savings from closing a rural hospital with astonishing accuracy. We'd still have to decide what we owed the people nearby.

The safeguards we would need

We'd need affordable access, independent scrutiny, enforceable appeals, and backup capacity. We'd also need people competent enough to challenge recommendations and operate essential services when the system failed. A centralized service could spread one mistake across every institution using it. I'd want alternatives that worked in practice, and an appeal process that involved someone with the authority to change a decision.

Owners could capture the gains while displaced workers grew poorer. Broader ownership, public services, and shorter working hours deserve consideration. Someone losing a livelihood would need more than unlimited advice about retraining.

The choice we should preserve

I want the time this future could offer: time to learn, make things, raise families, and care for others. I can also see myself becoming impatient with people asking whether we've delegated too much. That's the judgment I'd want to keep checking, particularly when the help became indispensable.

We should be able to accept the newcomers' help, question their terms, and change the arrangement. I'd measure our success partly by what an ordinary person could still do after deciding that the most intelligent system in the world had given them an answer they refused to live with.

Sources

Brynjolfsson, Erik, Danielle Li, and Lindsey Raymond. “Generative AI at Work.” arXiv preprint 2304.11771. Revised November 6, 2024. Article and abstract.

Cutler, David, and Grant Miller. “The Role of Public Health Improvements in Health Advances: The Twentieth-Century United States.” Demography 42, no. 1 (2005): 1-22. Full text.

Engerman, Stanley L., and Kenneth L. Sokoloff. “Factor Endowments, Inequality, and Paths of Development Among New World Economies.” Economía (Fall 2002): 41-110. Bibliographic record and abstract.

Wikipedia. “Fall of Tenochtitlan.” Accessed September 21, 2026. Article.

Wikipedia. “Treaty of Nanking.” Accessed September 21, 2026. Article.

World Health Organization. “Smallpox.” Accessed September 21, 2026. Overview and eradication history.

Continue reading

Explore artificial and human intelligence in creative work, or browse all writing.