San Francisco Philosophy Reading Group

We’re very excited that Michael Timothy Bennett—author of a paper featured in one of our previous discussions—will join us in person to discuss one of his other papers. This will be our first discussion attended by the author of one of our featured texts, offering us a great opportunity to question, challenge, and develop the argument directly with him.

The two texts we will be reading (by both Bennett and Nick Bostrom) raise a central question: can intelligence be separated from values? Or do an intelligent system’s body, environment, and ways of acting shape the goals it can pursue?
We’ll read two contrasting papers on the orthogonality thesis.

In “The Superintelligent Will,” Nick Bostrom argues that intelligence and final goals can vary independently: in principle, a highly intelligent agent could pursue almost any objective. In “Lies, Damned Lies, and the Orthogonality Thesis,” Michael Timothy Bennett challenges this picture, arguing that it treats intelligence as a property of disembodied software while overlooking the body and environment through which intelligence becomes action.

Can AI know what you know? Is AI somehow limited?

For this session, we’ll read Friedrich Hayek’s “The Use of Knowledge in Society” (1945) alongside “The Meaning of Competition” (1946). Both essays ask what happens when we mistake social order for a problem of calculation.

Hayek argues that the knowledge needed to coordinate society is dispersed, local, practical, and often impossible to centralize. Prices matter because they communicate fragments of knowledge no single person possesses. Competition matters because it discovers what no planner could know in advance.

Reading them together sets up the central question: if knowledge is bound to situation, context, and use, what can any central intelligence — a state, a firm, a bureaucracy, or an AI system — really know?