Michael T. Bennett - Can Intelligence Want Anything?

We’re very excited that Michael Timothy Bennett—author of a paper featured in one of our previous discussions—will join us in person to discuss one of his other papers. This will be our first discussion attended by the author of one of our featured texts, offering us a great opportunity to question, challenge, and develop the argument directly with him.

The two texts we will be reading (by both Bennett and Nick Bostrom) raise a central question: can intelligence be separated from values? Or do an intelligent system’s body, environment, and ways of acting shape the goals it can pursue?
We’ll read two contrasting papers on the orthogonality thesis.

In “The Superintelligent Will,” Nick Bostrom argues that intelligence and final goals can vary independently: in principle, a highly intelligent agent could pursue almost any objective. In “Lies, Damned Lies, and the Orthogonality Thesis,” Michael Timothy Bennett challenges this picture, arguing that it treats intelligence as a property of disembodied software while overlooking the body and environment through which intelligence becomes action.

The texts can be found here and here