Machines powered by AI increasingly mediate our social, cultural, economic, and political interactions. Understanding the behaviour of AI systems is essential to our ability to control their actions, reap their benefits, and minimize their harms. In these conversations, we discuss the need for a broad scientific research agenda to study machine behavior that incorporates the discipline of computer science while also drawing on expertise from across the social and behavioral sciences.We outline a set of questions fundamental to this emerging field, while we explore the technical, legal, and institutional constraints facing the study of machine behaviour. Concept, Design, and Research Iyad Rahwan, Manuel Cebrian, Nick Obradovich. With Collaboration and Support From William Powers.