Software Architectures for AI Systems: State of Practice and Challenges

Intelligent, AI software systems must process an ever-growing amount of data during operations. Engineering such systems is challenging as it requires integrating standard software components with data-intensive, distributed software platforms. Developing these systems requires close and ongoing collaborations between data scientists, who provide the domain knowledge, and software architects and engineers, who operationalize, deploy, and evolve the system. This chapter describes the state of practice and future research areas for AI and AI-based software system architectures. We identify and reflect on the fundamental engineering challenges for developing these systems, focusing on data collection, integration, inference, and continuous model updates and validation. From these challenges, we derive areas of future research for the software architecture community. These challenges include analyzing the observability of the AI system, identifying uncertainties and change management strategies, and developing new tools to support the architecture development process of the AI systems.