Using Outcomes to Measure Aggregate-Level Compliance – Justifications, Challenges, and Practices

Laws, rules, and regulations are intended to achieve goals, and the measurement of compliance is thought of as a way to verify if they function as intended. In this perspective, it may make sense to try and see not only whether and to what extent rules are complied with – but whether intended outcomes are achieved. In particular, the purpose of the chapter is measuring aggregate (rather than individual) compliance through outcomes measurement, and thus seeing the extent to which the entire regulatory system is functioning “as intended.” This contribution takes specifically the angle of outcomes measurement as proxy for the aggregate compliance level of regulated entities, providing an instrument to assess the performance of the regulatory system as a whole. Looking at several major regulatory fields, outcomes can be defined in terms of direct, physical-world results (occupational health and safety, food safety, environmental protection). The chapter also considers what the challenges are for the use of aggregate outcome data, how these can be managed, and what the practices of some leading regulatory agencies or services are. Often, these combine compliance measurement (direct) and outcomes measurement in order to compensate for each of the shortcomings and limitations of these approaches. Finally, the chapter briefly considers the obstacles and possible ways forward in applying “aggregate outcomes measurement” approaches – and the specific difficulties in using them in certain regulatory domains.