International organizations are increasingly focusing on organizational learning. The experience accumulated by development agencies throughout the world has become a source of organizational knowledge, which, according to Nonaka, is transferred through processes of socialization and externalization. Based upon three case studies and in-depth interviewing of World Bank managers and evaluators over two years, this article explores the contribution of evaluation to organizational learning. The study analyses the use patterns of evaluation as a source of knowledge within the World Bank. Findings show that participatory designs and processes favour socialization of tacit knowledge through interaction between organizational members. Theory-driven evaluations help externalize tacit into codified or explicit knowledge. Particular evaluation constructs – i.e. ‘chilling effect’ – provide vocabulary that clarifies discussion and debate for strategic planning. Overall, managers value those evaluation properties associated with (a) first-hand data collection within country case studies, and (b) theory-driven analyses, externalizing tacit insights coming from the field.