Reconceptualizing 'effectiveness' in environmental projects: can we measure values-related achievements?

There have been recent calls for a shift to an evidence-based paradigm in environmental management, grounded in systematic monitoring and evaluation, but achieving this will be complex and difficult. Evaluating the educational components of environmental initiatives presents particular challenges, because these programs often have multiple concurrent goals and may value 'human outcomes', such as value change, which are intangible and difficult to quantify. This paper describes a fresh approach based on co-creating an entirely new values-based assessment framework with expert practitioners worldwide. We first discuss the development of a generic framework of 'Proto-Indicators' (reference criteria constituting prototypes for measurable indicators), and then demonstrate its application within a reforestation project in Mexico where indicators and assessment tools were localized to enhance context-relevance. Rigorously derived using unitary validity, with an emphasis on relevance, practicability and logical consistency from user perspectives, this framework represents a step-wise advance in the evaluation of non-formal EE/ESD programs. This article also highlights three important principles with broader implications for evaluation, valuation and assessment processes within environmental management: namely peer-elicitation, localizability, and an explicit focus on ethical values. We discuss these principles in relation to the development of sustainability indicators at local and global levels, especially in relation to post-2015 Sustainable Development Goals.

Field trials of a novel toolkit for evaluating 'intangible' values-related dimensions of projects.

A novel toolkit has been developed, using an original approach to develop its components, for the purpose of evaluating 'soft' outcomes and processes that have previously been generally considered 'intangible': those which are specifically values based. This represents a step-wise, significant, change in provision for the assessment of values-based achievements that are of absolutely key importance to most civil society organisations (CSOs) and values-based businesses, and fills a known gap in evaluation practice. In this paper, we demonstrate the significance and rigour of the toolkit by presenting an evaluation of it in three diverse scenarios where different CSOs use it to co-evaluate locally relevant outcomes and processes to obtain results which are both meaningful to them and potentially comparable across organisations. A key strength of the toolkit is its original use of a prior generated, peer-elicited 'menu' of values-based indicators which provides a framework for user CSOs to localise. Principles of participatory, process-based and utilisation-focused evaluation are embedded in this toolkit and shown to be critical to its success, achieving high face-validity and wide applicability. The emerging contribution of this next-generation evaluation tool to other fields, such as environmental values, development and environmental sustainable development, shared values, business, education and organisational change is outlined.