I am really happy to announce the publication of the Synthesis Paper of the so called ‘Systemic M&E’ initiative. The paper is the synthesis of conversations that started in MaFI in June 2010 and a series of online and in-person conversations that took place in the second half of 2012. It captures the voices of practitioners, academics, donors and entrepreneurs who are trying to find better ways to monitor and evaluate the influence of development projects on market systems and learn more, better and faster from their interventions.

The paper flags up three critical issues related to targeting, accountability and sustainability and proposes possible ways forward.

1. Excessive focus on our direct effects on the poor: Projects are demanded to deliver tangible impacts at the level of the poor (or in more generic terms: the target group). Direct delivery approaches are the easiest way to achieve that. Systemic approaches, however, aim to catalyze change, change structural aspects of the systems, induce spillover effects to indirectly drive and scale up change. The focus of M&E systems needs to shift towards understanding how results are achieved by systemic approaches and support decisions to foster these processes of positive change.
2. Excessive focus on extraction of information for accountability to the donors: In order to prove to their donors and ultimately to the parliaments and tax-payers that a project actually changed the situation of the poor, rigid and overly formalistic monitoring frameworks are employed that add a lot of additional burden to the work of the project staff. The focus of accountability needs to shift from reporting “benefits” produced by projects for the poor towards producing knowledge about how the project influences change processes in the system.
3. Sustainability understood as longevity of our legacy: The understanding of sustainability as the permanence of what development agents do or provide to the poor drives the project cycle and the design of M&E frameworks. The dynamic and complex nature of market systems is, however, incompatible with this approach. The understanding of sustainability needs to shift from a narrow view of longevity of superficial changes produced by the project towards building resilience and adaptability of the market system through deeper structural change.

The second part of the paper proposes seven principles that are seen as a basis for practitioners and policy-makers to design and implement appropriate and usable systemic M&E frameworks. These principles are not only valid for M&E systems, but also for project design in general.

1. Indirectness of impact: Systemic impact is reached indirectly through collaborators in the field and through changes in structural characteristics of the system, not by delivering change to the poor.
2. Depth of impact: Impacts in systems can happen at various levels. If most changes are related to stocks and flows of goods, we know that sustainability and scalability will suffer. We need to pay closer attention to deeper and more structural changes in the market system, such as creation of new networks, associations, or business models; increased access to information; shifts in power dynamics; collaboration around jointly agreed objectives, etc.
3. Network-driven change: In a systemic intervention, the facilitator engages a set of networks composed of public and private market actors. These networks are the real drivers of systemic change. Networks can be activated through collaborators of the project, such as lead firms or farmer groups.
4. Unpredictability: Complex systems resist prediction. This undermines the effectiveness of the mechanistic and rigid tools and approaches that are currently used in market development. Flexibility, rapid learning systems and effective collaboration between facilitators, NGOs and donors are key requisites to deal and navigate the uncertainty of complex systems.
5. Sensitivity to external signals: Projects need to tread lightly as their mere presence can have an impact on the system. The bigger the project, the bigger the risk that it has an unintended and distorting effect on the system. Facilitators, hence, need to tread carefully and be flexible and nimble.
6. Information deficit: No one can understand the whole system. Every actor of the system has his own perspective of the system, not just because of their own interest, but also because they interpret the slice of the system they know. Hence, nobody can stand outside the system, understand the whole, and plan interventions accordingly – this is also true for the project. This principle reinforces the need for participation, continuous learning, and flexibility in inclusive market systems policy and practice.
7. Sustainability as adaptability: Instead of focusing on the immediate impact on the target group, it is important to assess if the system is building up the necessary conditions to avoid or minimize future shocks and benefit from new trends.

The Systemic M&E process is not yet done. We are very eager to continue this discussion. We are aware that there are still many points in the paper that are ambiguous and need some refinement through discussion. We are aware that there will be no single best practice in how to implement monitoring and evaluation frameworks or even more to design projects in complex systems. Even more, continuous discussion, exchange and learning is essential.

The paper is now available from the SEEP Network’s website. Please share it freely and widely. We are looking forward to more engaging discussions around the topic.