The reason that pilots almost always work is the so-called Hawthorn Effect. According to Wikipedia, “The Hawthorne effect (also referred to as the observer effect) is a type of reactivity in which individuals modify an aspect of their behavior in response to their awareness of being observed.” This is one of the reasons why I think that projects that are predominantly operating in a “from-pilot-to-scale” logic struggle to achieve that scale. Another reason why I am concerned with the prevalence of this logic in many projects is that they introduce often externally conceived solutions for what they analysed as ‘root causes’ into a system. Even if the solution scales, the social and institutional structure and dynamics of the system will most certainly not have changed, making the system not more but potentially less resilient. This is not what I would call systemic change.

Here is more from Wikipedia on the Hawthorn Effect:

The original research at the Hawthorne Works in Cicero, Illinois, on lighting changes and work structure changes such as working hours and break times was originally interpreted by Elton Mayo and others to mean that paying attention to overall worker needs would improve productivity. Later interpretations such as that done by Landsberger suggested that the novelty of being research subjects and the increased attention from such could lead to temporary increases in workers’ productivity. This interpretation was dubbed “the Hawthorne effect”.

What they did is to increase the level of lighting while carefully observing worker productivity. And indeed, with increased levels of lighting, workers became more productive. But then they reduced the level of lighting from the original level and, alas, workers became more productive.

Again from Wikipedia:

The Hawthorne Works had commissioned a study to see if their workers would become more productive in higher or lower levels of light. The workers’ productivity seemed to improve when changes were made, and slumped when the study ended. It was suggested that the productivity gain occurred as a result of the motivational effect on the workers of the interest being shown in them.

Now take a market systems development project – or any other project that aims at changing stakeholders’ behaviour by way of piloting this behaviour with a small number of stakeholders. The project gives a lot of attention to the chosen stakeholders. They discuss the current situation, make the stakeholder aware of the situation of the wider system, and they together come up with a (or more often the project simply introduces an expert-designed) solution to the problem at hand. If the entrepreneur agrees to pilot this solution, he or she are supported to do so with trainings and often even with funds to “buy-down risk” or “incentivise” the stakeholder to invest their own funds. Measurements are agreed to see if the pilot works and the project often provides staff to do the measurement. This picture might be a bit overdone to make a point, but I don’t think it is very far from reality.

If the solutions introduced via these pilots are not too far from what is feasible in the given context, they often work. But then, projects struggle to scale that behavioural change throughout the system. This problem of reaching scale has come up again and again as a major struggle of many projects during the time I worked for the BEAM Exchange. There are 545 results when I search for the term ‘scale’ on the BEAM Website. They cover webinars, guides that aim to help projects “getting to scale”, case studies, discussions, etc.

Could it just be that the reason why scale is difficult is that the pilot worked because of this “motivational effect” mentioned in the literature on the Hawthorn Effect and not that it was actually a good solution for the problems in the system? I don’t have an answer to that as I have not done any research to prove that this is indeed the case. But I think we should at least keep this in mind when we design projects.

But this is not the only reason why I am skeptical with regards to the dominance of this “from-pilot-to-scale” logic in market systems development (this is the field I know best, it might also be the case in other areas).

Let’s assume a project actually achieves scale in a solution that it introduced into a market system – and there are plenty of examples where that worked. Can we talk about systemic change? Certainly, there are system-wide changes of people using or profiting from this innovation. But has the overall capacity of the system to respond to change increased by actors taking up and copying an idea that was introduced from an outside project? Probably not.

This is why in our recent work with the BEAM Exchange, my colleague Shawn Cunningham and I rephrased the concept of systemic change. Systemic change is not about reaching scale with a more or less well thought through innovation that tackles an identified root cause in the system. Instead, we see systemic change as when …

… influential actors or networks of actors become aware of how change happens, and their role in realising the evolutionary potential of the economy. These influential actors need to develop the capability to engage in, collectively discover and continuously shape their institutional landscape.

Systemic change is about improving the capability of the actors in the system to live with change, not solving their problems. That is why our characterisation of systemic change is very close to resilience thinking.