The challenges of evaluating impact: going back to school


The ARCO Lab Summer School run in partnership with the Università degli studi di Firenze (Florence)

Whilst June across Scotland is typically associated with the heady final days of the school year, for GCU PhD student Noorseha Ayob, it was time instead to head back to school – the ARCO Lab Summer School on Methodologies for Impact Evaluation to be precise. The summer school has been designed in partnership with the University of Florence and was directed at graduate students, PhD researchers as well as academics and consultants interested in the methods utilised in impact measurement.

The programme’s overall aim, Seha tells me, was to get to grips with cutting edge methods and techniques that can be used to help build evidence-based policy. The three day course attracted a diverse mix of attendees with students and academics from across the globe travelling to listen and learn from scholars at the so-called “methodological frontier” (Arcolab, 2014).

The course was largely quantitative in nature with a strong focus on statistics and econometrics. Seha highlights this as both an opportunity and a challenge: “I used software that I have never used before. It was pretty tough for me as I’ve mainly used qualitative methods for my research to date but I believe that this could really help me with projects in the future.” She feels that having the capacity to combine these quantitative and qualitative research approaches may be critical for young researchers like her who have to go on to compete in the competitive post-doctoral job market. She also acknowledges the importance of these opportunities for networking and knowledge sharing. From various international universities to the UN World Food Programme and the Benin Ministry of Evaluation, attendees of the ARCO Lab Summer School came from an extremely wide range of academic and policy backgrounds. Seha is herself focusing her doctoral research on the growth and development of social innovation across Malaysia – a topic that proved to be of interest to the participants of the programme. She tells me that it is through this sharing of research stories, trials and tribulations that make these opportunities so valuable: “to hear how others are doing it, experience new approaches and challenge yourself!”

The programme was an interesting mixture of case studies and practical ‘hands-on’ projects to help participants to become familiar with, and comfortable using, the most up-to-date and innovative statistical software. Many of these were being used to evaluate the evidence base for public sector initiatives, health and social care interventions and third sector evaluations (see here for more details).

Of course setting the programme, the first year it has been run, in the beautiful city of Florence was also another highlight of this experience. Seha assures me that despite the busy schedule and challenging course content, her flying visit to Florence did manage to squeeze in a spot of sightseeing in the sun. Vital indeed when considering the ‘summer’ weather waiting back home in Scotland.

For more information on the work of ARCO please see their website: To find out more about Seha’s on-going research visit her student page at GCU here. Many thanks to Seha for taking the time to reflect on her experience and for sharing her photographs with us – as close to the Florentine sunshine as we are likely to get!

Leave a Reply

Your email address will not be published. Required fields are marked *