Decolonial evaluation: centring communities and shifting power

What does it mean to decolonise how we evaluate our work? Catalyst believes that creating liberatory technology begins with questioning and reimagining our current approaches. That’s why, as part of the Tech Justice Road Trip project, we’ve embraced decolonial evaluation. It’s an approach that prioritises equity, inclusivity and community.

What decolonisation aims to do

Decolonisation is a process that aims to overturn and dismantle colonial power. It also looks to address colonialism’s legacies which include:

• racism
• economic inequality
• cultural imperialism
• environmental degradation

It’s an invitation to step outside, scrutinise and challenge our western, Eurocentric, misogynistic, caste-based society.

Decolonisation - a definition

Because decolonisation is a contested term, there’s no agreed definition of it. The Community-based Global Learning Collaborative describes it as: 

“...work that belongs to all of us, everywhere. It asks us to think about our relationship with Indigenous lands that colonizers have unjustly claimed, re-defined and repurposed all over the world. 

It asks us to embrace responsibility as opposed to accepting fault. Lastly, decolonization is a path forward to creating systems which are just and equitable, addressing inequality through education, dialogue, communication, and action.”

As well as nations and states, professions, disciplines and systems can be decolonised too. This involves re-examining their theory and practice through a decolonial lens. Disciplines and systems being sloly decolonised include:

• education
• social work
• therapy
• medicine

Traditional evaluation

In its report Decolonising evaluation: whose value counts? Christian Aid describes traditional evaluation models as problematic and extractive. 

External evaluators are often parachuted in to assess projects. They use Eurocentric methods to collect data from individuals and communities. They analyse data based on limited knowledge of a project’s context. And they draw conclusions that meet criteria that have already been established. In these models, evaluations aren’t designed by communities. 

What’s prized is “...Western or scientific understandings of how we generate knowledge and what type and whose knowledge counts in our assessments of value.” - Christian Aid, talking about traditional evaluation

Decolonial evaluation

Decolonial evaluation is more equitable, inclusive and respectful of everyone involved.

It recognises and values other types of knowledge and approaches (not just western-focused ones).

It questions power dynamics.

It also aligns with Catalyst’s values. That’s why we’ve taken this approach in our tech justice work. 

The project’s direction is being shaped by a ‘Circle’. The circle is made up of partner organisations and individuals contributing to the project. Circle member Zainab Ekrayem from Infocus Consulting leads the project's evaluation. She explained that using a decolonised approach means, “...we can:

• address historical inequities  
• enhance the relevance and accuracy of our evaluation work
• shift power and ownership to local communities
• identify power dynamics, make them visible and challenge them
• nurture innovation and creativity – we are free to use different ways of gathering and presenting data”

8 steps to decolonising evaluation

Zainab outlined to us what decolonial evaluation looks like in practice.

1. Reflect on your own positionality and biases

For example, if you've studied at a western university you may have a tendency to prioritise, and value, qualitative data over quantitative data.

2. Prioritise local ownership and participation

This helps you avoid tokenistic participation. Tokenistic participation creates an illusion that the work is inclusive and power is being shared, when it isn’t happening in practice.

3. Acknowledge and incorporatemultiple ways of knowing

These include indigenous knowledge, storytelling and other non-Western methodologies.

4. Create an atmosphere of respect, trust and openness

Take time to foster relationships that make evaluation participants feel more comfortable and valued. Accept that this will likely need more time and cost more.

5. Analyse power dynamics and address inequities

Identify colonial power structures in your existing evaluation frameworks. Then work to dismantle them. This might include not employing external evaluators who don’t understand your project’s context, or the issues communities are dealing with.

6. Translate and communicate findings equitably

Make any outputs accessible, clear, concise and easy to understand. This could include using visual summaries, local languages, or community meetings instead of dense technical reports. It may mean that you have to cater for multiple audiences and disseminate your findings in different ways. But you can plan for this, and it will add value to your work.

7. Build your partners' evaluation capacity

Increase your partners’ skills and knowledge. So they become more able to carry out and participate in evaluations. For example, offering workshops on survey design and data analysis. Or access to affordable evaluation software. This builds people's capacity to collect and analyse data in the formats needed. Partners get closer to data and insights. You get to understand data and insights in context.

8. Commit to continuous learning

Get regular feedback from communities and use it to improve your practice. Host regular feedback sessions to assess whether evaluations align with community needs. Then adjust your methodologies accordingly. This is not just about evaluation processes but also how interventions, products and services operate.

Success is emergent and community-centred 

Unlike in traditional evaluation, success in decolonial evaluation is defined more broadly. It’s not just about the outcomes and impact your project achieves. It includes personal and collective dimensions. For example, your own experiences on the project. And questions like 'how collaborative and equitable was the process?' 'Did working together feel meaningful?' 'Were relationships prioritised?' And 'how much was power shared?'

Circle members have suggested short, mid and long term success measures for the project. These include:

• communities being at the heart of the work and at the centre of emerging tech
• having more practical examples of tech justice in action
• being able to learn about, and try, new ways of working
• making new connections and partnerships
• capturing the journey and drawing out learnings and opportunities
• acknowledging and speaking openly about the systemic barriers, including those reproduced in the circle
• community members being heard and respected, and wanting to use the tools that are developed
• partner organisations learning new things they can bring into future work.

One member also asked if the project is only successful if it’s high impact? 

Next steps for the Tech Justice Road Trip evaluation

Zainab has developed an evaluation framework using the perspectives and reflections that circle members shared. The framework is structured across 4 levels:

• individual
• community
• partner organisation
• project

Circle members have been invited to help prioritise the outcomes in the framework.

Other resources on decolonial evaluation

Zainab Ekrayem, Nish Doshi and Tom Keyte have written an introduction to decolonial evaluation, Shifting from extraction to nourishment. It covers, creating your evaluation framework, joyful data collection, analysing data and sharing your learnings.

The webinar Decolonising Evaluation: an in-depth look, is an hour-long recording of a panel discussion. The panel includes Nish Doshi, Community Organiser and decolonial design practitioner, and Angela Schlenkhoff-Hus, Senior Learning and Evaluation Consultant at inFocus Consulting. The discussion focuses on why decolonising your evaluation process is important. It includes examples from practitioners in the UK who are doing it. 

Thanks to Hannah Turner-Uaandja, Catalyst’s Participatory Engagement Lead and Zainab for leading the decolonial evaluation workshop!

‍

--

Photo by Mel Lituañas on Unsplash