The 2nd International Conference on Realist Evaluation and Synthesis: Advancing Principles, Strengthening Practice

Pre and Post Conference Workshops

CARES 2016 Pre and Post Conference Workshops

Pre-Conference

The pre-conference workshops will take place on Sunday 2nd October. 

Pre-Conference Morning Session - What is Realism?: Exploring Key Philosophical Concepts to Advance Realist Methodology

Time - 10am-12.30pm

Workshop facilitator - Dr Justin Jagosh

Realism relates to a set of intellectual movements that has people making assertions about the nature of reality (ontology) and how we should construct science, research, and methodology (epistemology) to accurately produce evidence that is complexity-sensitive, yet pragmatic for knowledge purposes. The field of Realism is diverse, encompassing critical, scientific and other applied forms. However a few key tenets are commonly shared. These are: (1) positions on ‘mind-independent reality’; (2) references to ‘ontological depth’; (3) calls for generative (rather than successionist) views on causation; (4) and an identified need for retroductive/abductive theorizing. This free symposium will invite participants to take a moment away from the day-to-day tasks involved in protocol development, data collection and analysis - to explore the rich tradition of realist philosophy and what it can do for our analytic minds in vindicating ontology and complexity-sensitive methodology.  


Pre-Conference Afternoon Session - Workshop A: Realist Approaches for Complex Programs

Time - 1.30pm - 4.30pm

Workshop facilitators - Dr Gill Westhorp

More Details...

Background

There is a perception in some circles that realist research and evaluation approaches can only be applied in relatively discrete projects or programs. This perception seems to flow from a belief that realist analysis requires a 'fine grained' understanding of causation in context and that this is not possible for large scale and/or multi-component programs.

This workshop will challenge this perception. It will explore ways of applying realist methodologies in complex - and in particular, large scale - programs. It will present two inter-related strategies for approaching realist evaluation for complex programs: conceptualizing programs as operating at different levels of systems (and the possibility of using different constructs of mechanism at different levels of systems); and working at different levels of abstraction for different tasks in the evaluation design and implementation process.

Specific competencies to be addressed will include:
- developing program theory for large scale programs
- conceptualizing mechanisms in large scale programs
- combining realist synthesis and primary data collection and analysis in research into, or evaluation of, large scale programs.

The workshop will include presentation of examples from large scale and/or complex evaluations; activities for participants to try applying particular ideas in relation to their own programs; discussion of activities to identify challenges - either to the ideas themselves or difficulties in their implementation; and opportunities for peer-to-peer discussion of alternate strategies and approaches.

By the end of the workshop participants should have clarified, through experiences in the session and discussion, their own ideas about the feasibility of using realist approaches in large scale programs. The workshop may itself generate new strategies for using realist approaches in large scale and complex programs.


Workshop B - Developing and Sustaining Realism: How can we cultivate a disputatious community of realist researchers?

Time - 1.30pm - 4.30pm

Workshop facilitators - Dr Mark Pearson, Dr Janet Harris, Dr Justin Jagosh & Dr Geoff Wong

More Details...

Realist Evaluation and Realist Synthesis (Pawson & Tilley 1997; Pawson 2006; Pawson 2013) are gaining considerable traction across international development and health and social care as a way of understanding ‘what works for whom in what circumstances and why’. We want to consider how the mainstreaming of the approach can take place concurrently with building and maintaining a culture of productive critique and disputation which reflects and fosters its central tenets.

Philosophers have long identified the interplay of the social and technical in the pursuit of science, whether focusing on belief systems (Thomas Kuhn), organised scepticism and disputation (Robert Merton and Donald Campbell), or argumentation (Peter Lipton). However, these philosophers of science were concerned with relatively well-circumscribed areas of practice - typically a research programme or a discipline. One of the strengths of the emerging realist community is its transdisciplinary nature, where different perspectives help us to apply the principles in diverse settings. The challenge comes from applying realist logic across an ever-increasing array of disciplines and fields of inquiry.

We therefore need to understand better how to work as a well-functioning community to develop and sustain a cross-disciplinary perspective on conducting Realist Evaluation and
Realist Synthesis.

In this workshop, we will:

-Identify with participants the challenges for the realist research community
-Provide a forum for people to exchange their experiences and insights Propose how we can work towards a critical and supportive community
-Identify the skills needed to play an active part in a disputatious community
-Offer an opportunity for participants to develop their disputation skills and to ‘practice disputation’, using key debates within realist research
-Foster discussion about how a vibrant realist research community can be sustained
-Facilitate participants to identify individual and collective future action to further the
development of disputation skills and cultivate the ‘disputatious realist research community’


Workshop C - Exploring an Approach to Coding and Analysing Qualitative Data for Evidence of CMO Configurations

Time - 1.30pm - 4.30pm

Workshop facilitators - Melanie Punton

More Details...

‘How do I analyse my interviews to test CMOs?’ This question is frequently asked by novice realist evaluators, but there is little specific guidance to help answer it. This session will outline the approach developed by a team of evaluators from Itad, and give participants an opportunity to try it for themselves.
At the heart of realist evaluation is the ‘context-mechanism-outcome’ (CMO) configuration – a theory or hypothesis about how a particular causal mechanism works in a specific context to lead to an outcome. Evaluators develop and refine their realist programme theory through identifying and testing CMO configurations, in order to arrive at conclusions about how and why a particular programme works, for whom, and in what circumstances. Qualitative interviewing is one of the most common methods of data collection in realist evaluation, and therefore one of the main tools used to identify, test and refine CMOs. However, few guidelines exist on how to organise and analyse evidence from interviews in order to do this.

The team evaluating the Building Capacity to Use Research Evidence (BCURE) programme have developed an approach to coding and analysing data from a large number of interviews collected over six countries and three years, as described in a recent CDI Practice Paper. The approach is a pragmatic one, developed in the face of time and resource limitations, and in the attempt to promote consistency across multiple researchers in different countries. It is not a definitive blueprint, but rather a set of suggested tools and techniques that may provide useful examples and ideas for other realist evaluators to build on.
The session will describe the BCURE evaluation approach to:

  • Structuring interview notes: The approach we took to write up our qualitative data, and how we resolved the requirement for detailed notes within the confines of limited resources for transcription.
  • Coding qualitative data: The coding framework we developed, using Microsoft Excel to extract linked configurations of Cs, Ms and Os from our interview transcripts.
  • Analysing coded data and synthesising findings: The approach we used to analyse our qualitative data, in order to synthesise findings from across six countries and refine our CMOs and programme theory.

Throughout the session, there will be plenty of opportunities for participants to apply elements of the approach in practice (to their own data if they have it), and discuss ways it could be adapted to their own realist research. 


Workshop D - Introduction to Realist Evaluation and Synthesis

Time - 1.30pm - 4.30pm

Workshop facilitators - Dr Emma Williams and Dr Sonia Dalkin

More Details...

This workshop will introduce the key concepts that underpin realist approaches, with special emphasis on evaluation and literature syntheses. Realist evaluation has been developed to investigate and explain how and why programs and polices work differently for different people or in different contexts. Realist synthesis applies the same underlying principles to the review of existing literature and evidence. The workshop will have interactive exercises to show how realist evaluations and realist literature reviews differ from other approaches in their intent, design and analysis.

The workshop is designed for people who want to determine how – and if – realist approaches might work for them, and to build understanding of basic concepts and terminology before participating in the CARES conference. It will also provide opportunities to share ideas and questions, and to establish networks prior to the conference.


Post-Conference

Putting it all together: Digesting the Conference Experience and Advancing Your Project

Thursday October 6th, 2016
10:00am – 4:00pm

Facilitators: Dr. Justin Jagosh and Dr. Janet Harris

The idea for this post-conference workshop came from a suggestion by a delegate at CARES 2014, who commented that he had learned a lot from the conference and would have benefitted from a post-conference day where he could reflect on the content of the conference and how it could be used to develop the design of his project. In the morning session (10:00am – 1:00pm) we will begin with whole-group and small group discussions about methodological challenges and debates from the conference, what was learned and what remains as on-going confusion. We will try to achieve clarity on key conceptual challenges. The afternoon session will focus on people’s projects and will be tailored to group needs. This may involve one-on-one consultation and participant presentations on key issues and challenges in their projects. The workshop is designed primarily for people who are currently working on a realist project and are seeking guidance - however anyone who is interested may attend.