Click here to return to the BERA Homepage

Consuming research: issues and strategies for communication and use


Return to front page of this collection of resources

Contents

  1. The discourse of research ‘dissemination’ and ‘impact’
  2. Improving research dissemination and impact’
  3. Models for assessing the impact/influence of research
  4. Further reading and bibliography

Introduction

This resource provides an overview of the key issues – accompanied by case study exemplification and points for discussion – involved in using research in the public sector, and of the major initiatives designed to improve the ‘dissemination’ and ‘impact’ of research.  In doing so, it highlights the factors that tend to govern the influence that research has in decision-making, whether inside the commissioning organisation or on policies and/or practices in the system as a whole.  Like the other resources in this series, the focus is mainly but not exclusively on commissioned research in a policy context.

The common terms used in talking about the communication and use of educational research – such as ‘dissemination’, ‘engagement’, ‘utilisation’, ‘knowledge transfer’, ‘consumption’, ‘impact’ – form a large part of the substantive discussion which follows.  This is partly because they are part of an emerging discourse which is still quite fluid and partly because they are all metaphorical terms, whether or not people are always conscious of their connotations.  Accordingly, no attempt has been made to offer concise definitions which would pre-empt that discussion.

Useful on-line documents are signposted at relevant points in this resource, and a bibliography/list of further reading is given at the end. (Sometimes relevant/interesting material exists only for commissioned social science in general, rather than specifically for educational research; and/or for government-commissioned research rather than for research commissioned by other types of body.)

The discourse of research ‘dissemination’ and ‘impact’

In this resource, we explore the discourse of research ‘dissemination’ and ‘impact’ as well as at the strategies that have been developed to improve the communication and application of research.  Questions of how and why research should be communicated to non-specialist audiences have come to occupy more and more space – in the academic, policy and practice worlds – over the last decade or so.  On the one hand, the discourse around research reporting has expanded to encompass notions of ‘dissemination’, ‘engagement’, ‘utilisation’, ‘transfer’, ‘consumption’, ‘impact’ and so forth – different metaphors arising from quite different realms of activity, however faint the allusions may be in this context.  On the other hand, the extensive and varied efforts at national and local levels to improve the communication and take-up of research findings have helped to reveal the many and important ways in which the social practices of research and policy-making differ from each other.
 
The main starting point for ideas about dissemination and impact is the ‘evidence agenda’ concerned to find out ‘what works’, which was discussed in Resource 1.  Educational research is often thought of as a social science (though see the Introduction to Resource 1 for a contrasting view) and as such is argued, or assumed, to have potential relevance for both strategic and tactical decisions made in this sector of social policy-making and practice – healthcare, social care, criminology being other sectors where the evidence agenda has been strongly urged on, and by, researchers, policy-makers and practitioners.  If the substantial research effort in these domains cannot add to policy-makers’ and practitioners’ knowledge, aid them in decision-making and help them solve social and educational problems, then what is it for? This apparently common-sense position can be lent additional force by arguing – as Davies (1999) does – that in the absence of research such decisions can only be made on the basis of ‘political ideology, conventional wisdom, folklore [or] wishful thinking’.

The proposition cannot be as straightforward as it looks, however, because the national investment in the production of research has apparently not been reflected in perceptions of its use by and usefulness to decision-makers.  This has been the refrain of all the recent major reports investigating the influence of education/social science research on policy-making (Excellence in Research on Schools; Great Expectations: the Social Sciences in Britain; Evidence in Education: Linking Research and Policy; and Punching Our Weight: The Humanities and Social Sciences in Public Policy Making – all referenced in Resource 1).  Between them, these reports span a decade of investment in research and its dissemination; and they continue to point to what would appear to be intractable challenges:

  • the research produced is not fit for policy purposes (more because it is inaccessible or inconclusive than because it is judged to be of poor quality);
  • communications between policy-makers and researchers are weak;
  • policy-makers do not make good use of research in education/the social sciences.

The following section will outline some of the activities and initiatives designed to address these problems.  First, however, it will be useful to consider these challenges a little further in the light of Carol Weiss’ observation:

‘It takes an extraordinary concatenation of circumstances for research to influence policy directly….   [rather] research helps people reconsider issues, it helps them think differently, it helps them re-conceptualise what the problem is and how prevalent it is, it helps them discard some old assumptions, it punctures old myths.’  (Weiss 1991)

So is it the case that expectations about the capacity of policy to be informed by, if not based on, research are simply unrealistic?  Remember that Weiss is writing before the current UK ambitions had been formulated and followed up with the establishment of, for example, dedicated research centres plus the apparatus for ‘systematic reviews’, the National Educational Research Forum (which has been re-formed as the Strategic Forum for Research in Education), the National Teacher Research Panel (NTRP) and most importantly the large-scale activities of the Teaching and Learning Research Programme (TLRP) and the Applied Educational Research Scheme (AERS) in Scotland – both of the latter with a remit to involve research ‘users’ at every stage of the research process.

As was discussed in Resource 1, there are real and enduring differences between the aims and practices of policy-making and those of research and inquiry, which may be summarized thus:

  • policy is formulated and changed by factors that, singly or in combination, are often more cogent than evidence: values/ideology, manifesto commitments, public opinion and limits imposed by what is economically feasible, to name just a few;
  • research often deals with specific elements of policy, so it is not a straightforward matter to ‘read off’ policy from research, to integrate delimited research findings into an entire policy direction;
  • at the same time, the knowledge gained through research into social processes is inevitably and irreducibly complex and provisional – or, if you prefer, research is messy and ‘does not suggest a single course of action.’ (OECD/CERI);
  • it is normal and desirable for academic research to be published in peer-reviewed journals; both the time elapsed between research and publication, and the extremely large number of journals in which research relevant to education is to be found, are barriers to its timely use in policy environments;
  • in any case, assumptions about ‘knowledge transfer’ that seem to underpin many efforts to increase the take-up of research ignore the fact that ‘active’ knowledge is typically implicit and located in people rather than in writings.

In the face of these challenges, there seem to be two broad kinds of response.  One is to maintain a broadly rationalist position – essentially, people will be able and willing to make evidence-based decisions provided the product can be got right and the communication strategy is effective.  Government departments commissioning research and research organisations working close to government are likely to take this view.  A sophisticated example of the position is to be found in the CfBT’s Evidence Matters report:

‘Greater imagination is needed about the form in which research is made manifest. Too many projects simply result in a report and perhaps an event to “disseminate” it. It is no surprise that such minimal impact strategies simply address tiny specialist audiences and leave untouched the mass of end-users who might need to hear the message. Dissemination is simply the beginning of a strategy for active exploitation of knowledge…  Teamworking needs to extend to include the panoply of communication specialists who distil messages and get them out to the audiences that need to hear them. Publishing, journalism, video making, visual design, web design, events management are all key partners in making research count…’ (Morris 2009 for the CfBT)

More straightforward is the DCSF:

‘Our annual Research Conference is the main forum through which we aim to engage our key partners with the latest findings from our research, build collaborative working relationships on research issues, and raise the overall profile of our analytical work.’ (DCSF Analysis and Evidence Strategy, Chapter 5, ‘Access to our evidence’)

Simpler still is Charles Desforges:

‘… a straightforward recipe for achieving educational policy objectives whilst at the same time saving vast amounts of public money…’ (Desforges, Foreword to Morris 2009.)

The other kind of response is to replay the Weiss gambit, emphasising ‘the ways in which educational research is typically produced and utilized… as part of a complex conversation about a diversity of purposes, effects and judgements rather than a more technically oriented implementation of “what works”’ (Bates 2002);  and pointing to what educational research itself tells us about ‘knowledge transfer’ (see, for example, Fielding et al. 2005).  This research suggests that the processes and activities that support research-informed decision-making are situated, context-specific and embedded in institutional cultures over time;  a central part is played by non-systematic factors such as professional identity and informal/social activity, and by personal relationships of trust and collaboration that cannot be mandated.  For interesting accounts of specific situations and initiatives, see Moss and Huxford 2007, Somekh 2007.
 
As Desforges himself goes on to say immediately after the sentence quoted above: ‘What is called for is a deeper consideration about the connectivity between research, policy and practice and between researchers and policymakers/practitioners.  Amongst other things, practical working relationships are needed…’  Edwards (see TLRP seminar series reference below) has proposed that socio-cultural activity theory may be helpful in developing our understanding of such non-rational processes.  Additionally, it is obvious to many that emotions play a role too – not least fear and anger – though it is harder to find written accounts of this affective dimension.

Hammersley (2002) theorizes the issues by proposing a model of research in relation to policy-making in which research offers neither engineering ‘solutions’ nor ‘enlightened’ intellectual critique but instead different kinds of ‘cognitive resources’.  He argues that this model both captures the way in which social research actually influences policy-making (and practice) and also removes the need for researchers to conceive of themselves as ‘technicians, legislators or revolutionaries’ – though it does ultimately seem to be another rationalist model.

With the intention both of exploring such issues further and of promoting ‘user’ involvement in research, the TLRP hosted a seminar series ‘Making a Difference: working with users to develop educational research’, which involved teachers, civil servants and think-tank consultants as well as academics.  The seminars resulted in some useful papers – for a selection, go to http://www.tlrp.org/themes/seminar/edwards/current.html and click on:

The seminar series raised a number of basic questions, such as ‘impact’ on whom and on what?  is ‘benefit’ a more helpful notion? do we give sufficient room for mutual meaning-making between researchers and policy-makers, given how complex decision-making in social environments is? 

One of the premises of the seminars was that research ‘users’ need to be involved in a research project from its very beginning, in order to secure their support and so that their needs and interests can help shape the project.  (Some of the practical ways of doing this, such as advisory boards, are noted in Section 11 of Resource 3.) 

However, the term ‘user’, like many of the other terms in common use, assumes some of the very things we might want to question.  In using metaphors, we need to be conscious of what gets foregrounded and what gets left out (see Fielding 2003), and what effect the unexamined use of the same metaphors has on the way we think.  Articulating them rather simply, we can list the terms and their embedded metaphors and assumptions as follows:

Term Metaphor/presupposition(s)
dissemination research as package of epistemic objects, scattered in hope that some of them will land on fertile ground and grow into…?
communication research as social product whose meanings are essentially unproblematic but need to be made attractive to audience
application ‘the appliance of science’, research as technological solution to social engineering problem
user, utilisation research as epistemic product made by some people for others to use
consumer, consumption research as epistemic product made by some people that others can buy for their individual use – but is the customer always right?
impact research as epistemic object colliding with, and resulting in change of direction of, target object
(knowledge) transfer research as epistemic entity that can travel from its source to some destination elsewhere
(knowledge) exchange research as epistemic entity that can travel in both directions between different locations OR research as one kind of epistemic currency that can be exchanged for another kind
brokerage the notion that intermediaries – people or processes – are needed when knowledge is transferred or exchanged
engagement variety of connotations, including binding commitment (to a person or project), interlocking (as in gearing), instigation of military action

In the main, these terms presume that the responsibility for making research ‘travel’ lies with the researcher/producer.  Philippa Cordingley, chief executive of the Centre for the Use of Research and Evidence in Education (CUREE), proposes a different version of this locative model, one of professional learning in which research knowledge is ‘pulled’ by the practitioner rather than ‘pushed’ by the researcher, and the effort is led by the practitioner’s questions not just the researcher’s data (see, for example, Cordingley 2000, Cordingley et al. 2007).  CUREE have developed a range of strategies and professional development tools designed to help teachers appraise and interpret the ideas and evidence in a wide range of research studies, and relate them to their own needs and contexts. 

Whether policy-makers need, and would use, similar pedagogic strategies is less certain.  Nonetheless, the idea that ‘users’ or ‘consumers’ of research have an active role to play in the co-construction of knowledge is an important line of argument to develop, especially when it comes to the task of drawing out implications from research findings – see Levin 2004, Davies et al. 2008, as well as the following section.

In all this, however, we should not neglect the role of serendipity and opportunism in whether and how research ideas get taken up – being in the right place at the right time, knowing the right people with whom to have conversations;  which suggests that there is much still to be explored from an ethnographic perspective about how and how far research ‘travels’ in different policy-making environments. 

Some researchers have accordingly turned their attention to analysing and theorising the organisational processes of research ‘dissemination’ and ‘impact’.  The cross-sectoral work of the Research Unit for Research Utilisation has been ground-breaking in this area – see, for example, Nutley et al. 2003, and the website: http://www.ruru.ac.uk/publications.html which has links to the very many articles, reports and conference papers published by the team.  A crucial point they, and others, make is that the practical utility of some research in education  – the study of the history of education being a good example – may not be self-evident, direct or immediate, and that therefore its influence on policy (or practice) may be years in the making and/or not clearly traceable.  This does not mean, of course, that such studies are irrelevant or unworthy of public funding – though they will probably not form part of commissioned research projects.

Lastly, it would be a mistake to suppose that every research project can or ought to aspire to influence decision-making.  As is noted in the summary of Seminar 5 in the TLRP series ‘Making a Difference: working with users to develop educational research’:

‘…policy-makers are primarily interested in the “balance of evidence” [which points to]… the importance of research synthesis.  As Donald McIntyre reflected at the end of the day, “There is a danger of trying to push one research project too hard. Individual researchers trying to get their specific project’s ideas or findings across can be part of the problem”.’

TASK A

Think about a research report or article with which you are familiar.  What, if anything, might prevent it from being influential in a relevant policy-making environment?  List its strengths and weaknesses from a ‘user’ perspective.

 

TASK B
Download and read:

Noting Philip Davies’ observation in the latter that ‘policy-makers reply primarily on three sources of information: commissions, gurus and think-tanks … social science researchers working in academia are rarely consulted’, do you think ‘user engagement’ is primarily an epistemological or a socio-cultural challenge?

An attempt to measure research impact

Before we leave this discussion, it is instructive to look at the efforts of the Research Councils UK to get a grip on research ‘impact’.  In 2006, the so-called Warry Report, Increasing the Economic Impact of the Research Councils, recommended that the Research Councils needed to demonstrate more clearly the impact they achieve from their investments.  In their Action Plan of January 2007, Increasing the Economic Impact of the Research Councils , Research Councils UK (RCUK) described how the Research Councils would take forward the specific recommendations made;  and commissioned two studies (http://www.rcuk.ac.uk/news/ecimpstudy.htm) to assist in the implementation of the plan.  The first of these was a survey of users (to ‘establish the extent to which the Research Councils are meeting the needs of their user communities’) and the second a study to establish a quantified baseline assessment of the economic impact of the work of the Research Councils.  Speaking about ‘Excellence with Impact’, the 2007 progress report on implementing the Warry recommendations – downloadable at: http://www.rcuk.ac.uk/cmsweb/downloads/rcuk/economicimpact/excellenceimpact.pdf, Professor Philip Esler, Chair of RCUK, said:

‘The study clearly demonstrates that our research is having powerful and far-reaching impacts and benefits. The knowledge and expertise gained through our investment in people and innovation keep the UK at the cutting-edge of technology, build a strong economy and improve the health and quality of life of its people. The challenge now is to maximize these effects by embedding economic impact within the strategies, delivery, and organisation of the Research Councils.’ http://www.rcuk.ac.uk/news/warry.htm

The results of the first of the two baseline studies are reported at: http://www.rcuk.ac.uk/cmsweb/downloads/rcuk/economicimpact/ussurvey.pdf

But the second study came to grief, as reported in 2008 by Times Higher Education:

‘The research councils have abandoned plans to develop a formula to calculate the economic impact of the research projects that they fund. The seven councils have conceded that it is not possible to accurately quantify the value to the economy of a diverse range of research projects… In October [2007], Research Councils UK mooted the intention to change the peer-review process to ensure that the future impact of research projects was considered when deciding which proposals to fund. Philip Esler, RCUK’s “knowledge transfer champion” and chief executive of the Arts and Humanities Research Council, also said the councils were developing an algorithm to measure impact. “We want to get to the stage where we can say that with an investment of x we get a return of y”, he said. But last week he told Times Higher Education that there were so many variables that the councils had “given up” trying to find a formula.’ (Corbyn 2008)

Meanwhile, the University and College Union (UCU) had responded to the RCUK proposals with some critical observations, amongst which is this summary of the limitations of ‘economic impact’ as a criterion:

‘The UCU believes that research proposals must continue to be assessed primarily on their scientific or artistic merit. A simplistic notion of “usable research” can often overlook the fact that what is useful is often neither immediately useful nor recognisable as eventually useful, i.e. theoretical advances often come decades before practical applications are realised. It is important that the introduction of the Warry recommendations do not inadvertently stifle basic or theoretical research. At the same time, we welcome attempts to identify and widen the potential beneficiaries of research, particularly in the more practically-oriented disciplines. For example, we would like policy-relevant research to be valued, the civic purpose of the university to be affirmed and research-informed teaching to be recognised. However, we are concerned that the Warry report has adopted a narrow, pro-business notion of economic impact. Economic stakeholders should include community groups, voluntary organisations, trade unions and NGOs – and not simply employers. Any research council training courses and policy indicators, therefore, will need to take on board a pluralistic notion of economic activity. Finally, higher education research has an important role to play in strengthening democratic citizenship, enhancing social inclusion and promoting environmental responsibility… we would like the research councils to do more to develop their procedures in ways which allow a practical assessment of the social, cultural and democratic benefits of research.’ http://www.ucu.org.uk/media/pdf/3/g/ucu_peerreviewresp_jan07_1.pdf

The debate and its corollary activities will be continuing as the higher education funding councils develop the proposed new system for research assessment, the Research Excellence Framework:  , an extract from which reads as follows:

‘Demonstrating impact

We propose that institutions will need to submit evidence of impact for assessment in the REF. This will take the form of: 

  • an impact statement describing the breadth of interactions with research users, an overview of positive impacts achieved by the unit as a whole
  • a number of case studies briefly illustrating specific examples of impact (we propose one case study for every 5 to 10 members of staff).

Indicators of impact selected from a common ‘menu’ will support these statements (such as research income generated from research users, and indicators of collaboration). We intend that research users will be involved heavily in assessing impact across the full range of panels.’

 

Improving research dissemination and impact

This section gives a flavour of the wide range of activities and initiatives that have been developed to improve research ‘dissemination’ and ‘impact’ over the last ten years.  In relation to education policy-making, most of these have been on the supply side, that is, concerned with the production and communication of research knowledge.  Most of them have also been concerned with increasing the ‘impact’ of research that has not been directly commissioned by government or other agencies, on the understandable assumption that policy-makers take more notice of research they have contracted – though, as research managers in such organisations know, this is not always a correct perception, and the activities outlined below will be useful in a contract research context as well.

The relationship between research and teaching has a rather different history from that between research and policy, since it is rooted in the Stenhouse-inspired research-informed professional practice movement which has grown in strength from the 1970s onwards.  This is not the place to discuss practitioner research in all its manifestations and from all its numerous perspectives – we can just note a few facts in support of the idea that the relationship between research and policy in education is different from that between research and practice, in terms of both expectations and of actual social practices. Many teachers are engaged in doing research themselves, as part of their initial teacher education or higher degree study and/or because the school supports action research and inquiry-led learning; the professional standards for teachers require teachers to make use of data and evidence; TLRP and AERS have involved teachers as members of research teams;   the General Teaching Councils all support teacher research and enquiry with bursaries and/or accreditation frameworks; and most importantly, perhaps, the dominant rationale for teachers’ engagement with research is as continuing professional learning – what might be called a pedagogical rather than an evidential concept of research. 

Teachers’ appetite for research which looks as if it will help their work with students means that occasionally the route from research to policy is via practice – a well-known example is the work on formative assessment and assessment for learning, led by Paul Black and Dylan Wiliam (since the original pamphlet, Inside the Black Box, Black and Wiliam 1998, there have been many more publications elaborating the AfL principles and practice), which informed and changed classroom practice several years before the government adopted, and adapted, the principles for policy.  Another example is the extensive, in-depth programme of work on student voice, Consulting Pupils about Teaching and Learning, by Jean Rudduck, Michael Fielding and others.

So one possible strategy for increasing the policy impact of research is to create a groundswell of support and active engagement by the teaching profession, which policy-makers will sooner or later take seriously. 

Usually, however, improving ‘dissemination’ and ‘impact’ is taken to mean one or more of the following:

  • improving the quality of research for policy;
  • making research more relevant;
  • making research more accessible, including improving the way research is presented to policy-makers;
  • increasing the policy ‘read-off’ from research;
  • strengthening the roles and skills of research mediators and brokers;
  • changing the mode from ‘knowledge transmission’ to ‘co-construction of knowledge’;
  • changing the model from ‘impact’ to ‘diffusion’.

 We now look briefly at some issues and examples under each of these headings.

Improving the quality of research for policy

An initial point to be made about quality is simply that it would be unhelpful to consider all the other aspects that may lead to ‘impact’, such as relevance, clarity, accessibility and so forth, in isolation from the quality of the research, that is, the rigour of its premises, design, methods, analysis and inferences.  A research report may be clear, topical, appealing and plausible without deserving our attention on account of those characteristics alone – nonetheless these are important features in research that aspires to be influential.  An attempt to define quality in a broad sense – to hold on to epistemic authority whilst encompassing value-for-use – is to be found in Furlong and Oancea 2005.

Another point to be made about the quality of research is not the old canard about lack of quantitative and empirical studies, tendency towards partisanship, deficiency of cumulative and replicated research, in education (see, for example, http://www.bera.ac.uk/files/2008/12/tooley_1998.pdf) – about which there was doubt even at the time the criticisms were made over ten years ago – but instead a reminder of the observation made by McIntyre, quoted above, about policy-makers’ interest in the ‘balance of evidence’ and the consequent need for syntheses of, and across, different research studies. 

A recent innovation in research synthesis is the so-called systematic review methodology (see, for example, Oakley 2003 and http://eppi.ioe.ac.uk/cms/), which starts from a specific policy or practice question to which there may or may not be a body of sufficiently well-attested answers, depending not least on whether research efforts have been focused on that particular question in the past.  Other forms of meta-analysis and synthesis can be just as valuable in decision-making.  The journal Educational Research Review – which publishes ‘meta-analytic reviews, narrative reviews and best-evidence syntheses’ – lists the following types of review as acceptable:

  • research reviews: reviews aimed at comparing research on similar or related topics;
  • theoretical reviews: reviews able to critically describe the evolution of theories and the way they are understood in different contexts;
  • methodological reviews: reviews devoted to methods and methodologies used in education;
  • thematic reviews: reviews based on description of particular areas of the literature, or particular educational approaches or learning models;
  • theoretical contributions: state-of-the-art papers relating issues, comparisons, and analyses to the application of methods and models to the educational process;
  • research critiques: reviews on selected educational topics reflecting implications for the field of education;
  • forum papers: shorter articles presenting new ideas, or responses to published material stimulating debate, but well founded in the existing literature;
  • instructional [teaching] techniques: reports on instructional techniques when the use of adequate controls demonstrates the validity of the findings.

There is an online guide at http://www.elsevier.com/framework_products/promis_misc/edurevguidetowriting.pdf which gives brief definitions of different kinds of research review.

Reviews allow us to know something about the range, nature and quality of research as a generic criterion of a whole field, not only of individual studies, and this is very helpful arriving at judgements about what the balance of evidence shows on a given policy or practice issue.

Making research more relevant

The charge that much educational research is not ‘relevant’ to the needs and concerns of policy-makers and teachers is hard either to prove or to disprove – one person’s relevance is another’s whimsical detour from reality.  Sometimes ‘relevance’ is construed as practicality/practicability as distinct from theory, though there is nothing as practical as a good theory[1]. And, like all the other virtues associated with utility, relevance may not always reveal itself straight away;  a good example is the current work on cognitive neuroscience which (despite the claims of proponents of brain-gym and the like) has yet to yield findings that directly translate into pedagogical strategies.  But this is no reason to stop the work.

Commissioned research projects by their nature are more immediately relevant to policy concerns, being designed to provide decision-makers with findings they can use in a pre-defined context.  The disadvantage is that commissioned researchers may not be encouraged and funded to do the explicit theorising that both connects the work into existing research knowledge and allows further investigation of the issues in future. An example of a commissioned project which did seek to theorize the issues and then to pursue further investigation via a research council grant after the end of the contract was the Variations in Teachers’ Work, Lives and Effectiveness (VITAE) research – see: http://www.dcsf.gov.uk/research/data/uploadfiles/RR743.pdf

Apart from direct commissioning, another way of increasing perceived relevance to users’ needs and concerns is to involve users from the outset in determining research priorities.  Morris argues for national strategy along such lines:

‘We need to make changes in the way research is produced. Decisions about what is researched need to be open to a wider range of influences: problems in the practical world need greater priority particularly in relation to pedagogies for particular subject areas. Processes for establishing priorities collaboratively have been developed in some fields and need to be developed in education. Research and development need to be organized in larger-scale programmes running over many years to ensure that expertise builds up and knowledge accumulates. The design of programmes and projects needs to take into account the realities of the practical context in which the outcomes are to find their use. Plans for impact need to be drawn up and end-users involved, so that those who are implicated feel ownership of the results.’ (Morris 2009)

The TLRP included policy-makers and practitioners on its board, and projects were required to involve research users, although the nature and degree of involvement varied considerably between projects.  User involvement in agenda-setting can also be a valuable way of developing mutual understandings of what is researchable – not all the questions to which politicians and teachers want answers are amenable to research, especially to empirical investigation.

Making research more accessible

There are three levels at which research can be made ‘accessible’ to people who have a professional interest in research in education but who are not specialists:  the first and most basic is letting them know what research exists; the second is enabling them easily to obtain copies of research articles and reports on the questions and themes that interest them; and the third is making sure that the material is written and presented in a style and format that is comprehensible by them.

In some ways, the basic level of accessibility – knowing what research exists in a given field – is the most challenging.  There is a staggering number not only of research studies (over 8000 articles, reports, book chapters, etc., were submitted for consideration by the education sub-panel in the Research Assessment Exercise 2008) but also of academic and professional journals in which such studies may be reported.  Nor is online publication is an answer in itself.  A report from the National Foundation for Educational Research back in 2003 identified 400,000 websites on school improvement in the UK alone, for example, and 6,000,000 world-wide.  So ‘dissemination’ is evidently not the problem – or rather, is making a problem of a different kind: so many seeds are being sown that it’s impossible to see the wood for the trees.

This has been an organic proliferation rather than a strategic growth, and it makes the task of finding research harder than it ideally should be.  It makes no sense at all for policy-makers to spend time hand-searching individual journals or websites in the hope of finding relevant material, and so it would not be surprising if they tended to rely on commissioned reports for research intelligence.  In any case, to be able to take due account of research they need a sense of what studies are being conducted before they are actually published:

‘policy makers would like to have access to a comprehensive schedule of all research from external organisations to be published say over the next 12 months.  Some of this information is available through current systems e.g. CERUK, but it is by no means comprehensive.’  (Research manager quoted in Saunders 2007b)

CERUK is a free database of ‘current education and children’s services research in the UK’: http://www.ceruk.ac.uk/ whose accuracy and coverage depend largely on researchers’ cooperation – though as researchers are also a large research ‘user’ constituency they might be thought to have a direct interest in contributing to such a database.

With regard to completed and published research, the apparatus for on-line searching – including databases, portals and search engines – has developed significantly over the last ten years, but their usefulness depends on such prior factors as the quality of journal article abstracts. The Educational Research Review guidelines say:

‘an abstract is a brief, comprehensive summary of the contents of an article allowing readers to survey contents quickly. It enables abstracting and information services to index and retrieve articles… it should be readable, brief and self-contained… [and] needs to reflect the purpose and content correctly with its focus on the most important concepts, findings or implications.’
http://www.elsevier.com/framework_products/promis_misc/edurevguidetowriting.pdf

The state of literature searching in education was critically analysed a few years ago in a paper written from the perspective of an organisation which tries to make research practically useful for teachers (Bell et al. 2002);  the authors note that abstracts do not always exist and, where they do, may give inadequate or even misleading information, particularly in online journals.  For teachers and other users, ‘abstracts have become in effect gatekeepers and could be preventing practitioner access to much potentially “useable” research’.  The paper provides examples of both helpful and unhelpful titles and abstracts.  On a similar theme, Sebba (see: http://www7.nationalacademies.org/core/Sebba_Remarks_Journal.html) notes that researchers feel impelled to make the titles of their articles catchy rather than prosaically informative, and that abstracts too often function as introductions to their articles rather than as summaries.  This means that a large proportion of articles need to be called up and read in their entirety before the extent of their relevance or otherwise becomes evident – which in turn makes literature searching and research reviewing expensive and laborious.

To address these and other issues, the so-called REPOSE guidelines for reporting empirical research in education have been developed in consultation with journal editors: see http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=759 (and also Newman and Elbourne 2005). These advise that abstracts should be structured to include:  

  • background
  • research question
  • methods
  • results
  • conclusions

The second accessibility issue is about making it easier for policy-makers and other users to get hold of research findings.  The DCSF has a publication strategy which includes written reports, presentations, workshops, on-line summaries, Ministerial briefings and answers to Parliamentary Questions, and broadcast and press media, most of which can be accessed via the website. See: http://www.dcsf.gov.uk/research/data/general/Analysis%20and%20Evidence%20Strategy%20(AES-2008).pdf  Chapter 5, Access to Our Evidence

The majority of non-commissioned research has traditionally been published in subscription-only journals but, as remarked before, over the last decade there has been a concentrated development of online open access resources.  Many of these are also attempting to structure and organize research in ways that make it more useable – for example, by making it possible to look for trustworthy research under a particular educational theme using key terms, rather than having to search by a known journal title on the one hand or with a generic search engine that throws up too many random items on the other.  The most well-known online resources of this kind in education, each with somewhat different aims and audiences, are probably: 

The third accessibility issue is about the quality of the writing (rather than of the research itself), and how it can be made ‘fit for audience’.  Researchers will have recourse to the various academic journal guidelines (like the those provided by Educational Research Review above) when thinking about submitting an article.  But, from the perspective of research users in government and other agencies, these are the important points to consider:

  •  Clarify questions and theory of change;
  • Provide the challenge function;
  • Identify impacts and ways of implementation;
  • Identify the key message(s) from research;
  • Identify limitations of evidence;
  • Communication clearly and concisely;
  • Use appropriate format (e.g. 1:3:25);
  • Don’t patronise/over-simplify/dumb-down;
  • Don’t ‘blind with science’, jargon, sociologese;
  • Persistence and Opportunism.

 (Presented by Richard Bartholomew, DfES, in Seminar 5 of the TLRP seminar series ‘Making a Difference: working with users to develop educational research’: http://www.tlrp.org/themes/seminar/edwards/current.html

Researchers should also read the guidance materials on the websites of (i) the professional association for educational researchers and (ii) the government’s social research unit:

Researchers – and research managers – may find the TLRP resources on communications and impact http://www.tlrp.org/capacity/rm/comms_impact.html helpful too.

TASK
Draw up a list of the main differences, as you see them, between the guidelines set by one or two academic journals of your choice and those published by BERA and GSR to improve communication with non-academic audiences.  What are the key features they have in common?  Selecting three pieces of research writing – a peer-reviewed journal article, a feature in a professional journal and a government-commissioned research summary – compare them in terms of their communicability and usefulness.  Do guidelines help?

Increasing the policy read-off of research

One of the problems noted earlier with the evidence-based education agenda is that it is very hard to take a direct read-off for a whole policy direction from individual research projects, which is one of the reasons why researchers find it challenging to draw out convincing and workable implications from their research.  

So some researchers are starting from the other end – what social and economic ‘levers’ have policy-makers used across different public policy sectors and what can this tell us about the implementation of particular theories of change? This name given to this approach is realist synthesis, of whom Ray Pawson is a leading exponent.  Here is his explanation of what the approach offers:

‘Realist synthesis utilises a “generative” approach to causation. According to this perspective it is not “programmes” that work: rather it is the underlying reasons or resources that they offer subjects that generates change. Causation is also reckoned to be contingent. Whether the choices or capacities on offer in an initiative are acted upon depends on the nature of their subjects and the circumstances of the initiative. The vital ingredients of programme ontology are thus its “generative mechanisms” and its “contiguous context”. Data extraction in realist synthesis thus takes the form of an interrogation of the base-line inquiries for information on “what-works-for-whom-in-what-circumstances”. The approach to generalisation is also different. The policy community is not offered a “best buy” (approach “x” or case “y” seems to be the most successful) but a tailored, “transferable theory” (this programme theory works in these respects, for these subjects, in these kinds of situations).’  (Extract from: ‘Supplementary reading 5: The Promise of “Realist Synthesis”’ downloadable from http://www.leeds.ac.uk/realistsynthesis/supreadings.htm) 

Pawson’s work has developed far enough to provide exemplars of theories of programme change (which he calls ‘middle-range theories’) – such as incentives, zero tolerance, performance targets, and naming and shaming – together with a model for conducting research syntheses on this basis; but so far it seems that no large-scale realist-syntheses have been conducted which could test the approach and particularly its technical practicability in terms of reported studies that could be used.

Strengthening the roles and skills of research mediators and knowledge brokers

It has become a truism that for research to be influential in policy and/or practice it needs to be ‘translated’, made into the kind of knowledge that ‘travels’; and that this requires the role and skills of research mediators, knowledge brokers and the like.  For arguments specific to education, see, for example, CUREE website http://www.curee.co.uk/, Edwards et al. 2007, Saunders 2004.

It is not easy to find discursive literature about the mediator’s role and skillset (though Saunders 2007b looks at the research mediation role of research managers), but we can surmise that the requisite skills would include:

  • a good sense of the direction that public opinion and policy are moving in, so as to be able to give early alerts to policy-makers of possibly relevant research, and to give researchers advice about how to frame their findings;
  • good social networks and communicative relationships with both ‘producers’ of research and policy colleagues;
  • a capacity to empathise with the problems and challenges of both;
  • a capacity to understand and interpret the technical complexities of research, to develop an intelligent overview of research in a given field and to identify the potential implications for policy and practice agendas;
  • facility in writing attractively and convincingly for lay audiences.

 
In 2006, the ESRC set up a research brokers’ scheme that was designed ‘to help the ESRC achieve one of its four core strategic objectives: to increase the impact of its research on policy and practice’:  http://www.esrc.ac.uk/ESRCInfoCentre/Images/research_broker_specification_tcm6-14266.pdf – although it has not proved possible through an internet search either to locate a description of the required skills for the post nor to find out what the scheme achieved.  With a detectably similar rationale, a secondment from the ESRC to GSRU was made in 2008 to ‘help… social science feed into policymaking’ (BERA 2008). The post-holder was reported as commenting: ‘One of the principal challenges of this project will be to turn around the ways in which academia communicates with government, which are not always successful.’  The secondment was for nine months but, again, it has not been possible to find out how successful the initiative has been.

In practice, research mediation seems most often to have been concerned with the re-presentation of research findings and in particular with enlisting the skills of experienced journalists who are also sympathetic to research and evidence – the TLRP and the National Research and Development Centre for Adult Literacy and Numeracy (NRDC) have both employed senior journalists on long-term contracts.

Changing the mode from ‘knowledge transmission’ to ‘co-construction of knowledge’

At some point in the ‘transfer’ or ‘application’ of research the responsibility for creating knowledge ceases to rest with researchers alone.  As ideas and evidence from research become active in a policy or practice environment, processes of selecting, appraising, interpreting and drawing inferences from research findings go on in a more or less formal and systematic way.  Conventionally, researchers have relinquished their responsibility for creating knowledge at the point of finishing the project report or giving a summary presentation, though policy-makers have often wanted researchers to go somewhat further in helping them tease out the policy implications of the research (see, for example, Richard Bartholomew’s slide above).  When researchers can do this, it is already a form of ‘co-constructing’ knowledge; and where conditions are right the discursive process may be extended into a knowledge-creating relationship which is collaborative and reflexive, in which the provisionality of research findings is accepted and their integrity protected, and where the contribution of different kinds of expertise – technical, academic and political – is acknowledged. 

The process is obviously harder where there is little or no ongoing contact between researchers ‘producers’ and ‘users’, and indeed geographical as much as metaphorical distance between them is cited as a key problem in studies of research impact.  There are ways round this – as was mentioned above, CUREE have developed a wide-ranging portfolio of materials which help to convert research ideas and evidence into professional learning activities for practitioners.  These start from the kinds of professional, practical and ethical issues which preoccupy practitioners, rather than with the research.  There are examples of these on the GTC (England) http://www.gtce.org.uk/research/cpd_resources/. 

As has been mentioned, TLRP and AERS have been committed to working closely with potential users of the research throughout the research process.  See, for example, http://www.tlrp.org/users/userfullnames.html:

‘Through professional partnerships between researchers and users the relevance, significance, quality and impact of each project’s research can be enhanced. Indeed, research users often become wonderfully effective “champions” for the outputs of research activity.’

A particular instance of co-construction of knowledge was the workshop that TLRP organized with policy-makers at the DfES and other national agencies in 2005 on the outcomes of learning. The key features of this activity can be summarized as follows:

  1. researchers took the initiative in setting up a workshop in which policy-makers could engage in ‘low-stakes’ dialogue with researchers about the ideas and findings from some new and important research projects;
  2. the workshop presentations underpinned empirical research findings with learning theory and espoused values of social justice;
  3. the presentations synthesized the common issues and messages across different projects conducted in different sectors and with different methodologies;
  4. discussion was then organized in small groups on different aspects and implications of the research;
  5. after the workshop, stakeholder views and priorities were combined with research ideas and evidence, and assembled in a publication designed to address a major new education policy initiative at that time, Personalised Learning;
  6. researchers worked ‘behind the scenes’ with a research manager at the DfES to sustain channels of communication.

Obviously, a great deal hangs on how skilfully these occasions are chaired and facilitated, and whether the policy-makers present at them are high enough up the food chain – but in this whole area people are learning by doing.   Often influence is evident only with hindsight, a good example being the EPPE (Effectiveness of Pre-School Primary Education) research programme, which – through a combination of a compelling quantitative and qualitative longitudinal research design and sustained working relationships both with policy-makers in government and practitioners in the field – can now be seen to have played a crucial role in shaping policy on pre-school provision over a number of years.

Edwards et al. (2007), citing Gibbons’ work on Mode 1 and Mode 2 knowledge creation, wonder if user engagement may eventually lead to ‘a revolution in relations to make them more fluid and less managed [which] might reveal new options for education as a social science’.  They go on:

‘This step [of self-organising participants jointly producing socially robust knowledge] is some way down the line from the user engagement we found in TLRP projects.  It suggests a move into a fluid system of distributed expertise; where mediation across boundaries is not necessary and people bring to bear their specialist knowledge on jointly-constructed problems of practice.’

Finally in this section, then, we look at an example which may have exhibited some of those characteristics.

Changing the model from ‘impact’ to ‘diffusion’

The following extended excerpt is about the way that ‘dissemination’ of the PELRS (Developing Pedagogies with E-Learning Resources) research project was conceived and (at least partially) enacted.

‘PELRS by that point was no longer a bounded “project” in the usual sense… there was some important, contextualised, learning for the way we go about policy;  most of all, in trying to  re-formulate the issues and challenges of “dissemination” and “scaling up”.  Among other things, this meant taking the risk that other agencies and organisations wanted and were able to make the PELRS principles and practice their own. This exchange of ideas between researchers, practitioners and policy advisers took place at meetings of the Advisory Group, interactive seminars, conversations, presentations and papers, and through the PELRS website. 

This model of dissemination does not rely on the performative production of a text, the “final report”, but instead on a whole range of loosely-related tools in the Vygotskyan sense operating in real time, which allows a continual re-invention, rather than a closing-down, of the research.

The PELRS work went on to have further applications to the policy environment.  Its ideas and practices spoke directly to the government agenda on “personalised learning” – an agenda which the GTC supported but which [they] also wished to strengthen and deepen.  The GTC… used the ideas and evidence from the PELRS project for several of its public policy pronouncements, including:
 

  • response to the Gilbert Review ‘Teaching and Learning in 2020’, August 2006
  • formal advice to government on personalised learning, March 2007
  • submission of evidence to the Primary Review, March 2007
  • response to the QCA Secondary Curriculum Review, May 2007

…  At the time of writing, the GTC continues to draw heavily on PELRS material for conferences, seminars and colloquia around the theme of personalisation. This all suggests that the ‘return on investment’ in PELRS has been high – for the schools and teachers involved and for the wider professional and policy communities.’ (excerpted from Saunders and Somekh 2009)

An inescapable message from the foregoing discussion and the range of examples given is that increasing the ‘impact’ of research requires as much intellectual and social resource – dispersed through the system – as the research activity itself. Traditionally, research monies are almost all front-loaded into the carrying-out of research projects, with rather token requirements for ‘dissemination’ which do not take good account of what we know even at this relatively early stage about research communication and influence. 

In future, it is likely that we shall see more awareness-raising and professional development for the skills of mediation and negotiation; and a greater focus on two-way modes of communication, where ‘research meets its publics’ (Edwards et al., op.cit.) and learns from them. Workshops and the like are useful not least because – being constructed around conversations between mixed groups of academic researchers, policy staff and advisers, and practitioners – they help us to move away from dualities and polarities, and embrace appropriate complexity and plurality.  If we think about research-informed decision-making in this dispersed and reciprocal way rather than as part of the modernising grand plan of central government, opportunities may continue to open up that offer high leverage and value-for-use without compromising research integrity.

TASK
Download and read the TLRP publication summarising the impact that TLRP has had on the development of teaching and learning in the UK:  Impact and Significance

http://www.tlrp.org/pub/documents/ImpactLeaflet.pdf

Draw up an outline ‘dissemination and impact’ strategy for a research project with which you are familiar and which you think deserves to be influential in educational policy.

 

[1]     pointed out by LEWIN, K. (1951). Field Theory in Social Science: Selected Theoretical Papers. New York, Harper Row.

Models for assessing the impact/influence of research

In the light of the above discussion, we will no doubt agree with Bates (2002) who suggests that, since the impact of educational research on both policy and practice is complex and indirect rather than linear and straightforward, the methodologies employed in assessing ‘impact’ need to be similarly complex. In thinking about how to assess and evaluate the impact of research, we will need to look, for example, at:

  • the timescale over which research might be expected to have an influence on the way policy-makers think and act (and ditto practitioners);
  • any changes in the ‘climate of opinion’ which might affect how particular research findings are perceived and received by policy-makers and practitioners;
  • the processes of knowledge mediation and the roles of intermediary individuals and organisations;
  • the strength of social networks and communities of practice through which ideas and evidence flow;
  • the ways in which research-literate practitioners engage in and with research;
  • how and why research evidence becomes embedded in organisational processes;
  • how and why research evidence becomes embedded in regulatory frameworks.

The Research Unit for Research Utilisation has done much useful ground-clearing and analytical work in this area. For example, the abstract to a recent paper notes:

‘Social science research undoubtedly does impact on public policy and practice but such non-academic impacts are rarely amenable to precise, quantitative metrics. In the interests of accountability, it is however possible to find proxy indicators of connectivity with research users and these may form steps toward impacts. Understanding these connections can lead to a deeper appreciation of the factors that shape the processes leading to research uptake.’ (Meagher et al. 2008)

The study in question adopted a qualitative approach to identify the flows of knowledge, expertise and influence that take place during the process of knowledge transfer; the researchers trialled a method for assessing policy and practice impacts of social science research, from which they identified five factors that influence and enhance the process of knowledge exchange between researchers and users.

TASK

Read the RURU paper and, in discussion with other colleagues, add your own examples from research projects with which you are collectively familiar to illustrate the five factors that enhance knowledge exchange and increase the possibility of impact.

How could you use the framework to enhance the influence of future research for which you have responsibility?

To comment on any aspect of this resource, please e-mail: lesley@lesleysaunders.org.uk

Further reading and bibliography

BATES, R. (2002). ‘The impact of educational research: alternative methodologies and conclusions’, Research Papers in Education 17, 4, 403–8.

BELL, M., CORDINGLEY, P., CURTIS, A., EVANS, D., HUGHES, S. and SHREEVE, A.
(2002). ‘Bringing research resources to practitioner users via web technology: lessons learned to date’. Paper presented at the annual conference of the British Educational Research Association, University of Exeter, England, 12–14 September.

BERA (2008) ‘Philip Cowley: your man in the Treasury’, Profile, Research Intelligence 105, November.

BIESTA, G. (2009). ‘Educational research, democracy and TLRP.’ Keynote presentation at the TLRP Research Conference, London, Institute of Education, 19 March. Downloadable at: http://www.tlrp.org/conference/2009/downloads.html

BLACK, P. and WILIAM, D. (1998). Inside the Black Box: Raising Standards Through Classroom Assessment. London: NFER Nelson.

COFFIELD, F., EDWARD, S., FINLAY, I., HODGSON, A., SPOURS, K., STEER, R. and GREGSON, M. (2007). ‘How policy impacts on practice and how practice does not impact on policy’, British Educational Research Journal, 33, 5, 723–41.

CORBYN, Z. (2008). ‘Councils admit defeat in hunt for algorithm to show economic value of research’, Times Higher Education, 6 March. Online at: http://www.timeshighereducation.co.uk/story.asp?storycode=400973

CORDINGLEY, P. (2000). ‘Teacher perspectives on the accessibility and usability of research outputs.’ Paper presented at British Educational Research Association Conference, Cardiff University, 7-9 September.

CORDINGLEY, P., BELL, M. and EVANS, D. (2007). ‘Communicating new knowledge to practitioners’. London: TLRP. Online at http://www.tlrp.org/capacity/rm/wt/bell1.html

DAVIES, P. (1999). ‘What is evidence-based education?’, British Journal of Educational Studies, 47, 2, 108–21.

DAVIES, H.T.O., NUTLEY, S. and WALTER, I. (2008). Why “knowledge transfer” is misconceived for applied social research Journal of Health Services Research and Policy, 13, 3,188–90.

DEPARTMENT OF EDUCATION, TRAINING AND YOUTH AFFAIRS (DETYA) (2000). The Impact of Educational Research. Canberra: Commonwealth of Australia. Downloaded June 2002 from: www.detya.gov.au/highered/respubs/impact/overview.htm

EDWARDS, A., SEBBA, J. and RICKINSON, M. (2007). ‘Working with users: some implications for educational research’, British Educational Research Journal, 33, 5, 647–61.

FIELDING, M. (2003). ‘The impact of impact’, Cambridge Journal of Education, 33, 2, 289–95.

FIELDING, M., BRAGG, S., CRAIG, J., CUNNINGHAM, I., ERAUT, M., GILLINSON, S., HORNE, M., ROBINSON, C. and THORP, J. (2005). Factors Influencing the Transfer of Good Practice London: DfES. Downloadable at:http://www.dcsf.gov.uk/research/data/uploadfiles/RR615.pdf

FURLONG, J. and OANCEA, A. (2005). Assessing Quality in Applied and Practice-based Educational Research: A Framework for Discussion Oxford: OUDES.

HAMMERSLEY, M. (2002). Educational Research, Policymaking and Practice, Paul Chapman.

HANNEY, S. (2004). ‘Assessing the impact of research on policy: concepts, models and methods’. Paper presented at a seminar, Assessing Research Impact, St Andrews University, 15 January.

LEVIN, B. (2003a). ‘Increasing the impact and value of research in education’, Educators’ Notebook, 14, 1.

LEVIN, B. (2003b). ‘Increasing the impact of research’, presentation to University of South Australia, 3 March. Downloaded 24 July 2003 from: http://home.cc.umanitoba.ca/~levin/increasing_impact_ofresearch_files/frame.htm

LEVIN, B. (2004). ‘Making research matter more’, Education Policy Analysis Archives, Downloadable at: http://epaa.asu.edu/epaa/v12n56/v12n56.pdf

MEAGHER L., LYALL, C. and NUTLEY, S. (2008). Flows of knowledge, expertise and influence: a method for assessing policy and practice impacts from social science research’ Research Evaluation, 17, 3, 163-73

MORRIS, A. (2004). ‘Research impact’. Paper presented at Market Research Society Conference ‘Social Policy Research in the Information Age’, London, 17 February.

MORRIS, A. (2009), Evidence Matters: Towards Informed Professionalism for Educators. Reading: CfBT. http://www.cfbt.com/evidenceforeducation/pdf/Evidence%20Matters%20v5(WEB)%20FINAL3.pdf

MOSS, G. and HUXFORD, L. (2007). ‘Exploring literacy policy-making from the inside out’. SAUNDERS, L. (Ed) Educational Research and Policy-making: Exploring the Border Country Between Research and Policy London:  Routledge.

NEWMAN, M. and ELBOURNE, D. (2005). ‘Improving the usability of educational research: guidelines for the REPOrting of primary empirical research Studies in Education (The REPOSE Guidelines)’. Evaluation and Research in Education, 18, 4, 201–12.

NUTLEY, S.M., PERCY-SMITH, J. and SOLESBURY, W. (2003). Models of Research Impact: a Cross-Sector Review of Literature and Practice. London: Learning and Skills Development Agency.

OAKLEY, A. (2003). ‘Research evidence, knowledge management and educational practice: early lessons from a systematic approach’, London Review of Education, 1, 1, 21–33.

PERCY-SMITH, J., BURDEN, T., DARLOW, A., DAWSON, L., HAWTIN, M. and LADI, S. (2002). Promoting Change through Research: the Impact of Research in Local Government. Joseph Rowntree Foundation.

SAUNDERS, L. (2004). Grounding the Democratic Imagination: Developing the Relationship between Research and Policy in Education. Professorial Lecture. London: Institute of Education.

SAUNDERS, L. (Ed) (2007a). Educational Research and Policy-making: Exploring the Border Country Between Research and Policy. London:  Routledge.

SAUNDERS, L. (2007b). ‘Go-betweens, gofers or mediators? Exploring the role and responsibilities of research managers in policy organisations’. In: SAUNDERS, L. (Ed) Educational Research and Policy-making: Exploring the Border Country Between Research and Policy. London:  Routledge.

SAUNDERS, L. and SOMEKH, B. (2009). ‘Action research and educational change: the professional voice of teachers in curriculum and pedagogy’.  In:  NOFFKE, S. and SOMEKH, B. (Eds) Handbook of Educational Action Research.  London:  Sage Publications.

SOMEKH, B. (2007). ‘The interplay between policy and research in relation to ICT in education in the UK: issues from twenty years of programme evaluation In: SAUNDERS, L. (Ed) Educational Research and Policy-making: Exploring the Border Country Between Research and Policy. London:  Routledge.

START, D. and HOVLAND, I. (2004). Tools for Policy Impact: A Handbook for Researchers. Overseas Development Institute. Downloaded 5 September 2005 from:http://www.odi.org.uk/RAPID/Lessons/Index.html

TAYLOR, C. (2007) ‘Resources from the Research Capacity Building Network and Journal.’ London: TLRP. Online at http://www.tlrp.org/capacity/rm/wt/taylor

THOMAS, G. and PRING, R. (Eds) (2004). Evidence-Based Practice in Education. Conducting Educational Research series. Maidenhead: Open University Press.

WALTER, I., NUTLEY, S. and DAVIES, H. (2003). ‘Developing a taxonomy of interventions used to increase the impact of research’. Downloadable at: http://www.ruru.ac.uk/PDFs/Taxonomy%20development%20paper%20070103.pdf

WEISS, C.H. (1979). ‘The many meanings of research utilisation’, Public Administration Review, 39, 5, 426–31.

WEISS, C.H. (1991). ‘Policy research: data, ideas or arguments?’ In: WAGNER, P., WEISS, C.H., WITTROCK, B. and WOLLMAN, H. (Eds) Social Sciences and Modern States: National Experiences and Theoretical Crossroads.Cambridge: Cambridge University Press.

WILLINSKY, J. (2000). If Only We Knew: Increasing the Public Value of Social-Science Research. London: Routledge.

Return to top of page