The Processes of Commissioning Research

Return to front page of this collection of resources

Contents

1. Introduction
2. An explanation of some key terms
3. Commissioning research: organisational priorities and processes
4. Some specific organisational factors affecting whether, when and how research is contracted
5. Framework Agreements and partnership and consortium arrangements
6. The research specification and ITT
7. Points to bear in mind when responding to an ITT
8. Evaluating bids/proposals, including criteria
9. Asking for/receiving feedback on unsuccessful bids/proposals
10. Pre-contract negotiation
11. Ways in which research commissioning might be improved
12. Further reading and bibliography

Introduction

This resource provides an overview of the key issues – accompanied by case study exemplification and points for discussion – involved in research commissioning in the public sector, including: how and why decisions to commission research and/or evaluation projects are made, the processes and procedures involved in issuing an invitation to tender, the criteria that may be used for short-listing/selection of preferred bidders, and feedback to unsuccessful tenderers. It also offers pointers to critically evaluating research commissioning and to identifying how the process might be improved.

Explanation of some common terms

Contract research is research commissioned and procured by an external organisation – in the private or public sector – within defined parameters of time, budget, objectives and sometimes design. These parameters constitute the main components of the contract, which is a legally-binding agreement between the agency and the research provider (whether individual or institutional). The degree of detail and the specific terms of any contract vary from agency to agency, and most research providers have legal/financial specialists whose job is to check the small print to ensure that the terms are acceptable from the institution’s point of view. It is customary to refer to the commissioning agency (and/or the representative thereof) as the client or customer, to whom the researcher as contractor is providing a specified service; this terminology itself indicates that the relationship is a much more direct, and directive, one than that between researchers and grant-givers or funders.

Much, though by no means all, of the research that is commissioned by public (or private) bodies is an evaluation of a particular policy (in its various forms of a model, an initiative, a programme, a project, a service, a product, an event), usually of the extent to which the policy is meeting or has met its aims, together with, for example, an assessment of its impact on different stakeholder groups, key success factors, major barriers and risk factors, any unintended consequences and possible recommendations for future actions. Evaluations are consequently not open-ended, especially in terms of outcomes, in the way that research projects may be, but they make extensive use of research methods and instruments – which is why the terms are often bracketed together, or even used interchangeably.

The commissioning process is set in train through a specification or brief, normally in writing (though with very varying degrees of steering and/or elaboration), that sets out the research/evaluation remit in the form of objectives, outputs and timeline. Many specifications also define the features of the preferred design and methodological approach, and some even specify particular methods. Most organisations like to offer their overview of the key research/evaluation issues, including the main policy drivers; they may also specify what is excluded from the research/evaluation (such as any discussion of the value of the underlying policy). Increasingly, they specify some dissemination requirements. Organisations differ about whether they include budgetary information or not.

The next stage is for the commissioning organisation to issue the specification along with a formal invitation to tender. This may appear as a public advertisement in relevant media or be distributed to selected research providers who have previously been identified according to one or other method/set of criteria. The invitation to tender (often abbreviated to ITT – not to be confused with Initial Teacher Training!) may ask interested research providers to send a brief expression of interest in order that the organisation can gauge (i) the extent of expertise/interest in the field; (ii) the level of internal resource the commissioning process will require; or it may ask that a full proposal is submitted directly.

The proposal – also known as a bid or a tender – is the document that is prepared by the research provider in response to the invitation to tender. This will need to address all the specified components and show how each of the requirements will be met, or else provide cogent reasons for proposing deviations/alternatives. Organisations vary in the level and type of information/rationale they require, and in the clarity with which they express these requirements, so research providers need to weigh up the pros and cons of using off-the-shelf proposal formats.

Following receipt of proposals, organisations often draw up a shortlist of bidders before offering a contract; and shortlisted researchers are then usually invited to an interview where they have an opportunity to make a presentation, answer questions and engage in some preliminary discussion about the project. Following this, the organisation may identify a preferred bidder, with whom they will undertake negotiations around, for example, timeline and/or budget, before issuing a contract. Occasionally, a project may fail to be commissioned, even at this late stage.

Organisations in the public sector are bound by generic procurement regulations, which apply to any external requisition of goods or services and are intended to ensure propriety, efficiency and value for money in public expenditure. If the value of the goods/services exceeds a set threshold (which currently is not very high at 133,000 euros for government departments/agencies), EU regulations come into force.

1. Commissioning research: organisational priorities and processes

These resources assume that researchers – and research managers/advisers – will benefit from knowing more about, and reflecting on, the political and organisational context for research commissioning (though of course researchers can just wait passively for calls for expressions of interest or invitations to tender to be issued).  This resource is concerned with typical organisational processes and procedures that influence the business of commissioning research.  The micro-politics within an organisation, the realpolitik of processes and procedures at a managerial level, are a critical factor governing at least some of what happens in research commissioning – yet there is little or no direct evidence let alone academic literature which sheds empirical or theoretical light on them, as distinct, say, from good practice guides, mandatory frameworks and so forth (as exemplified throughout these Resources).  So some of this section is based on inference and reasonable surmise.

In trying to understand what will stimulate and feed an organisation’s appetite for research, a good place to start is with the organisation’s published policy priorities – see, for example, the 2007 Children’s Plan published by English government’s ministry for education (currently the Department for Children, Schools and Families).  Inevitably, such policy documents are rhetorically framed, but they also show fairly clearly what is likely to be within and out of scope in terms of research and evaluation foci over a medium-term timescale. Researchers should take careful account of these signals in developing their capacity for responding to calls for expressions of interest or invitations to tender;  they will want to think about the implications of an organisation’s policy agenda – both its general direction and the specific objectives – for the kinds of work might need to be commissioned.

As a general crude rule, organisations are interested in what bangs they can get for their bucks: at policy level, finding out what is the most (cost-)effective way of raising literacy standards amongst children and adults, for example, given the scale of expenditure involved for different design and implementation options, the numerous other uses to which the same monies could justifiably be put, and the value the public will place on it.  And at a research project level, what kind and scale of study conducted in what timeframe is likely to provide the most clear and credible information about the initiative’s impact on literacy for key target groups of children and adults.  So this is a world of compromises, best fits and limited resources, not of ideal scenarios with money no object.

Where research might fit into particular phases in the ‘policy cycle’ is also something to reflect upon:  when an embryonic policy agenda has been signalled in a government Green Paper, for example, or even merely outlined in a sentence or two of a Minister’s speech, some early thinking on options for implementation, possible barriers and pitfalls, appropriate success criteria and so forth might well get a hearing.  Whereas policy initiatives that are already ‘mature’ in the sense of being established on the ground through a variety of structures, programmes and activities are not good candidates for formative policy thinking;  what they will require are impact evaluation studies, whether national/large-scale or small-scale/local, or a combination of both.

Next, both researchers and research managers/advisers need to consider what might be the relative importance, in any given policy context, of the following types of research and evidence, and what the distribution of requisite skills is amongst potential contractors (list taken from ‘Professional Policy Making for the Twenty First Century’

• expert knowledge
• existing domestic and international research
• existing statistics
• stakeholder consultation
• evaluation of previous policies
• new research, if appropriate
• secondary resources, including the internet
• analysis of the outcome of consultation
• costings of policy options
• results of economic or statistical modelling.

 
You might also want to consider if there is anything you would want to add, in terms of categories of useful evidence and/or generative methodological approaches, to this list.

As well as their policy remit and priorities, organisations may produce/publish a separate policy and/or strategy for research – see, for example, the DCSF’s 2008 Analysis and Evidence Strategy. This is a comprehensive overview of the Department’s research activity and how it relates to the policy agenda;  it gives a clear indication of what kinds of analytical, evaluative and research work is being conducted and planned, which includes:

• ‘Launching a range of research and evaluation studies to support Aiming High for Disabled Children: Better Support for Families which sets out the Government’s strategy for improving the lives and well-being of disabled children and their families…
• Providing the data and analysis to assess the progress being made in improving the educational attainment and emotional and behavioural health of children in care or leaving care…
• Developing a more broadly-focused annual Children in Need census, and using the contextual data collected by the Tellus survey to examine which types of children are most likely to report bullying and the situations in which this is most likely to occur…
• A range of statistical modelling work, for example, of test and exam results at national level and investigating the relationships between pupil absence and pupil characteristics, attainment and progress…
• A review of the evidence underpinning our policy stance on social mobility and links with child poverty, focusing on what has changed since the last evidence-gathering exercise and what this means for the strategic direction of policy…
• Using the growing wealth of data from the Longitudinal Study of Young People in England (LSYPE) to understand the associations between “risk-signalling” activities (e.g. alcohol and drug-taking) and earlier attitudes or circumstances in young people’s lives, so that we might better learn where and what preventative strategies might be most effective and continue with research to understand how to increase the resilience of young people so that they are less likely to make harmful choices for themselves in the first place…
• Establishing a Parents’ Panel and undertaking associated surveys to collect vital information about what parents and carers think about the Department’s work, in particular their views about the services which they and their children use.  It will also help us to establish a baseline indicator of parental confidence from which progress can be tracked in future years, and collect and analyse information about how opinions differ between parents and carers according to their characteristics…’

 
There is a conspicuous emphasis here on longitudinal surveys and statistical analyses, but research reviews, consultations and qualitative studies also figure: Annex B of the document provides a list of all the internal and externally-commissioned studies that were approved for the 2008–09 financial year.  Researchers can use the Department’s web-based facility to view the new research projects that will be commissioned, and to register expressions of interest in being invited to tender for them.  

Equivalent sites for the Scottish government can be found here and here; for Northern Ireland here; and at the Welsh Assembly here.  

Some new education research is commissioned by teachers’ and headteachers’ professional associations/trade unions – for example, the National Union of Teachers has sponsored or directly commissioned influential research into teachers’ professional learning and development; and the Association of Teachers and Lecturers is well known for its research reports on assessment and testing. Such organisations invariably commission research to support their campaigns and members’ services and, although their research budgets tend to be small, the media coverage and political influence of their research is comparatively impressive.

Turning to third-sector organisations more generally, one example is the Joseph Rowntree Foundation , an independent charitable organisation which funds research in education, with the explicit aim to ‘understand the root causes of social problems, identify ways of overcoming them, and show how social needs can be met in practice’. The website makes it clear that the Foundation does not accept speculative enquiries for funding, but invites submissions in response to a call for proposals.  It does not generally support:

• ‘unsolicited proposals;
• educational bursaries or sponsorship for individuals for research or further education and training courses;
• proposals that do not have the potential to make a difference to policy or practice in the UK;
• projects outside the topics within our current priorities;
• development projects that are not innovative;
• development projects from which no general lessons can be drawn…’

 
Another charitable body that supports educational research in pursuit of social amelioration is the Nuffield Foundation :

‘Nuffield has supported innovative research and development in education for over 50 years. Current supported work builds on this tradition, prioritising areas in which influence on policy and practice can be brought to bear.  There are four areas of priority at the moment:

• education 14-19
• speech and language difficulties
• assessment
• curriculum policy and practice

‘We support a range of different kinds of activities in these areas: research, intervention studies, development projects, reviews, seminars, capacity building and practical work… As part of our attempt to consolidate and target support we do not accept unsolicited grant applications for work on any aspect of education… From time to time we may launch new areas and call for proposals – these will be posted on the front page of the website… 

‘The Foundation keeps an ‘open door’ to proposals of exceptional merit for research projects or practical innovations that lie outside our main programme areas, but that meet Trustees’ wider interests. These must have some bearing on our widest charitable object – ‘the advancement of social well-being’. We are particularly interested in projects which identify change or interventions which will have a practical implications for policy or practice, or that will improve the quality of research evidence in areas of public debate.’

As well as this kind of official and practical information, other intelligence – if it can be accessed – can be very useful too:  Rudd et al. (2004) identified the different types of documentation, not always in the public domain, that can throw light on organisations’ thinking, such as business strategies and corporate plans;  and the report also listed the search strategies the research team used to try to obtain relevant documents, including ‘website searches, database searches, electronic calls for information, individual letters and face-to-face discussions’.

This is because – as well as being familiar with the organisation’s ostensible policy priorities – what you ideally need to know is what is driving, or inhibiting, the organisation’s willingness and capacity to commission and use research at a pragmatic or even purely opportunistic level.  For it is quite possible, if/when you have been contracted to conduct a research project, to get to the point of reading the client’s comments on your draft report before you get a glimmer of what really lies behind the commission.  These include internal power plays for scarce resources, competition for influence among stakeholders and/or the general public, demonstration of commitment to consultation, desire for early indications of an initiative’s success – none of which is proof of institutional bad faith or a disingenuous attitude towards evidence, but which it is helpful for a contractor to know about early on in the process.  Insights may or may not be forthcoming during the tendering process, though you can try probing for background information at this point. You can certainly be more proactive in doing so once you have been awarded the contract, and you will want to equip yourself with as much intelligence as possible in order to manage any unwritten expectations. Research managers/advisers often play a crucial, and difficult, mediating role here – see Resource 3, Section 10 [HYPERLINK HERE], for some discussion on the processes of mediation and negotiation.

At least as important as the realpolitik is the question of the policy question:  identifying and refining what needs to be known and who needs to know it.  Not all policy problems can be turned into research, let alone researchable, questions, of course, and a key role for research managers/advisers is to help their policy colleagues work through which aspects and dimensions of the problem are subject to empirical investigation and which are not.  This is part of the process of consulting internal constituencies – see the next section.

In principle, research can as well, and as valuably, offer non-empirical policy-relevant knowledge;  it may, for example, ‘put the problem in a broader historical or social context; it may confer deeper understanding by correcting assumptions, providing insights into people’s experiences, revealing the complex nature of the problem’ (Bridges et al. 2008a) – though these contributions are more likely to be realised in the form of a preliminary phase or underpinning component of a mainly empirical project than as a freestanding contract.

TASK

Download the English government’s strategy for children at http://www.dcsf.gov.uk/childrensplan/downloads/The_Childrens_Plan.pdf.  Choose ONE chapter (or section within a chapter), and draw up a list of the key questions that the Parliamentary Committee on Children, Schools and Families might ask about the strategy’s impact in three years’ time.  Identify what kinds of evidence the government might wish to provide by way of success criteria in response to ONE of these questions;  and then outline a programme of appropriate research studies (singly or in combination) that might help to provide such evidence.  You can compare your ideas with the 2008–09 programme of research Annex B.

(Note: The Children, Schools and Families Committee is appointed by the House of Commons to examine the administration, expenditure and policy of the Department for Children, Schools and Families.  It also covers the work of non-departmental public bodies such as Ofsted;  see: http://www.parliament.uk/parliamentary_committees/parliamentary_committees16.cfm)

OR

Read the information about research funding on any of these organisations’ websites: Joseph Rowntree Foundation, Nuffield Foundation, Esmee Fairbairn Foundation, Sutton Trust, Wellcome Trust. Then draw up a list of draft criteria, as if for discussion by the board of trustees, that will guide applicants in responding to a call for new research proposals to be funded by the charity.

2. Some specific organisational factors affecting whether, when and how research is contracted

It is worth setting out briefly the kinds of organisational factors that can affect whether, when and how research is contracted, including timescale, budgets, procurement rules and consultation with internal and external constituencies.

Timescales are usually closely tied to the budget-setting process: usual practice is for proposals for research expenditure to be submitted on an annual basis, alongside funding proposals from other internal teams/divisions, for approval by whatever executive authority exercises financial powers and duties on behalf of the organisation – this authority may be held by groups ranging from a small senior leadership team to a large board of trustees, with corresponding degrees of procedural complexity.  In public sector bodies where governance is exercised by elected members, the process of budget submission, negotiation, re-submission and approval can take six months, with the consequence that research priorities for the succeeding year need to be identified before current studies have reported or, occasionally, even begun.

Partly for this reason, and also because budget-setting is an art rather than a science, underspends may accrue over the course of a year.  Since there are restrictions on whether, and by how much, public sector organisations are permitted to carry over unspent monies into the next financial year, it is quite common for a small flurry of invitations to tender to be issued within the last financial quarter.  These tend to be small-scale projects with a short turn-around time, but may be worth putting in for – some of these projects may be transmuted into full-scale projects the following year if the findings point to the need for further investigation. (A point to note here is that central and local government departments and agencies usually operate on a financial year that runs from 1 April to 31 March, whilst private sector and third sector organisations may work with the calendar or academic year.)

Naturally, the size of research budgets out of which projects can be paid for varies enormously depending on the size, type and remit of the organisation.  So, for example, according to White (2007) in 2005–06 the total funding committed to all analytical work externally commissioned by the Department for Education and Skills (England) (as it was then) was around £139m.  As she points out, this figure included funding for ongoing longitudinal and other long-term projects;  when this is excluded, the figure for newly-contracted research and evaluation work was £13.7m.  In 2006–07, however, the equivalent figure fell to £4.5m – still more than ten times as much as the research budget of the General Teaching Council for England in the same period.  As far as the charitable foundations are concerned, it is estimated by the Association of Charitable Foundations that in 2004–05 a grand total of £273m was awarded for education (including but not only or mainly for educational research): http://www.acf.org.uk/uploadedFiles/Publications_and_resources/Publications/0416B_TrustAndFoundationBriefingPaper.pdf .

However, the main financial question researchers in pursuit of a contracted project will want to ask – but to which they may not always receive a helpful answer – is ‘what is the budget for the particular project for which I am bidding?’  Government departments and other agencies are bound by procurement rules (see immediately below) which often prevent disclosure of budget information.  Increasingly, however, in recognition of the wasted effort on both sides that results from submitting and assessing bids that are completely out of scope financially, the practice is to give a range or an upper limit as an indication of available spend.

Procurement is the term used for the buying of goods and services by or on behalf of government and its agencies;  because this concerns an officially-sanctioned use of public money, strict rules and procedures to assure legality, propriety, fairness and transparency are in force.  Read the guidance at:

• http://www.dcsf.gov.uk/procurement/
• http://www.gsr.gov.uk/professional_guidance/procurement.asp
• http://www.nationalschool.gov.uk/policyhub/magenta_book/

Because procurement rules are comprehensive, detailed and seem to guarantee legality, etc., government regulations are often adopted or adapted by non-government bodies. However, procurement as a process has its origins in the purchase of military and technical equipment; the extension of such regulations to the purchase of services – especially what might be called the ‘intellectual services’ of research – has its difficulties:  rules and procedures may be interpreted and/or applied inflexibly or without regard for the special nature of research – for example, that aspects of design and methodology may be partly contingent on factors outside the control of the contractor, or that the outcomes of the research cannot be known in advance.  If there is not proper scope within the contracting and management process for open conversation and negotiation to solve problems that arise along the way, procurement ceases to be an assurance of probity and becomes an obstacle to quality. The Social Research Association .

The ‘procurement’ issue can be particularly problematic in organisations where the purchasing of all goods and services is done by a central finance team without specific knowledge of how research is designed and produced;  whereas the DCSF is large enough to have its own research-buying team:
 

‘The [DCSF] research buying unit is responsible for the Department’s research budget and for the planning, approval and commissioning of all projects funded from it. The printing and publishing of research reports and briefs [sic]. They hold and maintain a database of research contractors with information on services and specialisms and staff details. Also responsible for the database of external research, drawing together details of research funded by organisations outside the Department, linking to other similar databases.’ (DCSF research website)

Adherence to organisational procurement regulations is at the end of the pipe, as it were, of the research commissioning process. The development of a research programme or even of a single project is normally the outcome of a series of internal discussions and consultations, any of which may be more or less formal, involving minuted or unminuted meetings with policy and communications colleagues, elected members, trustees. 

In addition to such internal processes, organisations may expend a lot of effort on gauging public and professional opinion on specific issues in education, through written/on-line consultation exercises and through open or, more usually, invited meetings with key stakeholders in different localities around the country.  From these, problems and ‘hot topics’ emerge which may then be used to inform the general direction of research and evaluation, and sometimes even lead to specific projects.  Research is also, of course, used to elicit public and professional opinion more systematically, through ad hoc, thematic or omnibus surveys – for example, the members’ surveys regularly conducted by the teachers’/headteacher’s professional associations, or the teacher omnibus survey . Any of these sources of opinion may stimulate new research and evaluation studies in particular areas.

In addition, the DCSF hosts, or used to host, informal annual meetings in which a small group of prominent academics and opinion-formers in education were invited to a closed session with senior civil servants to give their frank views on the current critical issues in education as they saw them, and the ways in which scholarship and research might be brought to bear on better understanding and resolving them.  There was no guarantee that all these issues would find their way into the Department’s commissioned research programme, but it was a useful way of tapping into expert opinion without the need to reach consensus or conclusion.

Once the proposed programme of research has been broadly agreed, there may still be internal procedures to negotiate, as in the case of the DCSF:

‘The Research Approvals Committee (RAC) works alongside the PEG [Policy Evaluation Group]. Its function is to scrutinise all new research and evaluation proposals we are to fund which will be carried out under a contract with an external organisation. The RAC is chaired by our Chief Economist and comprises senior analysts and policy-makers from across the Department. The aims of the RAC are to:

• quality-assure the new proposals and help avoid duplication of research and evaluation effort across the Department;
• help make links between otherwise disparate research and evaluation projects;
• make sure that our research and evaluation is undertaken in the most cost-effective way;
• make sure that ministers are aware of the totality of research and evaluation commissioned by  the Department.

 

‘The Star Chamber is one of the main vehicles in our drive to reduce bureaucracy on local authority children’s services (including education) and schools. It was formed in 1999 to review our existing and proposed data collection exercises and was re-launched in October 2006 with a wider remit and enhanced powers. The Star Chamber makes sure that new data collection exercises do not create unnecessary burdens and do not duplicate in content existing collections.  It also seeks to keep data requests to the absolute minimum needed to develop policy and will amalgamate and streamline exercises wherever possible.’ (excerpted from Chapter 6 of the 2008 DCSF Analysis and Evidence Strategy. 

In combination, these processes and procedures may result in a considerable time-lapse between the initial conceptualisation of an organisation’s research agenda and the issuing of invitations to tender.

TASK
Download and read this presentation:
http://www.laria.gov.uk/content/articles/commissioning_research_24092008/pdfs/percy_smith.pdf

With a fellow student or close colleague, discuss its implications in the light of your own experience of bidding for or contracting research – or, if your own experience is limited, discuss the presentation and together prepare a short aide-memoire.of the key issues surrounding procurement for your future use.

OR

Convene a meeting with a small group of other students or close colleagues, and draw up a list of your agreed top three ‘hot topics’ in education which ought to attract further systematic investigation through targeted research funding.  You must, of course, construct a reasoned case for your choice.  You can use a suitable activity to do this – a balloon debate, a diamond nine, etc.

3. Framework Agreements and partnership and consortium arrangements

 
This section deals briefly with a form of commissioning which government departments and local authorities are using in order to save time and money, to reduce duplication of sources and studies, to strengthen collaborative relationships with research providers, and to improve the coherence and connectivity of evidence.  Essentially, a broad programme of research (or other work) is commissioned from a panel of several contractors, and sometimes jointly by more than one department/client, on the basis of a framework of required services and activities over an extended period – as distinct, say, from a single department contracting a sole provider for a specific project. 

There are normally two stages involved in being selected to join a framework panel.  The first is a response to a published call for expressions of interest from provider organisations: this must supply outline information that demonstrates competence in at least one relevant area of research.  Organisations successfully passing this screening assessment are then invited to submit detailed proposals, demonstrating experience, expertise and capacity, in substantive areas of interest to them and of relevance to the tender.

As you might suppose, although each tender will be different the requirements are necessarily complex and the procedures time-consuming; this resource can do no more than point you in the direction of some background information.  The key thing to bear in mind, from potential contractors’ point of view, is that in working on the programme researchers will need to pool their expertise, probably across as well as within institutions.  They will need to consider how to establish robust and manageable consortium structures and strong partnership working arrangements: ideally, some such arrangements should have been in place before responding to the invitation to tender, as inter-institution relationships will certainly be put to the test over the contract period.  It is likely that this mode of letting research contracts will become increasingly common so it is worth acquainting yourself with its general features.  Some useful information and/or examples are as follows:

Presentation from the Department of Work and Pensions (DWP): this presentation offers a rationale for Framework Agreements.

Presentation from the Office of the Deputy Prime Minister (ODPM): go to www.ccsr.ac.uk/methods/archive/socialresearchfunding/ and click on item ‘Paul McCafferty, Head of the Local and Regional Government Research Unit, ODPM’  This presentation deals with Framework Agreements in the course of giving much other useful information about government-commissioned research.

Example from the Learning and Skills Council (LSC):

Example from the British Educational Communications and Technology Agency (BECTA).

TASK

Working with a fellow student or close colleague, look at these resources and identify the main potential advantages of Framework Agreements from both the client’s and contractors’ point of view; then consider any aspects of internal cultures and practices which could act as barriers to the successful management of such agreements.

 

4. The research specification and ITT

Leaving aside the construction of Framework Agreements and accompanying invitations to tender (because it is a complicated and specialist area of research management), this section summarises the main elements in drawing up the specification and invitation to tender for a research project.

To begin with, research managers should be familiar with – not necessarily expert in – the affordances of a wide range of design approaches, methodological rationales and technical tools, including:

• quantitative methods and sampling
• qualitative methods
• experimental designs and trials
• combined methods – integrated combinations of different methods
• consultations
• research reviews
• evaluations
• analysis of administrative data and other secondary sources
• statistical models and techniques
• increasingly, realist synthesis approaches (as a conceptual framework rather than method).

See Cohen, Manion and Morris (2001) for a comprehensive discussion of different research styles, strategies, methods and techniques.

The primary purpose of a research specification is to render the policy question(s) researchable, and to suggest which approaches might be particularly appropriate to advancing the investigation without excluding alternatives.

A specification document works best if it:

• is straightforward to understand, in language and style, and in the way it is structured;
• sets out the aim(s) and purpose(s) of the planned study, including the policy context;
• outlines preferred options for the research design and invites researchers to suggest alternative methodologies, rather than specifying the exact methods to be followed;
• is frank about any problems that could affect the study (e.g. accessibility/accuracy of internal databases/datasets on which the study might rely; the degree to which hard-to-reach groups must be included in the study; sensitive policy issues);
• is crystal clear about timelines and deliverables, including whether there might be any leeway in the timescale, whether there is a possibility of any follow-on studies on successful completion of the current study, etc.;
• includes any dissemination activities or other requirements that are expected to ensue from the study, and makes appropriate financial and time provision available;
• gives a range or an upper limit as an indication of available budget;
• provides explicit criteria on which proposals will be assessed;
• identifies a key contact person who can be approached to answer any queries.

The invitation to tender should be specific about:

• the timeline and mode (electronic and/or number of hard copies) for submitting proposals;
• any restrictions on length of proposals;
• the person to whom the proposal should be sent;
• whether short-listed bidders will be invited to interview, together with specific dates if possible;
• when and how unsuccessful bidders will be notified;
• whether and by whom feedback will be offered to unsuccessful bidders (it is good practice to do so).

Sometimes this information is included in the specification document.

It is worth noting here that the tendering (and indeed the publication) schedule for research will be affected if a political election is called, because research contracting (and publication) must be put on hold for a period of six weeks before an election – a period rather oddly called purdah by government. If an election is expected, this should have been taken account of in the tendering timeline;  but of course an election may be unexpectedly called and this could interrupt the tendering process at any stage.

5. Points to bear in mind when responding to an ITT

Although for the most part this resource addresses the issues and processes of directly commissioned research, it may be worth including here some illustrative information about research which is grant-funded by third-sector bodies, given that many researchers will from time to time seek such grants.  The ESRC has a useful publication on third-sector research 

You can also find valuable intelligence on the TLRP and CCSR websites:

• http://www.tlrp.org/rcbn/capacity/Activities/Themes/Building/WritingProposals.pdf for help with writing an application for grant funding;
• www.ccsr.ac.uk/methods/archive/socialresearchfunding/ and click on item ‘Sharon Witherspoon, Deputy Director, Nuffield Foundation’, for information about research grants awarded by the Nuffield Foundation.

(Research Council funding itself is a different proposition, and is not dealt with in this resource; visit the ESRC website for a downloadable PDF

One of the major grant-funders, the Joseph Rowntree Foundation, has taken the trouble to offer guidance on the common weaknesses in research proposals submitted to the Foundation:

‘Common weaknesses of research and development proposals
Some proposals are turned down because the Foundation cannot give the topic sufficient priority even though the proposal is well thought out and presented. But proposals are more often rejected because insufficient information has been provided about key aspects, or because there is a lack of clarity about what is planned, and insufficient detail about methods to be used. For example:

• The proposal describes the background to the project at some length but gives very few details about aims and methods.

• The aims of the project are very vague or are couched in terms of the process, e.g. the aim of the project is to carry out a ‘descriptive study of X’ or a ’survey of Y’.

• The design of the study lacks clarity or robustness so that there is a mismatch between the issues being addressed and the approach adopted, or what is planned is over ambitious.

• Information about data collection is insufficiently detailed. If interviews are going to be carried out, it is important to give information about the number of interviews; how the sample would be selected; and the form in which the data would be collected. (Any proposal which talks in terms of `some people will be interviewed` is unlikely to be funded).

• Details of the way the data will be recorded and analysed are lacking. This is particularly the case where the use of focus groups is proposed.

• Information about the proposer’s own knowledge and skills is lacking. In addition to the standard information provided on a CV, it is helpful to have a short summary – two or three sentences – of the experience that members of the team carrying out the work have had using the methods being proposed.

• Timescales and staff resources are unrealistic or inappropriate. This includes too much work being planned for the time available, or too much of the fieldwork and analysis being left to inexperienced research staff.

• It is unclear how the proposed outputs could be derived from the material collected.’

from ‘How to Write a Good Proposal’

In fact, these points could equally well apply to proposals which are responding to ITTs:  especially the point about explaining how – meaning with what kinds of defensible conceptual approach and framework, rather than with what type of software – data will be analysed.

In addition, writers of responsive proposals should take care to:

• address each of the requirements as set out in the research specification – this sounds self-evident, but a surprising number of proposals fall at this first hurdle;
• be familiar with/confident about the assessment criteria (if given), and ensure that these have explicitly been met;
• avoid submitting a proposal which is a (thinly-disguised) version of a study which the researcher(s) would like to conduct, as opposed to the study which is being requested;
• weigh up the pros and cons of using off-the-shelf proposals – although they save researchers’ time, most commissioners of research recognise the ploy and may not welcome it;
• if different people have contributed different sections, ensure the joins do not show;
• avoid playing back the identical words of the specification – exhibit some independence of professional thinking; but do not try to re-write the specification;
• if there are major problems with the intelligibility or feasibility of the specification as written, however (for example, a timescale that would necessitate visiting schools during school holidays), contact the commissioning organisation to discuss them before you spend time and effort on writing anything;  if necessary, withdraw from the bidding – other bidders are likely to respond similarly, with the result that the commissioning team will have to go back to the drawing board;
• even if this is not required by the specification, identify and assess the likelihood and severity of the possible risks that the proposed project might encounter, from both internal and external sources (e.g. staff turnover;  low response rate to questionnaire survey); and briefly state how these risks would be reduced or handled;
• consider how to cost the project as efficiently as possible;  but avoid under-costing in order to try to present a cheaper bid: nothing is to be gained by it and it risks (i) not being convincing as a feasible proposal anyway; (ii) your needing to approach your institution for financial subsidy at a later date; (iii) over-committing the project researchers; (iv) not delivering the project to time/budget; (v) misrepresenting the true scale of research costs, to the detriment of the field as a whole;
• leave sufficient time for a senior internal colleague who is not directly involved in the proposal to check (and, if appropriate, sign off) the proposal before it is sent.

• if it is not already clear, weigh up the relative emphasis of what appears to be being asked for, e.g. review of a field, exploratory/developmental research, empirical evidence, evaluative judgement, competition for influence;

The English ministry of education offers some general guidance which, if nothing else, shows what civil servants are looking for: http://www.dcsf.gov.uk/procurement/bidding.shtml

It briefly covers the following aspects of the process:

Preparing an effective tender

Tender evaluation

Debriefing

Department contract terms and conditions

TASK

Using the website resources above, draft a list of the main differences, from a researcher’s perspective, between writing an application for research grant funding and a proposal in response to an ITT.

6. Evaluating bids/proposals, including criteria

As mentioned, research specifications and ITTs should say explicitly what the criteria will be for evaluating bids.  Whilst these will vary somewhat between projects and commissioning organisations, they will normally cover such categories as:

• demonstrable understanding of the policy context for this study (e.g. recent and related policy initiatives);
• fitness for purpose of the research design:  a clear rationale that shows how the aims will be met, and the outcomes produced, through the methods proposed;  what the key research issues and challenges are; what concepts and theories will particularly contribute to a deeper understanding of these; what account has been/will be taken of existing evidence;
• demonstrable capacity, particularly in terms of appropriate staffing, to undertake this project;
• previous track record of delivery to time and budget;
• project management arrangements;
• risk identification and proposed management;
• value for money (not the same as ‘cheapest’).

 
Many commissioning organisations will also ask for detailed breakdowns of costings under different sub-headings, exact number of hours/days to be worked by different grades of project staff, CVs of all project staff, references from previous clients, etc.

If no such criteria are given in the specification/ITT or are too vague to be helpful, bidders should feel entitled to contact the commissioning organisation and ask.

If interviews are held for short-listed research teams (which usually conducted by a panel), performance at interview – including answers to specific questions on the proposal – will be taken into account as well as the written proposal.

Assessment of bids is also usually done by a panel, so that policy interests, stakeholder/lay concerns and internal research expertise are equally represented.  It is good practice (and in public sector bodies, mandatory) to make an accurate and impartial written record of how decisions were arrived at, not least in case there is a legal challenge from an unsuccessful bidder.  The information will be expunged at the end of the required period.

Increasingly, to meet criticisms from the academic community that the assessment process is not seen to be fair, impartial and/or fully informed by relevant expertise, commissioning organisations are deploying external experts in a version of academic peer review procedures to review proposals.  The reason why this is a ‘version’ of peer review is that at least some of the relevant experts are quite likely either to be competitor bidders or else to have been unsuccessful at the first stage of bidding and for that reason not a disinterested party.  So this is a process that needs to be treated with care and with regard to the spirit of the principle rather than unwavering adherence to the letter, especially since the peer review system was comprehensively reviewed in 2007 and found in need of strengthening (http://www.rcuk.ac.uk/cmsweb/downloads/rcuk/documents/responsereport.pdf)[1].

Alternatively, commissioning bodies may invite peer review at the draft report stage; the Welsh Assembly gives an informative account of why and how this procedure is managed: http://wales.gov.uk/about/aboutresearch/econoresearch/economicresearch/peerreviewarrangements/;jsessionid=2Wv2Jn6GR8GXpGBHJ253ydcS5KvFpL7FFpcdG8c8p5BCrPJ3dLhq!-98953499?lang=en

It is important to distinguish the task of safe-guarding the quality of research from that of supporting its salience and credibility as far as stakeholders (policy-makers and the general public) are concerned – both being necessary for a successful research outcome, defined in terms of validly influencing how the policy issue is thought of and of proposing workable solutions.  Very occasionally, for example, a bid which shows a more sophisticated understanding of the policy/practice context and the potential implications of the research may win over a proposal which is technically more exacting but which exhibits little sense of how the research articulates with the real world or why the study is being commissioned.

---

[1]     For a provocative position-piece on peer review in science, see: http://www.i-sis.org.uk/peerReviewUnderTheSpotlight.php

7. Asking for/receiving feedback on unsuccessful bids/proposals

Researchers should always feel entitled to seek feedback if their proposal was unsuccessful at initial or later stages of bidding: the dialogue benefits both researchers and research commissioners. Research commissioners are aware of the time and effort every proposal represents, which is a cost not only to the individual researchers and their institutions but also to the system as a whole. As the DCSF says (website reference above):

‘Within the limits of some commercial confidentiality, the Department will always offer to explain to unsuccessful tenderers why their bid failed. This debriefing can be by letter, phone or face–to-face… Debriefing should be viewed as a two-way process. Our comments are designed to be constructive and aim to draw your attention to certain weaknesses (and strengths), so that you can compete better for future work.’

This is another reason, of course, why commissioning organisations should keep detailed, fair and accurate notes of the assessment panel’s deliberations.

Sometimes feedback is given without needing to be asked for, as in this example from a letter to an unsuccessful bidder:

As indicated in the Invitation to Tender, the award criterion for this requirement was the most economically advantageous tender evaluated against the other award criteria given below:

Price and value for money                                        -           20%

Clear project proposals to meet the specification and demonstrate a level of resource and capacity to deliver the project                                           -           20%

Ability to demonstrate capability and experience in delivering services similar to those required                                                                  -           20%

Ability to deliver the project to agreed timescale        -           10%

 Ability to provide high quality research services in a sensitive policy context                -          20%

Compliance with conditions of contract                     -           10%

Our evaluation resulted in your bid receiving a score of 250 compared with the winning tenderer XXX, who scored 320.

You may request additional debrief information be made available to you…

Feedback is only useful if it is acted upon, and research teams would be well advised to hold an internal de-briefing session after each unsuccessful bid with the intention to learn lessons for next time.

8. Pre-contract negotiation

Research contracts are let on a case-by-case basis, and an agreed final version of the proposal document will usually form the substance of the contract. This will be accompanied by the organisation’s standard terms and conditions, which are often available to view on request beforehand. As noted above, the degree of detail and the specific terms of any contract vary from agency to agency, and most research providers have legal/financial specialists whose job is to check the small print to ensure that the terms are acceptable from the institution’s point of view.

Occasionally, issues may come to light in the period between a research team submitting a successful proposal – and thereby becoming the ‘preferred bidder’ – and the contract being drawn up and signed. These might include:

• changes in the policy context that affect the conduct of the research and/or its reception by potential research participants;
• events or circumstances in the broader population to be researched (such as a teachers’ strike in schools);
• in the light of the above, mutual agreement that a modified research design would be more feasible and/or cost-effective.

It is far better to have these discussions up-front so that the contract can be agreed as a realistic working document, and to pre-empt the need for arguments about variations or extensions to contract later on.  Inevitably, this will sometimes involve re-costings but if the initial costs have been calculated accurately and transparently it should not be difficult or time-consuming to redo them.  If the discussions result in a slightly delayed start, be sure to check whether the deadline and intervening milestones can be shifted accordingly.

Sometimes for no sinister reason there is a delay of more than just a day or two between the formal decision to let the contract and the actual contract being issued by the organisation’s legal department:  researchers may use their discretion about whether to begin work in the absence of the contract document, in the interests of maintaining good relationships with the client; or their institution may have a rule about this. 

Very occasionally, a project may fail to be contracted even at this late stage, because agreement cannot be reached between the research team and commissioning organisation and/or because the policy environment has radically changed – for example, as a result of a political election.

Ways in which research commissioning might be improved

Like all organisational procedures, there are ways in which the commissioning of research could be improved.  For a useful round-up of the issues, go to www.ccsr.ac.uk/methods/archive/socialresearchfunding/ and click on items ‘Professor Mike Hough, Institute of Criminal Policy Research, King’s College London’ and Dr Helen Barnes, Policy Studies Institute’.  These are presentations on government-commissioned and on charitable foundation research respectively, and give an overview of how commissioning/funding procedures and processes might be made more productive and constructive.

The Social Research Association – whose members include a considerable number of research managers in central and local government – has already done a great deal of influential work in this area, consulting researchers, holding briefing seminars with civil servants and publishing guidelines on its website: see Commissioning Social Research: A good practice guide – second edition and Curriculum Guidelines for Training Programmes on Research Commissioning: a report funded by the ESRC and prepared for the Social Research Association by Janie Percy-Smith and Alison Darlow (2006).

‘Effective commissioning occurs when the following principles are met and this is the case regardless of the size of the organisation doing the commissioning or the scale of the research being commissioned.

The need for, and purpose of, the research is clear: This underlines the importance of a general understanding of social research and its limitations and where research is useful in relation to specific issues. It also emphasises the importance of discussion between research managers/commissioners and research customers about the rationale for, and purpose of, the research.

Maximum use is made of researchers’ skills, experience and expertise: This will involve striking an appropriate balance in preparing the brief between providing sufficient detail about what is required while providing opportunities for researchers to suggest approaches and methods. It also involves the use of dialogue as a key tool in the commissioning process.

The commissioning process is open and transparent and the principles of fair play are adhered to. This involves a balance between making use of the experience of existing researchers while, at the same time, keeping the door open to new research providers.

The form of competition is ‘fit for purpose’. This involves an understanding of the different forms of competition, their limitations and their applicability to different types and sizes of research project. It also entails an understanding of the researcher’s viewpoint in terms of not wasting their time e.g. by inviting too many researchers to submit full tenders.

The brief is well written and provides clear information to tenderers. The information provided should include the purpose and aims and objectives of the research; constraints or difficulties that are likely to be encountered; timescales; and the commissioning process. It is also helpful to provide an indication of budget or the scale of the project so that time is not wasted on developing proposals that are beyond the scope of the budget.

There is effective communication between all parties to the process. This includes dialogue between procurement colleagues and research managers about the nature of the commissioning process; discussion between research managers and research customers about the rationale for, and purpose of the research; discussion with potential tenderers to enable them to prepare appropriate proposals; discussion with successful tenderers to agree the final shape of the research; and ongoing discussion with the researchers throughout the course of the project.

Adequate time is allowed both for the commissioning process and for the research itself. Good researchers are generally busy. In order to secure the services of the best researchers sufficient time needs to be allowed for them to develop a proposal and then, if successful, to undertake the research. Where research is needed urgently a more streamlined commissioning process should be used.’

Extract from SRA publication http://www.the-sra.org.uk/documents/pdfs/commissioning_final_report_0906.pdf

TASK

Read and consider carefully these presentations and guidelines.  If you are a research manager, draft a ten-point action plan for improving the commissioning process within your organisation.  If you are a researcher, summarise the key issues and recommendations as a workshop presentation to your colleagues; and add some ideas of your own about how researchers themselves might contribute to the improvement of the commissioning process.

To comment on any aspect of this resource, please e-mail lesley@lesleysaunders.org.uk

Further reading and bibliography

Selected guidance for researchers interested in submitting proposals for commissioned/funded research:

List of research funders (basic information on the research councils but not comprehensive on other sources):

http://www.hero.ac.uk/uk/research/research_funding_finder.cfm

Training on writing funding applications (generic, not specifically for research):

http://www.dsc.org.uk/Training/Fundraisingtechniques/@29642

Guidance on seeking funding from a charitable organisation: http://www.acf.org.uk/uploadedFiles/Seeking_funding/Applying%20to%20a%20charitable%20trust%20or%20foundation%20revised%202006.pdf

Guidance on costs allowed under full economic costing:
http://www.nuffieldfoundation.org/fileLibrary/pdf/Final_FEC_Web_text_July_07.pdf

BERESFORD, B. (undated). ‘Using evidence from research to support funding applications’. Powerpoint presentation, downloaded 9 March 2009 from http://www.ccnuk.org.uk/metadot/index.pl?id=2601&isa=Newsitem&field_name=t3&op=download_file

BRIDGES, D., SMEYERS, P. and SMITH, R. (Eds) (2008a) ‘Educational research and the practical judgement of policy makers’, Journal of Philosophy of Education, 42, Suppl. 1., 5– 14.

BRIDGES, D., CONROY, J., DAVIS, R., ELLIOTT, J., ENSLIN, P., GRIFFITHS, M., MCLEOD, G., FOREMAN PECK, L., LUKES, D., MURRAY, .J, OANCEA, A., PRING, R., SAUNDERS, L., SMEYERS, P., SMITH, R. and WATTS, M. (2008b). ‘Educational research and policy: epistemological perspectives.’ London: TLRP. Online at http://www.tlrp.org/capacity/rm/wt/pesgb1.html

BRITISH ACADEMY. (2008). Punching Our Weight: The Humanities and Social Sciences in Public Policy Making. Published online 17 September 2008:http://www.britac.ac.uk/reports/wilson/index.cfm 

COHEN, L., MANION, L. and MORRISON, K. (2001). Research Methods in Education. London:  Routledge Falmer.

LEVITT, R. and SOLESBURY, W.  (2005). ‘Evidence-informed policy: what difference do outsiders in Whitehall make?’ ESRC UK Centre for Evidence Based Policy and Practice:  Working Paper 23.

ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT, CENTRE FOR EDUCATIONAL RESEARCH AND INNOVATION. (2007). Evidence in Education:  Linking Research and Policy. Downloadable from:http://www.oecd.org/document/56/0,3343,en_2649_35845581_38796344_1_1_1_1,00.html

RUDD, P., RICKINSON, M. and BENEFIELD, P. (2004). Mapping Work on the Future of Teaching and Learning. Slough: NFER

SAUNDERS, L. (2004).  Grounding the Democratic Imagination:  Developing the Relationship between Research and Policy in Education.  Professorial Lecture.  London: Institute of Education.

SAUNDERS, L. (2007). (Ed) Educational Research and Policy-making:  Exploring the Border Country Between Research and PolicyLondon:  Routledge.

SAUNDERS, L. (2008) ‘Educational research commissioned by/for policy audiences.’ London: TLRP. Online at http://www.tlrp.org/capacity/rm/wt/saunders1.html

SOCIAL RESEARCH ASSOCIATION. (2005). Commissioning Social Research: A good practice guide – second edition. Downloadable at:
http://www.the-sra.org.uk/documents/pdfs/commissioning.pdf

STILGOE, J., IRWIN, A. and JONES, K. (2006). The Received Wisdom: Opening Up Expert Advice.  London:  Demos.  Downloadable from: http://www.demos.co.uk/files/receivedwisdom.pdf
 
WHITE, V. (2007). ‘Schools research in the English Ministry of Education: an inside view.’  In: SAUNDERS, L. (Ed) Educational Research and Policy-making:  Exploring the Border Country Between Research and Policy London:  Routledge.

Return to top of page