Final Evaluation of Supporting
Democratic Processes in Pakistan

Terms of reference

Final Evaluation of Supporting Democratic Processes in Pakistan

Overview

An International media development organization working to empower local civil society to give people the news and information they need, the ability to connect and the means to make their voices heard. 

Project background

This project is focused on a nascent movement in-country to increase the amount of government data available to the public. It introduces new approaches and techniques for journalists to use data in responsible news reports and features about government transparency and accountability, and couples this with complementary efforts to raise the profile and use of Pakistan’s existing Freedom of Information laws. The goal of the project is to “improve transparency and decrease official corruption by increasing the capacity and impact of journalists and civil society in holding local and provincial governments accountable to their citizens”.

The project aimed to achieve the following two objectives:

Objective 1: Increase local journalists’ skills to effectively report on provincial and local governments through accessing documentation evidence, applying Freedom of Information processes, and conducting advanced budget and program data analysis.

Objective 2: Strengthen the demand for greater official transparency by producing and promoting media coverage that highlights crucial issues through advanced evidence-based reporting methods.

Evaluation Purpose and Objectives

The purpose of this final evaluation is to assess the performance and success of the project. It also looks at signs of potential impact of project activities on beneficiaries and sustainability of results.

This evaluation has the following principal tasks:

• Assess the project impact on practical skills development of local investigative journalism responsible reporting capacity, followed by consolidated production and engagement that strengthens the application and awareness of investigative journalism responsible reporting tools over time.
• Assess performance of the project in terms of relevance, effectiveness, efficiency, impact and sustainability.
• Analyze underlying factors beyond organization control that affect the achievement of the project results.
• Provide recommendations for potential future interventions.

Evaluation Scope

The evaluation will use the Development Assistance Committee of the Economic Cooperation and Development (OECD/DAC) criteria, which is one of the most widely prominent and widely adopted criteria review of aid effectiveness. The criteria used to evaluate development interventions (relevance, effectiveness, efficiency, impact, and sustainability) are undoubtedly the most known and adopted features that emerged from the OEDC/DAC evaluation guidelines.

The following general key questions should be addressed through the evaluation:

1. Did the theory of change hold true?
2. Has the project responded to the needs of the target groups? (relevance)
3. Have the right things been done to achieve the project aim? Have the planned project objectives and outcomes achieved (or likely to be achieved), if not, what is needed to close the gap and improve potential for impact? (effectiveness)
4. Are the resources and inputs converted to outputs in a timely and cost-effective manner? Is the project achieving the planned outputs? Could the outputs be achieved with fewer resources? (efficient)
5. What results have been achieved in the implementation of the project? What are the contributing factors to the achievement of the project? What are the main challenges/barriers to achieving the implementation of project? (achievement)
6. What are the tangible outcomes (positive and negative) as a result of the project? Has there been positive or negative impact on beneficiaries as a whole? How has this impact been ascertained? What specific real difference has the activity made to the beneficiaries? Have certain groups benefited more or been negatively impacted upon more? (impact)
7. Are the results sustainable? If so how? Particular focus will be given to i) the delivery of a multifaceted journalist capacity building effort consisting of trainings, boot camps, and mentoring together with production-focused grants, which ensures both participant skills acquisition and their practical application; and ii) strengthening key relationships between media and government counterparts—as well as media and civil society—which are key to expanding the use of Freedom of Information Laws and scaling up advocacy for increased government transparency. (sustainability)
8. How could things be done better in the future? (future recommendations)

Timeframe and deliverables

 Anticipate around five-six weeks will be needed to conduct this evaluation, in a flexible working manner as follows:

Activity	Timeframe	Comments
Advertisement and scrutiny of applications	Deadline for  applications: May 23- 30, 2018	The consultant is expected to have initial meetings with staff in order  to inform inception report
Inception report	By June 6, 2018 the latest   Estimated time: 2 working days	Relevant documentation will be made available and the inception report will include: Literature review Draft methodology Develop data collection tools
Field phase: literature review, new data collection where relevant and analysis	By June 13, 2018 Estimated time: 5 – 7 days	A field report and sets of raw data are expected before final draft is submitted to Organization.
Report writing and presentation of first draft to Organization	By June 30, 2018, the latest Estimated time: 2-3 days	Organization to have an opportunity to comment and feedback on draft report if relevant.
Final report submission	By July 15, 2018, the latest	The consultant is expected to revise the draft report based on feedback from Organization. Any remaining sets of data to be submitted to Organization.

The final report should be presented in English and be no longer than 20 pages (excluding any relevant annexes). The report must include:

a)      An executive summary highlighting the main findings, conclusions and recommendations. , Recommendations

b)      A methodology section making explicit any limits to the evaluation

c)       An analysis of the project performance of the project in terms of relevance, effectiveness, efficiency, impact and sustainability

d)      An analysis of the technical aspects and achievement of the project

e)      Analysis of the underlying factors that affect the achievement of the project results

f)       Recommendations, being specific and action-oriented, and tailored to all relevant stakeholders (i.e. Organization and the donor)

Essential qualifications

• Master’s degree in social sciences, international development, evaluation or related field;
• At least 3 years’ experience conducting evaluations in difficult contexts, particularly for US donors
• Experience with qualitative and quantitative M&E data collection and analysis methods;
• Experience conducting evaluations for media development, or more than 5 years’ experience as evaluator
• Fluent English written and understanding of Pashtu or any other local Pakistani language
• Excellent inter-cultural communication skills/cultural sensitivity and the ability to forge strong cross-cultural relationships and build trust demonstrated through previous intercultural experience;
• Strong facilitation, presentation, writing and communication skills