Project Evaluation Consultant

PURPOSE and MAIN OBJECTIVES

Purpose

The purpose of this study is to know the impact of ARC’s interventions on the lives of refugees and host communities in terms of health, hygiene, GBV, and livelihood at one and gather successes and lesson learnt for ARC Pakistan on other.

The purpose of this evaluation will be to gain an independent assessment of the extent to which the programme has achieved its objective and outcomes. It will also highlight issues and challenges affecting effective and efficient implementation of outputs and their contribution to project outcomes and impact and recommend whether results obtained thus far warrant an extension of the project. This evaluation should help capture the lessons from the implementation of the project by ARC Pakistan in collaboration with its implementing partners:

 

1. Enhance accountability to beneficiaries

2. Guide future decisions on humanitarian responses in Pakistan after

3. More generally, improve ARC’s response to similar projects in the future

    

   It is worth mentioning that this project is a continuation of a program started in 2002. More than 600,000 Afghan refugees and Pakistani people who have been uprooted by their homes in the wake of conflict and disaster, and other affected vulnerable groups.

   
   Since the 1970’s, three million Afghan refugees have fled to Pakistan to escape drought, an oppressive Taliban government, and constant conflict and threats. ARC has been providing health care services to these refugees since establishing operations in Pakistan in 2002. While many people have returned to Afghanistan, more than one million remain in refugee camps in Pakistan.

    

   The American Refugee Committee currently provides quality services to meet the needs of more than 100,000 people. Following the Minimal Initial Service Package (MISP) standards, ARC is providing  Primary and Comprehensive Reproductive Healthcare, livelihood opportunities and Gender-Based Violence (GBV) services, complete GBV case management and referral services alongside vocational and livelihoods skill development trainings and awareness raising session on health and hygiene to Afghan refugees who have fled to Pakistan to escape drought, civil war, and constant conflict and threats by the oppressive Taliban government. The program was started in 2002 with financial support BPRM and it is continued without any break.

    

   Name of the Project: Community-based comprehensive health services and livelihoods opportunities for Afghan refugees and host communities in Balochistan, Pakistan.

    

   Project Goal: To improve the overall status of Afghan refugees and respective host communities through a comprehensive package of health, GBV and livelihoods.

    

   Project Objectives:

   Objective 1: Improve the health status of Afghan refugees and host communities in and around Surkhab, Saranan, and Mohammad Khail, through increased access to quality health services, including PHC, MCH, EmOC and nutrition, and support to gradual phase-out of direct INGO service provision.

    

   Objective 2: Promote well-being of Afghan communities through livelihoods programming including skills building, vocational trainings and enhancing income-generating activities

    

   Objective 3: Empower and promote protection of women and girls through integrated protection programming including the development of community support groups for men women and children, as well as psychosocial support and medical response for survivors of GBV and other protection violations

    

   Main Objective of the Study

1. Overall objective:

   To assess the overall relevance, effectiveness, impact, coherence, economy, efficiency and sustainability of ARC’s Project “Community-based comprehensive health services and livelihoods opportunities for Afghan refugees and host communities in Balochistan, Pakistan (2002-2014)”.

    

2. Specific objectives:

   To assess the extent to which the projects implemented by ARC, have achieved the programme’s initial objectives and outcomes and consequently met the needs of refugee and host communities in terms of health, livelihoods and GBV services;

    

   To identify key success stories/lessons and provide recommendations to strengthen ARC’s response to similar projects/emergencies in the future in Pakistan and elsewhere;

    

   The evaluation should assess the following elements:

1. Relevance/appropriateness
2. Connectedness
3. Coherence
4. Coverage
5. Efficiency
6. Effectiveness
7. Sustainability
8. Impact

 

With reference to ARC’s services delivery, the following specific questions should also be answered

• Was ARC’s involvement relevant, appropriate, coherent, and timely?

• What was the specific contribution of ARC and its IPs to the programme?

• What should have been managed differently?

• How has working in partnership benefitted the overall programme?

• How ARC’s partnering approach benefitted partners, and supported the work of the partners in the programme?

   

  ---

  SPECIFIC ACTIVITIES

   

  The consultant/team will adopt a consultative and participative approach. This will include field visits to project sites and meetings with communities / project direct beneficiaries in two districts to collect first hand information. The consultant/team will also meet with the Project team, relevant government agencies / partners at federal, provincial and district level, if need be, the consultant/team may also meet other agencies / projects engaged in similar interventions.

                                                               

DELIVERABLES and TIMELINE 

Deliverables of the external evaluation include a draft report and, after discussion and review by the team, a final report comprising:

Format of Final Project Evaluation Report (to be provided in English)

1. Title Page        
2. Table of Contents
3. Executive Summary
4. List of Acronyms and Abbreviations
5. Methodology: Description of the way the evaluation was planned and implemented
6. Description of the project including any major changes occurring since the proposal’s description of Project’s design, objectives, targets and working methods
7. Main findings with the support of SWOT analysis and summarizing strengths and gaps related to the following:

1. Implementation Strategies: Key Approaches

2. Project Efficiency

3. Project Effectiveness

4. Relevance

5. Sustainability and Impact

8. Lessons learned
9. Recommendations for project improvement
10. Recommended Change plan (stating changes, resources, timelines and responsibilities) to implement recommendations
11. References
12. Annexes (explained in the table)

 

Project Rating Matrix

A Project Rating Matrix will be annexed to the report and will rate the project according to the following criteria:

1. Attainment of objectives and planned results

2. Achievement of outputs and activities

3. Cost-effectiveness

4. Financial Planning

5. Impact

6. Sustainability

7. Stakeholder participation / public awareness

8. Implementation approach

9. Replicability

10. Monitoring and Evaluation

 

The ratings will be presented in the form of a table. Each of the ten categories shall be rated separately with brief justifications based on the findings of the main analysis. An overall rating for the project should also be given. The following rating system is to be applied:

HS= Highly Satisfactory

S = Satisfactory

MS = Moderately Satisfactory

MU = Moderately Unsatisfactory

U= Unsatisfactory

HU = Highly Unsatisfactory

 

The report should be in English and no less than 15,000 and no more of 20,000 words (excluding annexes). The dates each report should be submitted by will be agreed between ARC and the evaluator. Appendices should include the evaluation terms of reference, maps, beneficiary feedback and bibliography (all materials collected during the evaluation process should be lodged with ARC prior to termination of the contract). 

 

Draft and Final Reports: The report should be logically structured, contain evidence-based findings, conclusions, lessons and recommendations, and should be free of information that is not relevant to the overall analysis. The report should respond in detail to the key focus areas described above. It should include a set of specific recommendations formulated for the project, and identify the necessary actions required to be undertaken, who should undertake those and possible time-lines (if any). ARC team will provide comments on the Draft Report, and the consultant/team will finalize the report in view of these comments.

 

The report and all background documentation will be the property of ARC. It will be disseminated and publicized, particularly through ARC’s website.

 

Intended users of the evaluation

1. ARC
2. BPRM and other Donors
3. Partners
4. Humanitarian community in Pakistan

 

Methodology

The evaluator will include a description of the preferred methodology in the proposal. A more detailed methodology and a work plan will be later agreed by the evaluator and ARC.

• The methodology is initially expected to include:

• Use of international guidelines (OECD-DAC criteria for evaluating development assistance, ALNAP guide for evaluating humanitarian action, Sphere, the Red Cross Code of Conduct and HAP)

• Use of participatory approaches and feedbacks from participants, especially the beneficiaries

• The quality of the evaluation process will be assessed against the BOND Evidence of Effectiveness Principles.

   

  Role and responsibilities

1. ARC (Quetta and Islamabad):

1. Liaise with consultant, partners organizations and authorities;

2. Review and comment on proposed methodology (including evaluation tools);

3. Initial meeting or teleconference to review background information,  to finalize methodology and agree on work-plan;

4. Coordinate and organize logistics, as needed;

5. To provide key programme documents;

6. Review draft evaluation report and offer any additional clarification as needed;

7. Approve final report;

8. Pay consultant (Payment will be made in three installments, i.e. first payment of 30% at the beginning/sighing of contract, second payment of 40% at submission of first draft of the report and third and final payment will be released on the approval of the draft report by ARC. All payments will be made in the form of cross cheque

1. Consultant:

1. Develop evaluation methodology (including tools) and share accordingly with ARC for approval

2. Drafting of detailed work-plan and share accordingly with ARC for approval

3. Carrying out the evaluation as per the agreed methodology;

4. Develop case studies for each component, i.e. Health, nutrition, GBV, livelihood

5. Produce draft evaluation report (in English) and submit to ARC for review;

6. Incorporate comments and submit final report.

    

   Key working relationships

   The consultant will be working independently but will be supported by ARC’s staff in Balochistan and Islamabad in Pakistan

• ARC Islamabad: Country Representative/Snr. Program Coordinator, who will act as the focal-point for the evaluation process;
• ARC Quetta Pakistan: Program/Health Manager who will be providing support during field visits and other activities
• Partner agencies in Pakistan: Tameer e Khalq Foundation , Pakistan
• External: Government of Pakistan, other NGOs and INGOs, multilateral organisations (UN, etc), beneficiaries, and relevant coordination networks, etc.

 

Timeline

• The consultant should complete his work within 30 days.

• The timeframe will include time for preparation, field work, writing, feedbacks and finalization.

   

  ---

  QUALIFICATIONS

  Qualifications required from the consultant

  The evaluation will be conducted by one professional (or a team) with the following experience and skills:

1. Profile: Pakistani or international staff;

2. Fluency in English, Urdu and Pushto with excellent writing skills;

3. Relevant work experience in humanitarian relief;

4. Relevant evaluation experience of humanitarian aid programmes, including the use of participatory evaluation methods with beneficiaries;

5. Ability to work respectfully with national and international NGO partners.

6. Sound understanding of gender issues

7. No conflict of interest with the ongoing activities of ARC

    

   Applications are welcome from anyone who feels they can contribute to the team because they possess the listed expertise. Obviously, the more qualities that can be demonstrated, the better are the chances of selection.

1. The applicants will be short listed in accordance with ARC’s procurement/ recruitment procedures, will be interviewed and a contract will be issued to the most suitable applicant.

2. General information of the firm/individual/organization experience in the field of the assignment – number of years of professional experience, description of previous assignments/works; and

3. Information on availability for the assignment.

Desirable:

• Experience of working with faith based agencies, national NGOs and organizations that work in partnership;

• Experience, knowledge and clear understanding of Pakistan humanitarian and development context.