TERMS OF REFERENCE (TOR)

 FINAL EVALUATION WOMEN^X LAHORE, FAISALABAD AND ISLAMABAD 

A. PROGRAM OBJECTIVES

Women^x is a pilot program that was launched by the World Bank to support women entrepreneurs in Nigeria and Pakistan. The program is aimed at developing the capacity of women entrepreneurs – with at least 6 months’ business experience – to improve their management skills and business performance, thereby stimulating job creation and contributing to economic growth. The program also aims to foster a more innovative and inclusive entrepreneurial eco-system for women entrepreneurs. In Pakistan the program has been implemented by Enclude since 2014, starting in Karachi and Peshawar, and since 2016 in Lahore, Faisalabad and Islamabad. The program offers a comprehensive support package, which includes business training, mentoring and networking to financial access. 

The program aims to have a positive impact on the business practices and key performance indicators of participating women to test and further refine the approach. Combined with already existing experiences within the Bank and the private sector, the program is aiming to not only improve outcomes for the women participating, but also to generate learning, knowledge and deliverables that can assist with program replication across countries and global stakeholders.

B. ACTIVITIES 

 The program provides the following support for selected participants: 

1. Business Education: Tailored business education through local academic institutions, in particular GCU-Lahore, UAF-Faisalabad and MDI-Islamabad.
2. Soft Skills Training: Highly interactive training sessions focusing on soft skills, such as negotiation, leadership, branding and pitching, by experienced trainers from School of Leadership and GOAL Training.
3. Peer-to-Peer Coaching: Small group coaching sessions addressing personal issues and challenges that the women entrepreneurs face in their business.
4. Mentoring: One-on-on mentoring by experienced entrepreneurs and professionals to support a select group of women with demonstrated growth potential. 
5. Business Clinics: Advanced training sessions on specific topics suggested by the mentors for a select group of women for whom these topics are relevant.
6. Network events: Co-organization of network events and facilitation of women to take part in events organized by third parties, so as to introduce them to new networks.

C. PURPOSE OF THE EVALUATION

The purpose of these Terms of Reference is to provide a framework for planning and conducting the Final Evaluation for Women^x Pakistan’s performance in Lahore, Faisalabad and Islamabad over the implementation period (January 2016 – December 2018). The Final Evaluation will use a combination of quantitative and qualitative methods to ascertain the impact of the program. It will also aim to identify factors enhancing or limiting achievements of targets. 

As the program implementation is expected to end in October 2018, it will emphasize the results achieved and lessons learned for future programming in Pakistan and other women’s entrepreneurship programs in developing countries. 

All key stakeholders will have a highly participatory role in the planning and implementation of the final evaluation as well as input into the final Completion report.  It will be focused on client– level (i.e. at the women entrepreneur level) impacts and spillovers into the ecosystem (where relevant), establishing plausible links between inputs and recorded impacts.

D. SCOPE OF WORK

The consultant will work closely with the Women^x Program Manager and M&E Officer to obtain the required documentation for the evaluation.

1. Review of Documents: 

Undertake review of Women^x theory of change, results framework, and other relevant documents including, but not limited to, the following:

• Final proposal by Enclude to World Bank
• Enclude Quarterly and Annual reports
• The M&E log frame
• Base and end-line data for all program participants (cohort 1 has more limited data) 
• Feedback forms completed by participants
• e-learning platform used for Lahore cohorts, including content, design and usability    
• Any other program documents which will enable the evaluation team to get acquainted with the program design and intent

2. Methodology and data collection tools 

The evaluation will be based on quantitative and qualitative data, and will answer the following questions:

1. Did the participants of WomenX improve their business practices at the point of the end-line? (Improvement should be specifically defined for the purposes of the evaluation, and could incorporate dimensions such as increase in use of business accounts and other financial products/services,) 
2. Did the participants of WomenX report increased sales, profits or employees at the point of the end-line?
3. What are the most useful aspects of the program, as self-reported by WomenX participants? Useful here means contributing to improved business practices and outcomes, positive changes in entrepreneurial orientation (e.g. confidence, goalsetting, calculated risk-taking, planning, networking).  
4. What are the aspects of the program (design, content, delivery mechanisms) that participants of WomenX would recommend amending? Reflect on these outcomes and identify relevant findings. 
5. What is the evaluator’s view of the quality of the business education, the soft skills sessions, the mentoring, the peer-to-peer coaching and the networking activities, in terms of design and delivery? Based on the evaluator’s desk research and experience, how would it compare to other similar programs for women entrepreneurs in the region and globally, both in terms of results at the business level as well as at the program level?
6. What are the views of the various partners and stakeholders in the entrepreneurship ecosystem on the quality of the program, the effectiveness of the program in improving women’s businesses, and on recommendations for future programs? In addition, what, if any, spillover effects has the program had on the entrepreneurial ecosystem, particularly for women, in Pakistan? 
7. How can the program design and delivery model be improved to scale-up similar programs in Pakistan and elsewhere? 
8. What challenges for women entrepreneurs was the program able to address effectively and which ones were not/could not be addressed? The program is not intended to be a ‘silver bullet’ but it would be useful to evaluate what is and what is not achievable within the scope of the program. 
9. Has the program proven to be sustainable in terms of seeding the approach within multiple institutions and developing local capacity to deliver similar programs?  

The evaluator will have use of participant level baseline and end-line data for the participating women, feedback forms, as well as program materials noted above. It is expected that this quantitative data and its analysis will form a critical part of the evaluation. It will need to be augmented by qualitative data, from follow-up in-depth interviews with participants, program providers, other stakeholders in the entrepreneurial ecosystem, and the evaluator’s own conclusions upon reviewing program materials. To this end, the consultant, in close collaboration with the Enclude team, will:

• Develop a detailed methodology for the final evaluation, including mention of what additional material, aside from the baseline, end-line and feedback forms will be obtained, how it will be obtained (focus groups, questionnaires), what the questionnaire (if used) will include, and what approach will be adopted to ensure a representative sample are included. 
• It would also be important to garner feedback from entrepreneurship ecosystem players who may or may not have been directly involved in the program design and delivery in order to ascertain spillover effects. It will be critical to ensure that the methodology agreed is acceptable to Enclude and the World Bank, and that is well documented so that the findings can be presented in as robust a way as possible.
• Based upon reading of the program documents, propose any additional topics or issues for analysis in the evaluation to enhance its robustness and efficacy for program objectives going forward.

3. Field Data Collection

• Plan and coordinate logistics to collect the additional information and data based on methodology.
• Pre-test, edit, translate (if needed), finalize and reproduce the evaluation tools.
• Train and orient field interviewers and enumerators.
• Carry out the fieldwork to sampled respondents, including one on one interviews and phone interviews with the clients.

4. Data entry, analysis and reporting

• Enter, clean, synthesize, analyze, and interpret data from both the quantitative surveys and the qualitative studies using approved statistical packages.
• Prepare a draft evaluation report addressing the objectives and questions of this evaluation outlined in this TOR 
• Prepare a final completion report that includes revisions based on feedback on the draft report and presentation.
• Develop a Power Point presentation of evaluation findings, present and submit to Enclude

5. Scope of the Evaluation

The evaluation will include review of the project design and delivery, program management, beneficiary participation, and impact of the program on beneficiaries’ businesses over the course of the program (with natural limitations based on the date that end-line data was captured) and will obtain inputs from all stakeholders involved. The Independent Evaluator will work directly with the WomenX Project Manager to get access to existing documentation on the project and where needed, contact information for individuals to be interviewed for evaluation.

The Independent evaluator will also work in collaboration with Enclude to: 

• Organize an inception meeting with representatives of  key stakeholders of the program and entrepreneurship ecosystem in all three cities
• Develop a methodology for the evaluation which will include a sampling framework, sampling technique and sample sizes for any quantitative and qualitative surveys
• Plan and Coordinate the necessary logistics to collect the data in line with methodology
• Propose any additional issues for evaluation after reviewing existing project documents 

6. Expected Deliverables/Submission Timeline

November 2018: Inception Report – This will include evaluation design, methodology, detailed work plan (schedule with clear responsibilities for each task) and Inception meeting briefing. It will also include a proposed outline for the report and the Powerpoint.

January 2019: Draft Evaluation Report: This would be reviewed by the Project Manager and other key stakeholders to ensure that the evaluation meets the required quality criteria

February 2019: The Final Report: The final report will include the following:

-          Executive Summary

-          Introduction

-          Description of the evaluation methodology (including potential challenges in implementation of the methodology)

-          Evaluation of program results in accordance with program’s M&E results framework (outputs, outcomes, impacts), with heavy use of infographics (charts and figures)

-          Key findings, including on program design, content and quality of program, mode of delivery, M&E framework, impact on beneficiaries, efficiency, best practices, areas for improvement and lessons learned

-          Conclusion and Recommendations

-          Appendices: Charts, ToR, field visits, people interviewed, documents reviewed.

-          2 Fully bound hard copies and 1 electronic copy (CD)

-          Powerpoint summarizing the main report

In addition, all data sources used in the evaluation must be made electronically available to Enclude. The final report of the independent evaluation of the program will be submitted to Enclude no later than February 28, 2019 and shall cover the period from January 1^st, 2016 to December 31, 2018.

7. Language

All reports to be submitted by the independent evaluator must be in the English Language.

8. Proposal Composition

The proposal for the independent evaluation of WomenX shall be both technical and financial. The technical terms will be 70% of the assessment while the financial terms will be 30%.

a. Technical Proposal Format and Content: The independent evaluator is required to submit a Technical Proposal that shall provide the following information:

-          Brief description of the consulting firm (or individual) and an outline of recent experience on a similar assignment.

-          Comments and workable suggestions on the Terms of Reference that could improve the quality and effectiveness of the evaluation.

-          A description of the proposed approach, methodology and work plan for the evaluation.

-          The list of proposed professional staff team by area of expertise and positions that will be assigned to each staff team and their tasks. CVs should be included.

The Technical Proposal shall not include any financial information.

b. Financial Proposal: Financial proposal shall consist of all costs associated with the evaluation. All activities and items described in the Technical Proposal must be priced and cost broken down by activity.

9. Duty Station/Location 

The Independent Evaluation will take place in Lahore, Faisalabad and Islamabad.

10. Schedule of Payments

The consultant shall be paid the consultancy fee upon completion of the following milestones:

-          30% after adoption of the inception report

-          30% after presentation of the draft report

-          40% after the approval of the final report

The contract price shall be fixed regardless of changes in cost components.

11. Required Qualifications

• The evaluator must have extensive expertise and experience in the field of evaluation of development programmes (preferably in the area of entrepreneurship, SME support and/or private sector development with a gender focus). 
• The evaluation team must include a gender specialist.
• The evaluator must have demonstrable experience on project formulation and monitoring and evaluation of projects, and managing high-level relationships with public organizations.
• Knowledge of entrepreneurship programs globally, and the body of impact evaluation evidence, would be an asset

12. Reporting and Supervision

The evaluator will work under the overall supervision of the WomenX Project Manager. The evaluator will communicate on an ongoing basis and also closely coordinate with the Project Manager and with Enclude. 

13. Sources of Information

1. Baseline Survey on business practices and KPIs
2. Curricula developed by partner academic institutions
3. Teachers involved in Business Education from partner academic institutions: GCU (Lahore), MDI (Islamabad) and UAF (Faisalabad)
4. Trainers from School of Leadership and GOAL Training
5. Peer-to-Peer coaches in different cities
6. WomenX project team and mentor supervisor
7. Chambers of Commerce in different cities
8. Representative group of graduates (Phase 1 and Phase 2)
9. Representative group of mentors in different cities
10. Selection of Business Growth Action Plans developed by graduates