M&E

1.Introduction
2.Monitoring and Evaluation (M&E) Themes
2.1 Inter-M&E at the project level
2.2 Intra-M&E at the university level.
2.3 M&E at HEEPF Level
2.4 M&E at Higher Education System level
2.5 Tracer Studies
2.6 Evaluation Conferences
3. Monitoring System
3.1 Technical Monitoring System (TMS)
3.2 Financial Monitoring System (FMS)
4. Half Annual Progress Report (HAPR)
4.1 Half Annual Technical Progress Report (HATPR)
4.1.1Contents of the Half Annual Technical Progress Report (HATPR)
4.1.2 Supporting Documents
4.1.3 Prior Approval
4.1.4 Request for Installments
4.2 Half Annual Financial Progress Report (HAFPR)
4.2.1 Contents of the Half Annual Financial Progress Report (HAFPR)
4.2.2 Coding of Cost Items
4.2.3 Financial Reporting System
5. Quarterly Interim Progress Report (QIPR)
6. Periodical Site visits
7. Evaluation Methodology
8. Projects' Hand-Over Process
9. Monitoring and Evaluation of the Sustainability Plan

The guidelines presented here are outlined in seven sections. Section 2 of this manual is concerned with the monitoring and evaluation themes considered through HEEPF projects. While Section 3 delineates the technical aspects of the project, Section 4 deals with the financial issues. Section 5 addresses the Half Annual site visits whereas Section 6 outlines the main items considered in the performance evaluation of the projects. Annex 1 of the manual includes the required forms to be used for the preparation of the Half Annual Progress Report (PPR). This report describes the actual performance of the project. A financial request form should be included with the PPR and be prepared by the project management and accounting team. Two copies of this PPR along with the original and a copy of the supporting documents have to be submitted to HEEPF.

Project control is one of the major functions of project management. It enables the project management team to successfully achieve the project objectives. Successful projects are the key elements in achieving the noble goal of enhancing the higher education in Egypt.

Therefore, the HEEPF has established this technical and financial monitoring system. The primary objective of this system is to help the project management team of the funded projects achieve acceptable level of control over their projects in order to successfully complete them. In addition, this system assures fulfilling the World Bank regulations and higher education enhancement loan agreement.

Four levels of monitoring and evaluation of the funded projects are considered including:

• Inter-M&E at the project level.
• Intra-M&E at the university level.
• M&E at HEEPF level and
• M&E at Higher Education System level including:
  • HEEPF System Level Report.
  • Impact assessment and tracer studies.
  •  Evaluation conferences.

The principal of each funded project will be required to produce Half Annual progress reports (PPR) that summarize the project's implementation progress and impact on improved teaching and learning.  The reports will conform to the standard format described below:

The quantitative section of the report would include information on, but not limited to, the following categories:

• Number of new courses and degree programs created (if any ).
• Enrollment in these courses and degree programs.
• Total number of students involved in the project.
• Profile of the students involved (their academic standing; time spent on the project; project related Achivements (e.g., thesis, work experience, etc.)
• Amount of new equipment purchased and ways in which they have been utilized to enhance learning.
• Student related data (e.g., satisfaction ratings, number who have been successful in obtaining work related experience as part of their academic program)

The qualitative section of the report should contain a narrative account of the project's success in implementation, in achieving its stated objectives, and in having an impact in its area or field. Complete financial reports for the project should be included as well.

Monitoring and evaluation shall be carried out through the UPMU and faculty/university. Control of expenditures will be the joint responsibility of the project management team and the financial officer of the UPMU. In case of noting any inconvenience throughout the project implementation, the UPMU manager should promptly report to HEEPF.

Technical and financial monitoring and evaluation shall be carried out by the team assigned by HEEPF to the project. The monitoring team and selective peer reviewers will maintain Half Annual contact (mostly through written correspondence and telephone, especially if there are costs involved in site visits) throughout the life of the project with the project principal.  HEEPF will maintain copies of all PPRs generated by grant recipients and will consolidate the statistical data from these individual reports to provide a description of the fund as a whole, from its logistical functioning to the cumulative impact of the projects on teaching and learning.  These information will be included in the HEEPF annual reports

System Level HEEPF report will conclude with an assessment of any trends within higher education that have been affected (positively or negatively) by the activities of the fund.  This information should enable the Supreme Council of Universities (SCU) to assess the broader impact of the Fund within the HEEP. For instance, it might examine issues of HEIs management, faculty development, or system-wide relevance and efficiency and might conclude with policy recommendations for the higher education sector.

As part of overall monitoring and evaluation effort, the PMU will contract tracer studies to establish graduate employment patterns, for both subproject participants and university/TC graduates in general.  In addition, the PMU will contract random sample surveys of students and professors to obtain information on several aspects of Fund impact.  The World Bank will assist the HEEPF in producing terms of reference (TOR) for these studies and in identifying the skills profile of consultants who would be able to provide technical assistance (TA) for these studies.

 Twice during the course of the project, short conferences would be held to disseminate information about the funded projects and to evaluate their impact on the quality, efficiency, and relevance of higher education.  The first conference may coincide with or take place within six months prior the Mid-term Review of the HEEP (or the third year of the project life cycle).  The second conference would take place six months prior to the completion of the project.

The conference could have the following participation and format: a selection of up to six international peer evaluators and an equal number of Egyptian peer evaluators who would come to Egypt to participate in the conference for a period of one week.  During the first half of the week, the evaluators would divide into two groups.  Each group would make site visits to half a dozen projects (all those for which the group members were evaluators).  During the second half of the week, evaluators would assess the progress of the Fund within the context of the information gathered from site visits.  The peer evaluators would submit their individual reports following the conference.  One international and one Egyptian evaluator would be chosen from the group to write a summary report of the evaluation conference proceedings.  These proceedings would be an important input into the mid-term and final reviews of the project, and could possibly coincide with World Bank missions for that purpose.

The Technical Monitoring System (TMS) should address the actual progress of the project activities, the achieved outcomes during the report period, the problems faced by the project management team and the financial request for the next period. Three types of reports are required through the course of each project. The first is submitted every six months and is termed Half Annual Progress Report (HAPR). The other is submitted quarterly between any consecutive HAPR and is termed Quarterly Interim Progress Report (QIPR). At the end of the project, a Hand-Over Report (HOR) should be submitted with detailed description of deliverables and supporting documents should be enclosed.

The Financial Monitoring System (FMS) is guided by the approved contract budget of the project and serves several purposes including:

1. Planning and recording realistic projection of the funding required at specific point to ensure that the goods and services will be available when needed,
2. Tracking input against planned activities for management control, and
3. Providing a basis for reporting on use of financial or in-kind resources for implementation, evaluation, and audit purposes during the project life-cycle.

The FMS provides the Project Managers (PM's) and accountants with financial guidelines for effectively using the Budget Based Accounting and Control System (BBACS). The BBACS is a system to be used by the project management team to effectively record project's expenditures during project life-cycle. This will help in adequately satisfying the funds' control and sponsor's reporting requirements. BBACS also provides a basis of reviewing and reporting actual use of funds against project planned activities .

The HAPR consists of two parts; technical and financial. The contents of each part if given in details in the following sections.

The HATPR should contain the following items for the pervious period using the given forms in Annex 1.1 of the M&E reporting forms:

1. Brief summary of the project progress.
2. Achieved outputs/outcomes indicating an estimate for the measure of success along with list of the major problems encountered.
3. Brief description of activities related to dissemination, sustainability, quality control, and management of the progress.

In addition, the HATPR should contain the following items for the next period:

1. A detailed time schedule for the activities to be carried out.
2. The tentative staff, equipment, and implementation costs for anticipated deliverables.
3. A financial request to cover tentative activities cost.

The HATPR should be accompanied with the appropriate supporting documents. Such documents may include an international travel technical report demonstrating participation in training, conferences, workshops, site visits, etc. Management activities should be accompanied with minutes of meetings and any other relevant information. Samples of the deliverables either in hard- or soft-copies (even if outputs/outcomes are only partially accomplished) should be submitted. Supporting documents concerning quality control of the project are highly encouraged to be submitted.

Based on the contract document and its annexes, items that require prior approval include but not limited to:

1. International travel.
2. Employment of consultants.
3. Items in hard currency such as purchase, subscription, conference/workshop registration fees, etc.
4. Purchase of equipment and software, ..etc.
5. Subcontracting for more than 20,000 L.E..
6. Reallocation of the approved budget contingency [in the range of 10% limit], if it is not mentioned in the approved project proposal.
7. Changes in the management team.
8. Substantial changes in the project activities, implementation procedures that might have impact on the specific objectives and/or deliverables of the project.

Upon written approval of request to purchase of equipment by HEEPF, procurement procedures may be carried out in accordance with Annex II of the contract. For sustainability requirements care should be given during purchase of equipment to the availability of maintenance, repair, spare parts, etc.

For journals subscription, the project manager should submit a written approval by the institution authorization to continue subscription after the project period to ensure sustainability requirements.

Based on an approved detailed implementation plan for each period, request for installments should be submitted by the project manager including all expected expenditures during each month of this specified period. Upon the approval of this request, a monthly partial installment in local currency will be transferred to the project account. Disbursements in foreign currencies will be processed directly through the PMU at the Ministry of Higher Education (MoHE) against explicit instructions from the HEEPF. Maintaining and managing the project bank account is the responsibility of the project management team and the UPMU financial officer who holds the second signature of the project's bank account.

The HAFPR should contain the following items for the pervious period using the given forms in Annex 1.2 of the M&E reporting forms:

• Monthly time sheet for the entire project team demonstrating the deliverables of each member.
•  Monthly staff cost payroll.
•  Half Annual inventory of equipment.
•  Petty cash register.
•  Travel expenses report.
•  Control ledgers.
•  Payment vouchers.
•  Monthly bank reconciliation.
•  Remaining balance from previous installment.
• Half Annual expenditure report
•  Half Annual institutional co-financing report.

All expenditures for project activities are to be coded according to the following cost categories, (listed in Table 1).

Cost items that are not listed in Table (1) should be assigned to "others" (code 1.99, 2.99, 3.99) for consistency purposes. HEEPF will later assign additional codes for these new cost items.

All budget expenditures should satisfy the following characteristics:

Allowable : Conforming to the specific approved budget, expended within the approved time period, and incurred exclusively for the approved purpose of the fund. It must also comply with al World Bank regulations as described in Annex II of the contract.

Reasonable : Conforming to the ongoing market price of identical item under the same circumstances.

Allocatable: incurred exclusively for the purpose of the project and assigned to a specific activity as described in the project contract.

Approvable:All expenditures are subject to a prior approval from the HEEPF.

The monthly staff cost will be attached with a monthly time sheet for all project participants (Form F1, Annex 1.2 of the M&E reporting forms). A monthly payroll (Form F2, Annex 1.2 of the M&E reporting forms) will be prepared by the project accountant and approved by the project manager. This payroll will indicate the staff costs for each month of the last period and submitted to HEEPF. All staff expenses will be recorded in control ledgers (Form F6, F7 and F8) and submitted along with all original vouchers to HEEPF.

The PFPR should include:

• Control ledgers of all equipment expenses as shown in forms F6, F7 and F8 should be submitted along with all vouchers to HEEPF as described in Annex 1.2 of the M&E reporting forms.
• An inventory list of all equipments using Form F3, Annex 1.2 of the M&E reporting forms. This form should be submitted along with the original receipts.

All implementation expenses will be recorded in control ledgers as shown in forms F6, F7 and F8 and submitted along with vouchers to HEEPF as described in Annex 1.2 of the M&E reporting forms. The project manger is allowed to use a maximum of L.E. 5000 petty cash per period. The project manager should prepare a petty cash Register Form (Form F4, Annex 1.2 of the M&E reporting forms). The project manager has to replenish the petty cash voucher before the end of each period.

Small purchase for shopping and services less than L.E 500 may be made directly with the instructions of the project manager and should comply with the World Bank regulations as described in Annex II of the contract.

Domestic travels' cost will follow the approved HEEPF list. For international travelers, a request for cash advance for 80% of the maximum tabulated limits for accommodation and per-diem, as listed in the World Bank policy, should be submitted with a lead-time of at least three weeks for HEEPF to process the request. The cash will be estimated in accordance with the approved travel time period and travel destination not exceeding the maximum limit in the approved list. Within ten days of return, the travelers should submit a travel voucher to the project manager who will subsequently forward this voucher accompanied with the original used ticket stubs, paid hotel bills, the receipt of conference/workshops fees (if applicable), etc. to HEEPF for cash settlement.

The voucher must be signed by the traveler and approved by the project manager. After the voucher is processed, any unused travel advance must be promptly refunded. It is the responsibility of the project manager to ensure compliance with this procedure. Travel expenses should be reported using form (F5, Annex 1.2 of the M&E reporting forms). It should be noticed that HEEPF will cover only hotel accommodation costs. Any extra hotel costs such as phone calls, mini-bars, room services...etc. will not be reimbursed.

The QIPR consists of two sheets that deliberately reports the ongoing technical and financial progress accompanied by a quarterly bank statement as a proof of previous disbursements. The forms of the QIPR are given in Annex 2 of the M&E reporting forms.

Periodical site visits shall be arranged by the project's monitoring team with selected experts in coordination with the project team and the UPMU. The visit should include, but not limited to, the following:

• Visit to the project site.
• Meeting with the project management team.
• Meeting with the project implementation team.
• Meeting with the stakeholders and beneficiaries of the enhancement process through the project.
• Meeting with the departmental chairman, dean of the faculty and university president or vice-president.

Accompany of the UPMU director is essential during the entire visit. Site-visit report will be prepared by the visiting team and sometimes the project's management team will be asked to carry out self-evaluation in the prepared sheet for this purpose. Feed-back report will prepared by the visiting team and then forwarded to the project's manager, UPMU director and the university president.

Upon the site visits, meetings, demonstrations, presentations, workshops and revision of the progress reports, evaluation of the performance of each individual project will be undertaken based on the following criteria:

• Progress of the project directly towards the targeted objectives outlined in the project proposal.
• Getting the anticipated deliverables according to the time schedule of the project with the required quality.
• Fulfilling the work plan for all activities.
• Efficiency in execution of activities in terms of methodology, preparation, implementation technique, cost, time and quality.
• Technical follow-up within the project to monitor the quality and fulfilling the performance indicators for completion, success and impact besides the application on the beneficiaries of the enhancement process.
• Risk management along with the provisions and measures that have been considered to avoid any delays.
• Financial control process on the expenditures.
• Effectiveness of dissemination and adequacy of announcement/awareness about the project activities, outputs and outcomes. Dissemination is expected to be carried out for all the stakeholders of the project.
• Quality control themes to ensure the quality of the final product of each deliverable.
• Procedures guaranteed and applied to ascertain the sustainability of the project.
• Suitability and accessibility of the assigned location for the project and its furnituring in order to  ensure appropriate execution, mobility and application of the project outputs.
• Suitability, functionability, productivity and utilization of the equipment purchased through the project implementation.
• The amount and sufficiency of the co-financing offered by the university or by the external partners.
• Efficiency in the management system and procedures within the management team besides participation of all related and qualified members in his specialty and the appropriate documentation of the project phases.

According to the weighted average of a specific evaluation point system, the performance of each project shall be ranked with respect to the following categories:

• Distinguished.
• Good.
• Satisfactory with minor changes.
• Requiring major changes.
• Unsatisfactory at all.

The following procedures should be carried out in order to hand-over the project:

• A Hand-over committee (H.O.C.) should be formed by the university president consisting of at least six members (university member, institution/department member, UPMU director, P.M., external expert selected by HEEPF and one of the HEEPF M&E members.
• The P.M. should submit the final report within 30 days after the project closing date according to the contractual items 3 and 6.
• A hand-over conference should be held where institutional staff, specialists, beneficiaries/stakeholders and hand-over committee should be invited.
•  The H.O.C. should submit a report of remarks in view of the site visit, reviewing the final report, and the feed-back of the attendees of the conference. The P.M. should re-submit the final report after taking into account the H.O.C. committee remarks and carrying out all amendments/changes/additions. The report should include items for the quality control measures, impact assessment of the project, co-finance and sustainability plan for, at least, one year post the hand-over date.
• A hand-over statement has then to be signed by the H.O.C., university president and HEEPF director to endorse the sustainability plan.
• Finally, the remaining financial reconciliation shall be made for the project budget.

An Arabic template for the final report is appended in Annex 3 of the M&E reporting forms and can be easily downloaded from HEEPF website (http://www.heepf.org.eg) at the "Public Section".

The sustainability plan of each project will be clearly elaborated and included within the final report of the project and should be further endorsed by the university president as outlined in the hand-over statement. For M&E purposes for the sustainability plan, a sustainability monitoring committee (S.M.C.) should be formed by the university president consisting of at least four members (university member, institution/department member, UPMU director, P.M. of the sustainability process) in addition to the HEEPF consultants. At least two site visits should be conducted to evaluate the implementation of the sustainability plan. Quantitative as well as qualitative indicators have to be considered in the M&E process.

An Arabic template for the final report is appended in Annex 4 of the M&E reporting forms and can be easily downloaded from HEEPF website (http://www.heepf.edu.eg) at the "Public Section".