Monitoring, Evaluation & Research Program
Monitoring and Evaluation are essential for assessing progress or success of any program/project. It is through the monitoring and evaluation that one is able to assess the intervention output, out come and impact. The monitoring plan enables the program managers and project staff to monitor the input, process and output indicators and finally evaluate the program. Besides monitoring and evaluation, the research component also forms basis of different projects/programs. The research activity is linked with monitoring and evaluation component to envisage the project/program activities.
Monitoring and Evaluation Strategy
HANDS Monitoring and Evaluation program synergies all inputs and output by maintaining Monitoring Information System. MER also conduct rigorous evaluation of programs to provide a more balanced interpretation of performance, while monitoring information has been collected and used for ongoing management purposes.
Need Assessment/Baseline Research of Projects
HANDS MER program assess the need of target population and conduct baseline survey at the village level in identified areas through different qualitative and quantitative tools and techniques including PRA, social Mapping, well being, FGDs and structure questionnaire.
M&E conducted using a wide range of tools, methods and approaches. These include performance monitoring indicator. The logical framework; theory based evaluation, formal surveys such as service delivery surveys, citizen report cards, living standards measurement surveys (LSMS) and core welfare indicator questionnaire (CWIQ), rapid appraisal methods such as Key Informant Interviews, Focus Group Discussions and facilitated brain storming by staff and officials, participatory methods such as participatory M & E, public expenditure tracking surveys, rigorous impact evaluation, cost - benefit and cost - effectiveness analysis.
Major Step of Monitoring & Evaluation strategy
- Development of M&E strategy Document
- Development of AIP/Logical Framework
- Development of Management information System tools
- Define Level of Monitoring and Reporting
Major steps of Research Strategy
- Situation Analysis/ Rapid Appraisal/Baseline
- Midterm Evaluations
- End Project Evaluations
Strategy-I: Development of Logical Framework
The development of Logical Framework Analysis (LFA) is critical for monitoring and evaluation of any project. The document of LFA defines the hierarchy of objectives for every project and is critical for setting up of benchmarks for the target achievements.
Strategy-II: Development of Management information System
The measure the program performance and effectiveness, management information systems are developed including tools of monitoring and evaluation.
|S.No.||Project Name||Donor||District||Type of Research||Total|
|01||MUMKIN||Care International||Shahdadkot||KAP (Households)||780|
|Client Exits Interviews||55|
|Public Health Facilities||11|
|02||NARI||Packard Foundation||Dadu||Direct Beneficiaries||40|
|Public Health Facilities||31|
|Referral Health Care Facilities||4|
|Cost benefit analysis||597|
|03||MISALI||Government of Sindh||Badin||KAP||210|
|Village Screening through PRA||62|
|Public Health Facility||13|
|04||MISALI||Government of Sindh||Karachi Rural||KAPs||210|
|Public Health Facility||13|
|Village Screening through PRA||36|
|05||GAVI||GAVI Alliance Govt. of Pakistan||Matiari||HMCs||27|
|06||MARVI||Packard Foundation||Umerkot||MARVI Workers||30|
|07||FARAH Project||BP Pakistan||Sanghar||TBAs Screening||100|
|08||Noor Muhammad MISALI Village||PPAF & Shell||Karachi||KAP||48|
|09||Child Survival Project||Save the Children||Umerkot||Vaccinaction record of CBAs||48|
|Vaccinaction record of children 11-23 months||31|
Major steps of Research Strategy
Screening is conducted for identification of target population with selection criteria defined specifically for individual projects depending on the interventions.
This activity is conducted to analyze the existing situation thus it could also be the situation analysis. The situation analysis helps in setting up the bench marks for the progress and achievements of any project/program.
This is a research activity which facilitates the monitoring and evaluation activities of any project/program. Quarterly monitoring and Mid-Term Review (MTR) facilitate the identification of shortcomings to be addressed and benefits to be expanded upon in the project.
End Project Evaluations
Post-project or end project evaluations are conducted. The reports of the evaluation include results to date, as reflected by the measurement of targets, lessons-learnt and an overview of program issues. The findings of the report are shared with stakeholders at the national and district level through workshops.
The MUMKIN project aims to work closely with local government to train the Health Management Committees (HMCs) formulated through the devolution plan. The partner (TF) will ensure that the health committees are representative by women to sufficiently reflect the women-based SRH needs. The project will (a) build capacities of the committees to effectively manage the existing SRH service delivery facilities/services. This should ultimately lead to improved governance and accountability among service providers to provide quality SRH services. The project intervention involve 05 Union Councils of Taluka Kamber of the district.
- At least 4 checkups during the pregnancy was taken up by 20% of the women.
- 57.3% births were taking place at home and 42.7 % deliveries were at some health facility.
- 53.9% deliveries were conducted by unskilled birth attendant and 46% by skilled birth attendant
- 58% responded that in case of any need they can visit a health facility in the absence of male member.
- 74% female and 75.8% of the male respondents were aware about the family planning.
- Contraceptive prevalence rate is 16.9
HANDS NARI project is initiated to improve the maternal and child health status with the collaboration of District Health Department. HANDS will introduce an Output Based Aid (OBA) voucher scheme in two Talukas (Johi and Dadu) of district Dadu. The main purpose of project is to provide technical assistance to the public and private health sector to improve health services delivery with a focus on maternal reproductive and child health.
- Fertility rate 5
- Recommended 4 or more Antenatal Visit 20%
- Skill delivery assistance 46%
- Home delivery 57%
- Postnatal Care 66%
- Knowledge of Family Planning Methods 73%
- Contraceptive Prevalence 15.9%
- Unmet need 30%
- Majority of adolescent boys and girls perceive changes of puberty as problems
MISALI (Model for Integral Services & Learning Initiatives)
|Key Findings of MISALI Project||Badin||Karachi Rural|
|Average Household size||6.9%||5.6%|
|Illiterate Head of Household||49.7%||48%|
|Children age 5-16 have never attended school||49.8%||27.8%|
|Toilet Facility (Pit / Flush Latrine)||37.3%||77.4%|
|Making Drinking Water safe||16%||21%|
|Women work for Earning||42%||15.2%|
|Knowledge about at least one family planning method||82%||91%|
|Unmet need for contraception||5.7||25.7%|
MISALI village development project model is based on provision of an essential package of services including poverty alleviation, education, gender equality (women empowerment) to rural population of district Badin and Karachi. The project basically aimed at addressing the Millennium Development Goals (MDGs) in the target population.
The goal of the project was to assist the District Government in achieving the targets of MDGs 4 & 5 i.e. maternal and child health focusing on enhancing the accessibility and quality of health services through strengthening community monitoring and feedback system.
- More than 95% of the women reported to have first two shots of TT vaccination.
- BCG and OPV 0 dose was reported to be received by almost all infants. However, first second and third doses of DPT, OPV and Hepatitis B were reported to be availed by 82, 73 and 60% of infants respectively.
- 27.2% of the women had their last delivery in home. Around 40% women had their delivery in a public sector health facilities (govt. hospital, BHU, RHC) and the govt. hospital.
- 63% women have knowledge about contraceptive methods.
- 51% of the women reported that birth weight of their last newborn was taken.
- 73% of women had their delivery in a public or a private facility by a trained birth attendant.
- 70% of the women were affirmative for their visit for children's immunization.
- 71% Health Facilities maintained the MIS.
MARVI Mid Term Review
The Marginalized Areas RH and FP Viable Initiatives (MARVI) is a three year project funded by the David and Lucile Packard Foundation to Heath and Nutrition Development Society (HANDS). A Mid Term Review of MARVI Project was conducted assess the intermediate outcome indicators at output level. The objectives of Mid Term Review were to:
- Determine the extent to which the project as consistent with and supportive of the policy and program framework within which the project is placed.
- Evaluate the project performance with respect to efficiency (input and activity management) and effectiveness (output, achievements and progress towards achieving the purpose)
- Recommend any required change/modification according to the project scope in order to support effective implementation and accomplishment of the targets within project timeline.
- 89% of MARVI Workers know the recommended number of antenatal checkups
- 90% of MARVI Workers know the recommended number of postnatal checkups
- 46.6% of the MARVI workers were able to identify at least 04 danger signs during antenatal, natal and postnatal period.
- Majority of the MARVI Worker were able to identify the three delays. 80% could identify the first delay, 90% identified second delay and 77% identified the third delay.
- 41.1% of the TBAs were able to idetify at least 04 danger signs during antenatal, natal and postnatal period.
- 52.9% of the TBAs were able to idetify the first clean (this is cleaning of hands) 35% identified second clean (that is place of delivery) and the third clean (that is instrument safe delivery kits) was identified by 41.1%.
- 76% TBAs identified the first delay, 70.5% identified second delay and 70.5 identified the third delay.
- 95% women knew about at least one modern method of contraception.
- 31% of the married women sought antenatal care in their last pregnancy at least 3 visits from trained TBAs.
- 31% respondents seeking contraceptives from public health facilities and 69% respondents seeking contraceptives from private health facilities (MARVI contribution included 41%). Nearly half of the targeted.
- 79% of respondents aware of presence of MARVI and MARVI house within their villages.
- The volunteer blood donors have been identified in all 30 villages and their lists developed and displayed in 47% of MARVI houses.
- The village based telecommunication (Vphone) and transport sources were available and identified in all villages, their name and contact numbers were displayed in the MARVI house.
- 74% of the women got antenatal care during the last pregnancy
- 69% women delivered their last child at their home
- 82% women received some post natal care
- 14.5% of respondents were currently using some family planning methods. It reflects that there is an increase of 5% in current user of Family planning from the baseline.
- Ever user of FP methods were found to increase by 2% increased at 16% in MTR process. This percentage was 14% respectively.
Key Findings of BP Sanghar FARAH Project
- 90% TBAs refer case to other facilities.
- 84% TBAs referred pregnant mothers for TT vaccine.
- 95.6% TBAs did few preparations before delivery.
- 100% TBAs regularly visits in postnatal period
- 78% TBAs advised for FP while to the women.
- 95% TBAs wrapped newborn in a clean cloth after cleaned him with a separate piece of cloth.
- All TBAs tied cord of child from one place.
- All TBAs used new blade to cut the cord.
- 25% women went for antenatal checkups.
- 53% of pregnant women receiving two or more tetanus injections during the last pregnancy. TBAs were the main source to advise most of women to get TT vaccine.
- 16.3% mothers reported that they were asked for space in next pregnancy by TBA.
MISALI Noor Muhammad Village
Rais Noor Muhammad Village is a squatter settlement and the goal of the project was to improve the livelihood of dwellers of the villages. This survey captured information on all the household members regarding sex, education, age, socio economic status, nutritional and vaccination status of under 3 year children and knowledge. Attitude and practices of married child bearing age women.
- Major occupation was picking useable items from garbage
- 98% of the population above 12 years of age were involved in garbage picking for income generation
- 99% of household heads had never attended
- 92% of children aged 5-16 years never attended school
- 38% households have no toilet facility
- No of the household had electricity or gas supply
- 62% women had antenatal checking during last pregnancy
- Two third of the women had not received any tetanus vaccination
- Above 01 year old children had not received any EPI vaccination
Child Survival Project
The project improving child survival in a remote and underserved rural Umerkot district in Sindh Province of Pakistan. The survey was carried out in three UCs of district Umerkot.
- 89% live births from 850 pregnancies
- 70% Antenatal care sought in previous pregnancy
- 64% pregnant women had 2 TT shots
- 63% home deliveries
- 89% women had knowledge about family planning methods
- Contraceptive prevalence rate 18%
- 40% low birth weight
- 90% breast feeding infants
Strategy-IV: Periodical Reviews
Periodical review are conducted to assess the progress made against the targets identified in log frame through participatory approach and to identify issues and challenges for achieving the targets. Suggest and recommendations were given for improving the performance on continuous basis by the general managers.
HANDS MER program has professionals of diversified background to conduct the periodical reviews. Each district is reviewed and monitor by high qualified team. During the year 2009- 2010, 56 reviews were conducted including schools, Health Centers, Villages, community organization, health facilities etc were monitor through this process.
Strategy-V: Internal Evaluations
Review meetings were held at all levels regularly for the review and discussion regarding the projects progress. During the reporting period total 8653 meetings were held at different levels which include community level service providers, community organization Local Govt. Lines Department, Donor/Other NGOs, & Internal meeting of staff at District Office level.
Strategy-VI: Post Project Evaluations
MER compared actual project impacts against the agreed strategic plans to assess the performance of the project and conduct post project evaluation at the end of the project. HANDS in collaboration with British Petroleum (BP) signed an agreement to launch a project named "FARAH "TBAs trainings in remote and underserved rural union councils of Taluka Jam Nawaz Ali, District Sanghar, Sindh Province of Pakistan". The project was designed to improve Reproductive Health practices and Family Planning status in marginalized communities with the purpose to promote safe delivery and to reduce maternal and neonate mortality through capacity building of TBAs and women of child bearing age of the area.
Strategy-VII: Capacity Building
MER Program is focusing on human resource development of organizations and individuals for enhancing the organizational capacity and offered many training packages related to monitoring, evaluation and research in partnership with Human Resource and Institutional Development program of HANDS.
Capacity Building Packages
|Capacity Building Packages||No. of Days|
|Concepts of monitoring and evaluation||3|
|Community need assessment through qualitative tools||8|
|Writing Research Proposal||6|
|Development of monitoring supervisory tools||3|
|Community need assessment through quantitative tools||5|
|Conduction of facility assessment research||3|
|Development of community based MIS||3|
|Client Satisfaction Studies||3|
|Conduction of Evaluation||4|
|Research and Analytical report writing||5|
|Development of project database||5|
Monitoring / Supervisory Visits
During the current year total supervisory visits were 62100.
|S.No.||Conducted by||No. of visits
|2||District Project Manager||2956|
|3||Provincial Project Manager||292|
|4||District Executive Manager||2175|
|5||General Manager/ Managers||626|
|6||COs/other community group||22449|
|7||Local Govt./Lines Department||673|
Management Information System
HANDS Monitoring, Evaluation & Research program is responsible to establish and maintain Management Information System throughout organization from community to Regional Office and Head Office.
To measure the program performance and effectiveness, management information systems are used as tools of monitoring and evaluation. Planners and program managers use these information for their decision-making and modification or improvement of intervention strategies. The monitoring system involves all level of project/ program management and stakeholders. The Management Information System encompasses all the project activities from community level interventions to the top managerial level. The data from intervention level is collected and recorded at the district/project office and from there monthly, quarterly, annual and completion reports for the projects are generated as per requirement.
|Category of Report||Total No. of Reports Submitted|
|Monthly Progress Reports||576||533|
|District Executive Managers Reports||240||230|
|Quarterly Reports of Districts||60||55|
|Quarterly Reports of Projects||154||105|
|Monthly Progress Reports of General Managers||120||120|
|Quarterly Reports of General Managers||40||40|
- Regular Monthly Data collection
- Regular Data entry into Database
- Completion of Data reporting formats for each project
- Completion of Data reporting formats for District Office Managers
- Data transfer to Head Office Database
- Data management at each DO & then at HO
- Data Analysis
- Report writing (Information generation)
- Dissemination (report submission to Donors/ partners)
Status of Report of District, Projects and Program
MER collect report by all programs and feed into data base following table elaborates current year reports.