Monitoring,Evaluation & Research
HANDS is proud to possess a strong monitoring and evaluation system, which not only enables it to meet its program objectives successfully but also to ensure facilitation to other public and private institutions in the area of MER.
Millennium Development Goals are addressed through different projects of the organization and the MER unit through the development of LFAs. It ensures the inclusion of monitoring indicators to assess the progress and achievements of the projects against the MDGs. The indicators are set against the national targets and timelines to keep track of national development. Following are the key researches carried out by ME&R Program during last year for public and private institutions.
Strategies of MER Program
Major Step of Monitoring & Evaluation strategy
Development of M&E strategy Document
The strategy document defines the steps and procedures that will ensure the smooth implementation of the project and its targets achievement.
Strategy I: Development of Logical Framework
The development of Logical Framework Analysis (LFA) is critical for monitoring and evaluation of any project. The document of LFA defines the hierarchy of objectives for every project and is critical for setting up of benchmarks for the target achievements.
Strategy II: Development of Management information System
To measure the program performance and effectiveness, management information systems are developed including tools of monitoring and evaluation.
Defining Level of Monitoring and Reporting
Different levels of data collection are defined for the projects for generating reports for the project/programs. The data from intervention level is collected and recorded at the regional/project office and from there monthly, quarterly, annual and completion reports for the projects are generated as per requirement.
Strategy III: Research
Major steps of Research Strategy
Screening is conducted for identification of target population with selection criteria defined specifically for individual projects depending on the interventions.
Situation Analysis/ Baseline
This activity is conducted to analyze the existing situation thus it could also be the situation analysis. The situation analysis helps in setting up the bench marks for the progress and achievements of any project/ program
This is a research activity which facilitates the monitoring and evaluation activities of any project/program. Quarterly monitoring and Mid-Term Review (MTR) facilitate the identification of shortcomings to be addressed and benefits to be expanded upon in the project.
End Project Evaluations
Post-project or end project evaluations are conducted. The reports of the evaluation include results to date, as reflected by the measurement of targets, lessons-learnt and an overview of program issues. The findings of the report are shared with stakeholders at the national and district level through workshops.
HANDS MER program assess the need of target population and conduct baseline survey at the village level to identify areas through different qualitative and quantitative tools and techniques including PRA (Participatory Reflection Analysis) , social Mapping, well being, FGDs and structured questionnaire. MER Program conducted different need assessment in following projects.
School Health Screening Project
HANDS launched school health screening project in four talukas of District Thatta and Umerkot. UNICEF provided the financial support for this project to intervene in 240 Govt. Girls and Boys Primary Schools . Cross sectional survey was conducted to examine the general physical examination of children of 42310 students of 568 School of both districts.
Baseline Survey of MARVI Project
The Baseline Survey of Marginalized Areas RH and FP Viable Initiatives (MARVI) Project was conducted to assess the knowledge, attitude and practices on RH-FP, to identify the accessibility of RH-FP services & products and to determine the quality of RHFP services and availability of private and public sector facilities in the District. The sampling design adopted for the survey is a stratified systematic sample of households. Each union council was divided in to 30 clusters based on non-LHW areas. Each cluster comprised of 60-70 households. 5670 Households were surveyed for currently married women of reproductive age (15 – 49 years) who have ess than 2 years of age. Total 12 FGDs conducted including 06 each with married men women of reproductive age. Total 180 Client Exit Interviews were conducted of 36 government & private health facilities in rural, urban, semi urban and remote areas. Digital Mapping of all health facilities of the district was conducted with Geographical Information System (GIS) software by the Population Council Islamabad. The project is funded by David and Lucile Packard Foundation.
Key findings of MARVI Survey
• 78% of the women have been married before 20 years of age early marriage trend is still prevalent.
• Nearly 90% of the women in age groups of more than 40 had 5 or more children
• 52% of the husbands are literate as compared to 14% of women.
• Mean family size is 6.6 (±2.87)
Sexual and Reproductive Health Services for Adolescents and Youth
A qualitative research study for action research was conducted in Union Councils Makhdoom Bilawal and Khudadad in District Dadu. The purpose of the study was to assess the knowledge, perceptions, practices among adolescents and health care providers. Total 12 Focus Group Discussions, 18 In-depth Interviews from male and female adolescents groups and 12 Key Informants Interview from health care providers were conducted in rural areas.
Key findings of Youth Survey
• It was found that majority had more knowledge about physical changes than psychological changes.
• TV, friends, relatives and friends are to be the main source of information regarding sexual and reproductive changes.
Baseline Survey of Releasing Confidence & Creativity (RCC) Project
Baseline Survey for RCC Phase III school was conducted in 99 schools for District Matiari and Hyderabad. Data was collected from 46 schools of Phase I & II and 53 schools from Phase III. Some (62) of schools are for girls only, 36 are for boys only and one school is mix school.
Following activities were also carried out by MER program against the strategy of need assessment and baseline survey.
- Child Tracking Study conducted for 1232 children of ECD classes of RCC project in 50 School of 03 clusters.
- PRA conducted in 31 villages in RCC project.
- Focus Group Discussion conducted with the teachers for the training need assessment in RCC project.
- 76 Focus Groups Discussions conducted in Sindh Coastal Area Development Project (SCAD) funded by Pakistan Poverty Alleviation Fund (PPAF).
- House to House survey of 2 villages conducted for Primary Health Care Exposure of CMWs.
Strategy IV: Periodical Reviews
Program Reviews were conducted to assess the progress made against the targets identified in log frame through participatory approach, identify issues and challenges for achieving the targets and place recommendations for improving the performance till the project life.
Strategy V: Internal Evaluations
It is the responsibility of MER program to conduct evaluation of the program and program within the organization to assess the progress and performance against the target.
Findings of Internal evaluation of RCC Project
Descriptive cross-sectional survey to compare the available baseline information with the project evaluation was conducted after completion of 2nd phase of RCC project. The target areas of the study were the 50 Govt. Girls/Boys Primary School of District Matiari and Hyderabad.
Strategy VI: External Evaluations
HANDS services were hired many times by many public and private organizations to evaluate their programs.
External evaluation of READ Foundation
HANDS conducted an external evaluation of READ Foundation in Azad Jammu & Kashmir. Evaluation checklist, questionnaire and guidelines were used for the assessment and review of existing status and progress of the organization and its intervention units in perspective of the mission of the organization
Strategy VII: Post Project Evaluations
MER compared actual project impacts against the agreed strategic plans to assess the performance of the project and conduct post project evaluation at the end of the project. PAIMAN (Pakistan Initiative for Mothers And Neonates) was a project initiated by USAID & JSI, formally launched from July 2006 in Pakistan. HANDS conducted a Post Project Evaluation on completion of project. Descriptive cross sectional study was conducted to evaluate the project intervention. Qualitative and quantitative methods were used to collect the data and information. The result depicted that most of the communities consulted trained TBAs for the deliveries and preferred them for conduction of deliveries and post natal care. The other most likely possibility is that TBAs behaviors and practices had substantially improved after training and by continuous monitoring and supervision. The assessment of BHU Wahi Pandhi services revealed that the utilization of the BHU for antenatal registration and care, for deliveries and visits for immunization have markedly increased. The referrals from LHWs also contributed in increased utilization rate of the services at BHU Wahi Pandhi.
Strategy VIII: Capacity Building
MER Program is focusing on human resource development of organizations and individuals for enhancing the organizational capacity and offered many training packages related to monitoring, evaluation and research in partnership with Human Resource and Institutional Development program of HANDS.
During the year 2007-2008, MER submitted and presented following research papers and posters in national and international conferences.
1. Reproductive Health of Women in Kacho Desert district Dadu, Sindh – Where is the Way Forward? P. Khalid , F Anjum, Mirza M.S, Ahmed S, T,
2. “LHW -- A Sustainable Attribute Towards Achieving MDGs“ Jaffri R., Fatma A., Mirza M.S, Ahmed S. T
3. EmONC Availability & Accessibility still needed to play its role in Ensuring Safe Motherhood. Fatma A., Mirza M.S, Ahmed S. T
4. HANDS submitted and presented one research paper in International Conference on Information & Communication Technology and Development 2008 in India. J. Sherwani, Fatma A., Mirza M.S, Ahmed S. T
5. HealthLine; Speech based Access to Health Information by Low Literate UserJ. Sherwani, N.Ali, S. Mirza, A. Fatma, Y.Memo