Our Programmes

Monitoring, Evaluation & Research Program

Introduction

Monitoring and Evaluation are essential for assessing progress or success of any program/project. It is through the monitoring and evaluation that one is able to assess the intervention output, out come and impact. The monitoring plan enables the program managers and project staff to monitor the input, process and output indicators and finally evaluate the program. Besides monitoring and evaluation, the research component also forms basis of different projects/programs. The research activity is linked with monitoring and evaluation component to envisage the project/program activities.

Monitoring and Evaluation Strategy

HANDS Monitoring and Evaluation program synergies all inputs and output by maintaining Monitoring Information System. MER also conduct rigorous evaluation of programs to provide a more balanced interpretation of performance, while monitoring information has been collected and used for ongoing management purposes.

Noor Baloch is Interviewing a Community Person During Survey Noor Baloch is Interviewing a
Community Person During Survey
Dr. Anjum Fatima is Conducting an IDI with Govt. Official Dr. Anjum Fatima is Conducting an IDI
with Govt. Official

Need Assessment/Baseline Research of Projects

HANDS MER program assess the need of target population and conduct baseline survey at the village level in identified areas through different qualitative and quantitative tools and techniques including PRA, social Mapping, well being, FGDs and structure questionnaire.

M&E conducted using a wide range of tools, methods and approaches. These include performance monitoring indicator. The logical framework; theory based evaluation, formal surveys such as service delivery surveys, citizen report cards, living standards measurement surveys (LSMS) and core welfare indicator questionnaire (CWIQ), rapid appraisal methods such as Key Informant Interviews, Focus Group Discussions and facilitated brain storming by staff and officials, participatory methods such as participatory M & E, public expenditure tracking surveys, rigorous impact evaluation, cost - benefit and cost - effectiveness analysis.

Major Step of Monitoring & Evaluation strategy

  1. Development of M&E strategy Document
  2. Development of AIP/Logical Framework
  3. Development of Management information System tools
  4. Define Level of Monitoring and Reporting

Major steps of Research Strategy

  1. Screening
  2. Situation Analysis/ Rapid Appraisal/Baseline
  3. Midterm Evaluations
  4. End Project Evaluations

Strategy-I: Development of Logical Framework

The development of Logical Framework Analysis (LFA) is critical for monitoring and evaluation of any project. The document of LFA defines the hierarchy of objectives for every project and is critical for setting up of benchmarks for the target achievements.

Strategy-II: Development of Management information System

The measure the program performance and effectiveness, management information systems are developed including tools of monitoring and evaluation.

S.No. Project Name Donor District Type of Research Total
01 MUMKIN Care International Shahdadkot KAP (Households) 780
FGDs 8
IDIs 96
Client Exits Interviews 55
Inventory Record 10
Public Health Facilities 11
02 NARI Packard Foundation Dadu Direct Beneficiaries 40
KAP (Households) 420
Public Health Facilities 31
IDIs 40
Referral Health Care Facilities 4
Cost benefit analysis 597
03 MISALI Government of Sindh Badin KAP 210
IDIs 12
VHC Members 994
Households 2782
Village Screening through PRA 62
Public Health Facility 13
04 MISALI Government of Sindh Karachi Rural KAPs 210
IDIs 12
VHC Members 994
Households 3591
Public Health Facility 13
Village Screening through PRA 36
05 GAVI GAVI Alliance Govt. of Pakistan Matiari HMCs 27
VHC Members 994
Household 5070
KAP (Households) 210
06 MARVI Packard Foundation Umerkot MARVI Workers 30
TBAs 30
PDC/VDO 30
Health Facility 5
Client Exits 10
07 FARAH Project BP Pakistan Sanghar TBAs Screening 100
TBAs IDIs 68
Client IDIs 105
08 Noor Muhammad MISALI Village PPAF & Shell Karachi KAP 48
Household data 134
growth monitoring 31
09 Child Survival Project Save the Children Umerkot Vaccinaction record of CBAs 48
Vaccinaction record of children 11-23 months 31
Total 17992

Strategy-III: Research

Major steps of Research Strategy

Screening
Screening is conducted for identification of target population with selection criteria defined specifically for individual projects depending on the interventions.

Situation Analysis/Baseline
This activity is conducted to analyze the existing situation thus it could also be the situation analysis. The situation analysis helps in setting up the bench marks for the progress and achievements of any project/program.

Midterm Evaluations
This is a research activity which facilitates the monitoring and evaluation activities of any project/program. Quarterly monitoring and Mid-Term Review (MTR) facilitate the identification of shortcomings to be addressed and benefits to be expanded upon in the project.

End Project Evaluations
Post-project or end project evaluations are conducted. The reports of the evaluation include results to date, as reflected by the measurement of targets, lessons-learnt and an overview of program issues. The findings of the report are shared with stakeholders at the national and district level through workshops.

MUMKIN

The MUMKIN project aims to work closely with local government to train the Health Management Committees (HMCs) formulated through the devolution plan. The partner (TF) will ensure that the health committees are representative by women to sufficiently reflect the women-based SRH needs. The project will (a) build capacities of the committees to effectively manage the existing SRH service delivery facilities/services. This should ultimately lead to improved governance and accountability among service providers to provide quality SRH services. The project intervention involve 05 Union Councils of Taluka Kamber of the district.

Key Findings

  • At least 4 checkups during the pregnancy was taken up by 20% of the women.
  • 57.3% births were taking place at home and 42.7 % deliveries were at some health facility.
  • 53.9% deliveries were conducted by unskilled birth attendant and 46% by skilled birth attendant
  • 58% responded that in case of any need they can visit a health facility in the absence of male member.
  • 74% female and 75.8% of the male respondents were aware about the family planning.
  • Contraceptive prevalence rate is 16.9

NARI

HANDS NARI project is initiated to improve the maternal and child health status with the collaboration of District Health Department. HANDS will introduce an Output Based Aid (OBA) voucher scheme in two Talukas (Johi and Dadu) of district Dadu. The main purpose of project is to provide technical assistance to the public and private health sector to improve health services delivery with a focus on maternal reproductive and child health.

Key Findings

  • Fertility rate 5
  • Recommended 4 or more Antenatal Visit 20%
  • Skill delivery assistance 46%
  • Home delivery 57%
  • Postnatal Care 66%
  • Knowledge of Family Planning Methods 73%
  • Contraceptive Prevalence 15.9%
  • Unmet need 30%
  • Majority of adolescent boys and girls perceive changes of puberty as problems

MISALI (Model for Integral Services & Learning Initiatives)

Key Findings of MISALI Project Badin Karachi Rural
Average Household size 6.9% 5.6%
Illiterate Head of Household 49.7% 48%
Children age 5-16 have never attended school 49.8% 27.8%
Toilet Facility (Pit / Flush Latrine) 37.3% 77.4%
Making Drinking Water safe 16% 21%
Women work for Earning 42% 15.2%
Antenatal checkup 66.8% 70%
Knowledge about at least one family planning method 82% 91%
Contraceptive Prevalence 11.4 22%
Unmet need for contraception 5.7 25.7%

MISALI village development project model is based on provision of an essential package of services including poverty alleviation, education, gender equality (women empowerment) to rural population of district Badin and Karachi. The project basically aimed at addressing the Millennium Development Goals (MDGs) in the target population.

GAVI

The goal of the project was to assist the District Government in achieving the targets of MDGs 4 & 5 i.e. maternal and child health focusing on enhancing the accessibility and quality of health services through strengthening community monitoring and feedback system.

Key Findings

  • More than 95% of the women reported to have first two shots of TT vaccination.
  • BCG and OPV 0 dose was reported to be received by almost all infants. However, first second and third doses of DPT, OPV and Hepatitis B were reported to be availed by 82, 73 and 60% of infants respectively.
  • 27.2% of the women had their last delivery in home. Around 40% women had their delivery in a public sector health facilities (govt. hospital, BHU, RHC) and the govt. hospital.
  • 63% women have knowledge about contraceptive methods.
  • 51% of the women reported that birth weight of their last newborn was taken.
  • 73% of women had their delivery in a public or a private facility by a trained birth attendant.
  • 70% of the women were affirmative for their visit for children's immunization.
  • 71% Health Facilities maintained the MIS.

MARVI Mid Term Review

The Marginalized Areas RH and FP Viable Initiatives (MARVI) is a three year project funded by the David and Lucile Packard Foundation to Heath and Nutrition Development Society (HANDS). A Mid Term Review of MARVI Project was conducted assess the intermediate outcome indicators at output level. The objectives of Mid Term Review were to:

  • Determine the extent to which the project as consistent with and supportive of the policy and program framework within which the project is placed.
  • Evaluate the project performance with respect to efficiency (input and activity management) and effectiveness (output, achievements and progress towards achieving the purpose)
  • Recommend any required change/modification according to the project scope in order to support effective implementation and accomplishment of the targets within project timeline.

Key Findings

  • 89% of MARVI Workers know the recommended number of antenatal checkups
  • 90% of MARVI Workers know the recommended number of postnatal checkups
  • 46.6% of the MARVI workers were able to identify at least 04 danger signs during antenatal, natal and postnatal period.
  • Majority of the MARVI Worker were able to identify the three delays. 80% could identify the first delay, 90% identified second delay and 77% identified the third delay.
  • 41.1% of the TBAs were able to idetify at least 04 danger signs during antenatal, natal and postnatal period.
  • 52.9% of the TBAs were able to idetify the first clean (this is cleaning of hands) 35% identified second clean (that is place of delivery) and the third clean (that is instrument safe delivery kits) was identified by 41.1%.
  • 76% TBAs identified the first delay, 70.5% identified second delay and 70.5 identified the third delay.
  • 95% women knew about at least one modern method of contraception.
  • 31% of the married women sought antenatal care in their last pregnancy at least 3 visits from trained TBAs.
  • 31% respondents seeking contraceptives from public health facilities and 69% respondents seeking contraceptives from private health facilities (MARVI contribution included 41%). Nearly half of the targeted.
  • 79% of respondents aware of presence of MARVI and MARVI house within their villages.
  • The volunteer blood donors have been identified in all 30 villages and their lists developed and displayed in 47% of MARVI houses.
  • The village based telecommunication (Vphone) and transport sources were available and identified in all villages, their name and contact numbers were displayed in the MARVI house.
  • 74% of the women got antenatal care during the last pregnancy
  • 69% women delivered their last child at their home
  • 82% women received some post natal care
  • 14.5% of respondents were currently using some family planning methods. It reflects that there is an increase of 5% in current user of Family planning from the baseline.
  • Ever user of FP methods were found to increase by 2% increased at 16% in MTR process. This percentage was 14% respectively.
MARVI Worker is Providing Health Services MARVI Worker is Providing Health Services

Key Findings of BP Sanghar FARAH Project

  • 90% TBAs refer case to other facilities.
  • 84% TBAs referred pregnant mothers for TT vaccine.
  • 95.6% TBAs did few preparations before delivery.
  • 100% TBAs regularly visits in postnatal period
  • 78% TBAs advised for FP while to the women.
  • 95% TBAs wrapped newborn in a clean cloth after cleaned him with a separate piece of cloth.
  • All TBAs tied cord of child from one place.
  • All TBAs used new blade to cut the cord.
  • 25% women went for antenatal checkups.
  • 53% of pregnant women receiving two or more tetanus injections during the last pregnancy. TBAs were the main source to advise most of women to get TT vaccine.
  • 16.3% mothers reported that they were asked for space in next pregnancy by TBA.

MISALI Noor Muhammad Village

Rais Noor Muhammad Village is a squatter settlement and the goal of the project was to improve the livelihood of dwellers of the villages. This survey captured information on all the household members regarding sex, education, age, socio economic status, nutritional and vaccination status of under 3 year children and knowledge. Attitude and practices of married child bearing age women.

Key Findings

  • Major occupation was picking useable items from garbage
  • 98% of the population above 12 years of age were involved in garbage picking for income generation
  • 99% of household heads had never attended
  • 92% of children aged 5-16 years never attended school
  • 38% households have no toilet facility
  • No of the household had electricity or gas supply
  • 62% women had antenatal checking during last pregnancy
  • Two third of the women had not received any tetanus vaccination
  • Above 01 year old children had not received any EPI vaccination

Child Survival Project

The project improving child survival in a remote and underserved rural Umerkot district in Sindh Province of Pakistan. The survey was carried out in three UCs of district Umerkot.

Key Finding

  • 89% live births from 850 pregnancies
  • 70% Antenatal care sought in previous pregnancy
  • 64% pregnant women had 2 TT shots
  • 63% home deliveries
  • 89% women had knowledge about family planning methods
  • Contraceptive prevalence rate 18%
  • 40% low birth weight
  • 90% breast feeding infants

Strategy-IV: Periodical Reviews

Status of Periodical Reviews of Projects at Street Level Status of Periodical Reviews of Projects
at Street Level

Periodical review are conducted to assess the progress made against the targets identified in log frame through participatory approach and to identify issues and challenges for achieving the targets. Suggest and recommendations were given for improving the performance on continuous basis by the general managers.

HANDS MER program has professionals of diversified background to conduct the periodical reviews. Each district is reviewed and monitor by high qualified team. During the year 2009- 2010, 56 reviews were conducted including schools, Health Centers, Villages, community organization, health facilities etc were monitor through this process.

Strategy-V: Internal Evaluations

Regular Review Meetings Regular Review Meetings

Review meetings were held at all levels regularly for the review and discussion regarding the projects progress. During the reporting period total 8653 meetings were held at different levels which include community level service providers, community organization Local Govt. Lines Department, Donor/Other NGOs, & Internal meeting of staff at District Office level.

Strategy-VI: Post Project Evaluations

MER compared actual project impacts against the agreed strategic plans to assess the performance of the project and conduct post project evaluation at the end of the project. HANDS in collaboration with British Petroleum (BP) signed an agreement to launch a project named "FARAH "TBAs trainings in remote and underserved rural union councils of Taluka Jam Nawaz Ali, District Sanghar, Sindh Province of Pakistan". The project was designed to improve Reproductive Health practices and Family Planning status in marginalized communities with the purpose to promote safe delivery and to reduce maternal and neonate mortality through capacity building of TBAs and women of child bearing age of the area.

Strategy-VII: Capacity Building

MER Program is focusing on human resource development of organizations and individuals for enhancing the organizational capacity and offered many training packages related to monitoring, evaluation and research in partnership with Human Resource and Institutional Development program of HANDS.

Capacity Building Packages

Capacity Building Packages No. of Days
Concepts of monitoring and evaluation 3
Community need assessment through qualitative tools 8
Writing Research Proposal 6
Development of monitoring supervisory tools 3
Community need assessment through quantitative tools 5
Conduction of facility assessment research 3
Development of community based MIS 3
Client Satisfaction Studies 3
Conduction of Evaluation 4
Research and Analytical report writing 5
Development of project database 5

Monitoring / Supervisory Visits

During the current year total supervisory visits were 62100.

S.No. Conducted by No. of visits
Total
1 Field Teams 32565
2 District Project Manager 2956
3 Provincial Project Manager 292
4 District Executive Manager 2175
5 General Manager/ Managers 626
6 COs/other community group 22449
7 Local Govt./Lines Department 673
8 Donor/Other NGO 364
Total Visits 62100

Management Information System

HANDS Monitoring, Evaluation & Research program is responsible to establish and maintain Management Information System throughout organization from community to Regional Office and Head Office.

To measure the program performance and effectiveness, management information systems are used as tools of monitoring and evaluation. Planners and program managers use these information for their decision-making and modification or improvement of intervention strategies. The monitoring system involves all level of project/ program management and stakeholders. The Management Information System encompasses all the project activities from community level interventions to the top managerial level. The data from intervention level is collected and recorded at the district/project office and from there monthly, quarterly, annual and completion reports for the projects are generated as per requirement.


Information Flow Chart

Information Flow Chart

Reports Generation

Category of Report Total No. of Reports Submitted
Target Submitted
Monthly Progress Reports 576 533
District Executive Managers Reports 240 230
Quarterly Reports of Districts 60 55
Quarterly Reports of Projects 154 105
Monthly Progress Reports of General Managers 120 120
Quarterly Reports of General Managers 40 40
Annual Reports 10 8
Total 1200 1091
  • Regular Monthly Data collection
  • Regular Data entry into Database
  • Completion of Data reporting formats for each project
  • Completion of Data reporting formats for District Office Managers
  • Data transfer to Head Office Database
  • Data management at each DO & then at HO
  • Data Analysis
  • Report writing (Information generation)
  • Dissemination (report submission to Donors/ partners)

Status of Report of District, Projects and Program

MER collect report by all programs and feed into data base following table elaborates current year reports.