EMIT: Mobile Monitoring and Evaluation

Posted by kdetolly on Apr 28, 2010

System Description

EMIT is an application that allows facilitators to capture field data on cellphones and submit it via GPRS to a centralised database. Surveys are customised and data is monitored, verified and prepared for analysis in real time. Read more here.

From Pilot to National

The pilot was performed with the Community Media Trust (CMT), who used EMIT as a monitoring and evaluation (M&E) tool to capture information on their HIV prevention and treatment literacy sessions in clinics, their training programme and open day events held in public spaces in communities where they work. CMT had been struggling with long turnaround times:

“We had people capturing data online in their remote offices from their paper based forms collected during the week. The issue was that uploading data took a long time on a Friday when staff meet weekly for training and support purposes – the time taken to upload data was impacting negatively on the purpose of the meeting. If all data was not uploaded on one Friday, it was left for the following week resulting in backlogs. This resulted in issues for generating reports from head office in time for our funders and not including all the data for the reporting period because it was late – leading to under-reporting on the number of people reached in a given period. Also, delays meant it was difficult to give timely feedback to the facilitators and make management decisions based on the data.” - Debbie Kroon of CMT

For the purposes of the pilot study, seven facilitators were given handsets to use for data collection. Costs were measured through costs of the handsets, airtime usage and turnaround time was measured from the time of an event to when reporting was available electronically. Accuracy was also monitored, and after three monthly reviews EMIT and the paper-driven process were compared in detail.

Analysis of the results revealed a positive impact on CMT's efficiency. Costs had decreased, in particular the time costs associated with people uploading data at a Friday meeting, and accuracy had increased dramatically. Information was available for analysis immediately. Fieldworkers could submit data from anywhere, reducing errors and allowing greater penetration by researchers. Because incomplete forms could not be submitted, data was more comprehensive. Action could be taken quickly in response to feedback, as the organisation could use the results of research straight away. Workflow was streamlined, and managers were able to measure staff productivity in real time by monitoring the number of forms each employee submitted, conducting telephonic check ups to the clinics and generating accurate reports for funders in a timely fashion.

Later, CMT rolled out national implementation of this pilot to all their staff. Nationally, the results were consistent with the pilot exercise: research was more accurate, costs decreased significantly, productivity increased and research could be used immediately. Training was simple as most fieldworkers were already competent cellphone users, and managers found it easy to monitor staff working remotely.

At the behest of CMT, Johns Hopkins Health and Education South Africa (JHHESA) requested national implementation of the CMT pilot, which was rolled out in phases to 14 of their partner organisations with equal success: Currently the application supports 26 sites across South Africa that actively use the data capture system on a daily basis. These sites are both rural and urban and have shown great promise for extending internationally. These 26 sites to date have submitted over 70 000 forms successfully and managers are increasingly using this system to make key decisions in their organisations, and improve the way they operate.

Practical Pointers

CMT and JHHESA's partner organisations benefited enormously from individual analysis. Consulting with EMIT-linked business analysts proved especially useful as it enabled tailored solutions. EMIT is designed to be customised, and this flexibility was particularly helpful to organisations with complex forms and standard operating procedures.

Training proved unexpectedly easy, as most fieldworkers could already use cellphones. Training was extended to both facilitators and other staff, alongside individual consulting services.

Conclusions

The application's rapid growth indicates the value it adds to organisations requiring a low-cost, efficient data capture system. It is successful in both rural and urban areas. Increasingly, managers are buying into this system to facilitate greater productivity and more effective decision-making in their organisations.

Basic Information

Website URL:

http://www.emitmobile.co.za

Organization involved in the project?:

Cell-Life

Project goals:

Brief description of the project:

A national roll-out of EMIT as a mobile-based data monitoring and evaluation (M&E) system revealed a clear trend of lower costs, greater accuracy and a faster turnaround time on reporting. With proper training and widespread buy-in, fieldworkers used EMIT with successfully and managers found it an efficient and effective monitoring tool.

Target audience:

Field workers in mobile health.

Detailed Information

Mobile Tools Used:

EMIT

Length of Project (in months) :

Status:

Ongoing

What worked well? :

Training was easier than expected as the fieldworkers were already competent cell-phone users.

The data capturing system allowed for real-time access where CMT management could see submissions in real time and make follow up calls to the clinics to ensure their fieldworkers were in fact on duty.

A few organisations had ill-defined organisational processes, making it hard to implement a solution that had little existing foundation. In specific cases it was important to analyse existing processes and re-engineer processes that were redundant. This was done by having workshops and interviews with relevant parties to try and come up with a better defined process that could not only be auditable by funders but logical to the M&E manager. This was very successful as it allowed Cell-Life to analyse organisations and customize the technology accordingly.

What did not work? What were the challenges?:

In terms of the sector, there is a great need for broad-based IT skills as training is still required at most of the partner NGOs. For this reason, training has become a core part of the EMIT product offering.

The lack of network coverage in certain rural areas meant that data capturers had to go to areas with network coverage in order to send their collected forms.

There are challenges involved in the provision of cellphones. Clear policies for cellphone usage are necessary to try and reduce loss. In the near future, the EMIT application will be compatible with all Java enabled cell-phones, meaning that most facilitators will be able to use their personal cellphones, minimizing the cost of providing handsets and the management thereof.