evaluation

Lessons learned: Reflections on M4D Projects

Posted by MelissaUlbricht on Mar 03, 2011

With the increasing number of projects in this mobile-for-change field, there have been a fair share of failures.  We have tried to analyze those with project leaders in our series of FailFaires.

But a project does not need to be a failure to an provide an opportunity for public evaluation, reflection, and dialogue, as we see in a recent series of posts. We were excited to see posts that take an introspective approach: these entries assess program effectiveness, identify gaps in M4D projects, and discuss challenges and solutions in the field.

A post on the Grameen Foundation blog discusses a control trial to asses the impact of a mobile for health project in Uganda:

..We recently completed one of the first randomized control trials designed to assess the impact of a mobile phone-driven health service aimed at improving the lives of the poor.

IQSMS

Posted by bobjay on Oct 20, 2010
IQSMS data sheet 2613 Views
Organization that developed the Tool: 
Main Contact: 
Bobby Jefferson.
Problem or Need: 

Tanzania MOH and donors introduced reporting requirements for PMTCT (preventing mother-to-child transmission of HIV/AIDS) facilities to submit monthly reports to the district and regional levels for progam monitoring and decision making. Futures Group is responsible for collecting reports from PMTCT facilities across four regions. Sites are remote and there is no Internet, and no computers.

To meet the technical and budgetary challenges we developed IQSMS, an open source, freely available SMS reporting tool monitoring and evaluation data collection, to enable facilites to report on PMTCT indicators. IQSMS is used to track program activities at PMTCT sites in rural Tanzania using basic SMS mobile phone technology.

Main Contact Email : 
Brief Description: 

IQSMS is a freely available, software using SMS data in a predetermined format. Information is sent to a dedicated central laptop connected to Motorola mobile phones. The IQSMS software aggregates incoming SMS reports into an SQL database. Automated data quality checks, business rules and immediate notifications are sent to users ensuring that only validated data is added to the database. Aggregated PMTCT data is instantly available to district, regional and national managers based on role views.


FINALIST AND HONORABLE MENTION IN STOCKHOM CHALLENGE AWARD 2010

International Quality Short Message Services focus on development and implementation of a software technology that uses mobile phone to report data to a dedicated centralized computer. This will help coordinating medical care in various part of Tanzania. It is a project with great impact and potential for transferability particularly as the costs of telecom tend to decline with volume. It is a case of e-health, more clearly mobile e-health. International Quality Short Message Services makes health services available. a service which must be recognized. Congratulations.

Tool Category: 
Resides and runs on a computer with tethered modem or mobile phone
Key Features : 

The software uses SMS  data in a predetermined format, sent to a dedicated central laptop connected to a Motorola mobile phone. The IQSMS software records incoming SMS reports into an SQL database.  Automated data quality checks, business rules and immediate notifications are sent to users ensuring that only validated data is added to database. Aggregated PMTCT data is instantly available to district, regional and national managers based on role views.

IQSMS has reduced the time and costs associate with traveling to 535 facilities for data reporting, eliminated need to aggregate paper reports, reduced the need for expensive equipment, improved compliance in monthly reporting and has inspired confidence by rural healthcare workers that their results were accurately reported.

For the period from Jan –Dec 2009, 9867 total reports were submitted and 9400 reports were successfully validated. As the project takes advantage of mobile phones already owned by health workers, the  training requirements are relatively low.

Main Services: 
Information Resources/Information Databases
Tool Maturity: 
Currently deployed
Release Date: 
2009-12
Platforms: 
All phones -- SMS
Current Version: 
1.3
Program/Code Language: 
C/C++
Organizations Using the Tool: 

PMTCT facilities providing services to all mothers, fathers and children to infected or affected by HIV/AIDS. The plan is to scale up to include more PMTCT sites supported by AIDSRelief, the NGO we work with. The IQSMs has been demonstrated to the Ministry of Health and Social Welfare and we are thinking of introducing the system to other regions not supported by AIDSRelief.

Number of Current End Users: 
100-1,000
Number of current beneficiaries: 
100-1,000
Languages supported: 
English, Swahili
Is the Tool's Code Available?: 
Yes
Is an API available to interface with your tool?: 
Yes
Countries: 

Show Me the Literature on Mobile Data Collection!

Posted by MohiniBhavsar on Aug 31, 2010

One of the the key functions of mobile phones is their use in data collection. We have seen lots of online discussion here at MobileActive.org and elsewhere on the subject.

Here, we feature a peer-reviewed journal article from our growing list of resources on mobile data collection. In this 2009 paper, Ping et al. evaluated the effectiveness of PDA-based questionnaire verses a paper-based method for public health surveillance in Fiji.

Show Me the Literature on Mobile Data Collection! data sheet 3250 Views
Countries: Fiji

Handheld Computers for Self-Administered Sensitive Data Collection: A Comparative Study in Peru

Posted by MohiniBhavsar on Aug 18, 2010
Handheld Computers for Self-Administered Sensitive Data Collection: A Comparative Study in Peru data sheet 1144 Views
Author: 
Bernabe-Ortiz, A. et al.
Publication Type: 
Journal article
Publication Date: 
Mar 2008
Abstract: 

A PDA-based program for data collection was developed using Open-Source tools. In two cross-sectional studies, we compared data concerning sexual behavior collected with paper forms to data collected with PDA-based forms in Ancon (Lima).

The first study enrolled 200 participants (18–29 years). General agreement between data collected with paper format and handheld computers was 86%. Categorical variables agreement was between 70.5% and 98.5% while numeric variables agreement was between 57.1% and 79.8%. Agreement and correlation were higher in those who had completed at least high school than those with less education.

The second study enrolled 198 participants. Rates of responses to sensitive questions were similar between both kinds of questionnaires. However, the number of inconsistencies (p = 0.0001) and missing values (p = 0.001) were significantly higher in paper questionnaires.

This study showed the value of the use of handheld computers for collecting sensitive data, since a high level of agreement between paper and PDA responses was reached. In addition, a lower number of inconsistencies and missing values were found with the PDA-based system. This study has demonstrated that it is feasible to develop a low-cost application for handheld computers, and that PDAs are feasible alternatives for collecting field data in a developing country.