Guinée Gagnant – completes National Nutrition Survey with mobile devices despite EBOLA crisis

Guest post by Ismael Ngnie Teta, Chief Nutrition, UNICEF Guinea

The Guinea National Nutrition Survey 2015 was designed to provide critical information on the nutrition status of children and women of reproductive age and health service delivery as the Ebola epidemic recedes. To ensure the collection of timely, robust and quality data, survey teams entered data into ODK forms on smart phones. The introduction of this innovative approach was skillfully supported by UNICEF Guinea and partners including ACF, HKI, WFP, and Terres des Hommes.

The Ebola context significantly challenged the undertaking of the survey. As the survey teams had to collect data from areas where Ebola case tracking was still going on, the safety of the survey teams and respondents was the first order. A special protocol was designed for the survey including use of hygiene kits and plastic gloves to disinfect all scales, height boards and MUAC strips after each measure. Survey teams were composed of doctors and medical students, which gave them the opportunity to see how both Ebola and the regular seasonal changes affect the nutrition and health of the population.

SMART toise

The physical security of the teams was also of serious concern due to the risk of community resistance and violence. A loss of trust of communities in authorities was observed during the Ebola epidemic. To prevent any misunderstandings and security risks, survey objectives and dates of field work were shared in advance with sampled communities and close communication was maintained with survey teams.

The survey was carried out across the country, including in areas heavily affected by Ebola. A more in-depth sub-sampling was conducted in Kankan region where the population has always been considered at high risk of acute malnutrition. Data were collected on:

  • Anthropometry of children and women of reproductive age
  • Crude mortality rates in population overall and under-five children
  • Vitamin A, deworming and vaccination rates
  • Infant and young child feeding practices
  • Water, Sanitation, Hygiene and
  • Knowledge and practices around Ebola prevention and treatment.

Smart phones were used by the survey teams for the collection of all data except for the anthropometric measurements. As the new WHO flags in ODK questions were not yet available at the start of the survey, the standard SMART method of completing questionnaires and data entry into ENA in the field was done to ensure high data quality. This allowed coordinators to provide timely and appropriate feedback to fieldwork teams as data collection progressed.

C’est Parti! – Senegal launches it’s first National Nutrition Survey on mobile devices

Today the Senegalese Division de la Alimentation et Nutrition et la Survie d’Enfant (DANSE) along with the Agence Nationale de la Statistique et de la Démographie (ANSD) launches the National Nutrition Survey 2015.Sen 24Oct

Intensive advocacy for introduction of more robust and labor saving survey tools led the survey technical commitee to agree to use mobile devices for data collection at the last minute. The agile technical committee was able to quickly adopt the mobile data collection with active technical support from UNICEF and WFP. The Senegal National Nutrition Survey marks the first opportunity to use the newly developed ODK questionnaire with WHO flags in the field.

As decision to integrate mobile devices was made in early October.  The survey methodologies called for the data to be entered on paper questionnaires and the mobile device at the same time.  In the field test, this double effort was difficult for teams with less experience in surveys but immedately adopted by the more proficient teams.

The ODK tool has been rigorously tested found to be rapid, robust and simple to use. An ODK design pattern was identified in the field testing to present a data entry error.  ODK has safety mechanisms to prevents data loss that can block data entry if gross errors are made, for example if data are entered on the weight and height of a woman and then the interviewer changed the sex of the person to male.  This creates a data link error as the weight and height data are protected from deletion but are no longer linked to a household member. The solution is to exit without saving and start with a new questionnaire on the mobile device.

As data collection on mobile devices in Senegal has become more common over the past five years, the survey was able to employ a large number of interviewers with significant experience and capacity for mobile data collection.

A quick review of lessons learned during the training and testing process were:

  • Integrate the mobile device into the entire training process from beginning to end, for example in the registration of trainees, the standardization of anthropometrists, the standardization of anthropometric equipment and the survey questionnaire.
  • Identify the early adopters of technology who are skilled in use of mobile devices and use them as a resource to trainers to ensure all team members learn how to correctly use the mobile device.
  • Buy a small data plan for all mobile devices to send data immediately over 3G / Edge networks but lock access to online diversions like facebook and youtube with an app such as App Lock or similar.

Validation of WHO Anthropometric Flags now available in ODK questionnaires for nutrition surveys

The Open Data Kit (ODK) tool is created a revolution in data collection from mobile devices. It is brilliant for its ease of creation of complex questionnaires with dependencies with on any topic or indicators. The creators really deserve the gun powder prize for their enormous contribution.

The underlying XForms and XLSForms in ODK based on JavaRosa/OpenRosa are very difficult for integration of complex calculations such as those for WHO z-scores. But now an ODK questionnaire form with integrated validation of WHO Flags is available to everyone for free.

WHZ WHO flags

Open Data Kit created a revolution in paperless data collection after its launch in 2010. For environmental and cost saving reasons alone, the paperless data collection provides huge benefits. For improved data quality, these types of tools are unequalled.

To integrate the WHO flags into the form, the corresponding age and measures were calculated by sex of child and/or length/height for the flags for each indicator of child nutritional status:

  • weight for height (+/-5 standard deviations)
  • height for age (+/- 6 standard deviations)
  • weight for age (+5/-6 standard deviations)
  • MUAC for age (+/-6 standard deviations).

Then the curves for all the z-score limits were calculated and incorporated into the ODK questionnaire. Following statistical theory, there is a one in 17,000,000 chance that the anthropometric measures of a child can fall outside of the +/- 5 standard deviation range. This is equivalent to an event that occurs once in the time since recorded history began (~5000 years ago). These statistically improbable cases can be easily caused by entering height in the weight category, mistaking the child’s age by one year or simply guessing at the relative weight and height of a child. Unfortunately, some surveys can have up to 24% of flagged data.

The WHO flags questionnaire follows best practices and does not assume that the data collector has made an error when an indicator is detected outside the WHO flags. The data collection flow goes as follows:

  • Data are entered on age, height, measure by length/height, weight, MUAC and bilateral edema
  • The measures are assessed to determine if any indicators are flagged. If yes, a request for re-measure is made
  • If there are no flags, a random request for re-measure is made for 5% of the children measured, thus it is never evident to the data collector if the re-measure was triggered by a flag or random request
  • The once() function in ODK is used to protect the data from tinkering by constraining the form to make only one test of flags and one random measure on the first data entry.
  • The re-measure is made and data are entered into the form as the second measure of the child.

As the data are send via the 3G or Edge network, they are available immediately for analysis.  With the WHO flags and the request for the random re-measure, the survey coordination team can identify keying errors, gross errors and data manipulation during data collection and respond quickly to prevent these errors from invalidating the survey results.

Using RapidPro for Child Health Weeks

Child health weeks are population schedulable events coordinated by the Ministry of Health organised to ensure high coverage of an integrated package of highly cost-effective services to improving child health and survival provided in conjunction with routine services at health facilities. To support the district level planning and implementation of child health weeks, UNICEF has developed a RapidPro tool to support a comprehensive planning, monitoring and evaluation, to ensure high coverage for the child health week activities. Collection of real-time data and analysis will allow for a more transparent planning, rational use of resources and correct management responses.

Once every six months through child health weeks, health workers target children under the age of 5 years with a package of essential preventive health services usually including vitamin A supplementation, deworming, distribution of insecticide-treated bednets (ITN’s) and/or other services as deemed appropriate. In most countries, the responsibility for planning and managing child health weeks is devolved to the district level. The district health management team is in charge of planning, budgeting, conducting and reporting on activities.
Child Health Weeks

District Health Chiefs are responsible to coordinate both health workers and the community for service delivery. They rely on correct information for the management of stocks, human and financial resources. Careful planning and monitoring of activities helps to support the institutionalization of Child Health Week activities as a routine outreach approach even in contexts where countries are transitioning to a tailored age specific interventions such as the Expanded Program on Immunizations.

Before supervision and implementation staff can use the tool, they are required to register with their phone number, site and position. These data can be entered by individuals themselves or are available from national information systems. Next the data on sites, supervision offices, warehouses and their GPS coordinates are entered to provide an overall view of the system infrastructure.

Child health weeks are normally held two weeks out of the year. The RapidPro tool is designed to support the planning, implementation and follow-up before and after the week activity. Micro-planning in advance allows district chiefs to prepare stocks and human resource needs. The population of coverage area are collected but not always rigorously reviewed. Once the population data are entered, they can be reviewed on the Child Health Week dashboard. The district and implementation sites with poor micro-planning can be immediately detected and corrected through coordination or recounts as needed.

Planning and implementation data will be maintained in health management information systems such as DHIS2. Improved harmonised data on beneficiary populations will remove duplication of efforts across programs and support effective targeting in birth registration, maternal and neonatal survival, EPI, polio eradication and health systems strengthening. As the micro-plans are updated every six months, the data will continue to be reviewed and refined. The automated review of population counts at implementation site, ward and district levels will help to detect and correct of errors in planning figures.

As micro-plans and stock reviews are completed, procurement and pre-positioning of stocks begins. The data on stock movements can be monitored through the Child Health Weeks dashboard or more appropriately integrated into a national logistics management information systems (LMIS). As sites become adequately prepositioned with stock, they will be checked off from a management alerts page on the dashboard. This will allow the supervision staff to quickly follow up on planning issues to ensure preparations are completed in advance. Any necessary orientation or refresher training of service providers can be planned and tracked.

All communications materials and activities necessary to ensure community participation in the child health weeks must be in place and completed in time to ensure awareness and demand for services. Implementation personnel registered with RapidPro can be queried directly on their communication plans. Supervision staff can report on progress of communications work plans. Areas with delays or hindered communications activities can be highlighted on the management alerts page of dashboard for immediate management response.

During the Child Health Weeks, daily reporting on stocks and service delivery will help to maintain appropriate stocks, prevent stock outs and ensure effective coverage at the implementation site levels. Reminders for reporting and alerts on low or no stocks will help district level managers to effectively deliver the interventions. Monitoring using spot checks and supervisory visits can help to ensure quality data are reported.

The analysis of services delivered compared to the micro-plan target populations provide immediate coverage assessments. Mop-up activities can be launched quickly after the validation of low coverage within implementation areas or districts. These conditions are easily identified on maps and tables. District medical chiefs can immediately plan the mop-up activities and assign follow-up tasks to special mop-up teams who have demonstrated their effectiveness during normal work activities.

Coordination of Innovations for Nutrition, WASH and Health Systems Program Monitoring

The introduction of the RapidPro tool is a giant step forward as for improved monitoring and evaluation as users can build, test and fine-tune scalable mobile-based applications. UNICEF has a greater opportunity and responsibility to maximize the potential benefits and support quality standards in reporting while preventing the possible disadvantages of potential fractioning, duplication and over-burdening of reporting tasks caused by the proliferation of new applications and methods.

Introduction of innovation projects

New adaptable plug and play services simplify launching of innovation projects. At the same time many UNICEF offices and government counterparts have had a risk averse behaviour to adopt technological solutions. To get a foot in the door, innovation projects often followed opportunistic paths. In many cases, we have not been able to capitalize fully on the existing opportunities. In most developing countries, coordination issues are evident for innovation. Tech projects often fall under the lead of an innovation wing of the Ministry of Telecommunications. Production of nutrition, WASH and health data should naturally integrate into national information management systems. Governments have committed to use products such as DHIS2 but are still adapting the generic products into locally relevant tools. In a few countries, such as Uganda and Rwanda, government leaders assumed the coordination role to define and coordinate the innovations agenda. In these cases, the government has directly addressed the problems of proliferation of projects, duplication of efforts and demands of excessive reporting burdens.
1000days
The 1000 days – Critical time-period for UNICEF supported interventions for children

Coordination umbrella

The new opportunities for improved services on many different projects, platforms, target audiences and objectives could greatly benefit from an overall coordination framework. As proposed by the innovations section, the UNICEF supported projects building on the strengths of community health workers and the national health system would be buttressed by a 1000 days program structure. Through the global leadership and coordination for nutrition in the Scaling-Up Nutrition (SUN) initiative, UNICEF has lead efforts to introduce mobile tools to ensure the availability and utilization of essential interventions in the 1000 days between a woman’s pregnancy and her child’s second birthday.

UNICEF wants to strengthen community and health system services delivery by providing mobile services for support across the entire continuum-of-care throughout the 1000 days time period. An effective coordination umbrella can help to ensure that critical child survival and development interventions are available and supportive actions for those services are maintained. Opportunities for support include:

  • Implementation of antenatal care (Iron supplementation, Tetanus Toxoid vaccinations, danger signs)
  • Maternal and neonatal survival packages (EMOC)
  • Birth registration
  • Nutrition Package (MNPs, IYCF, VAS/Deworming, Management of SAM)
  • Expanded program for immunization
  • Community based management of pneumonia, diarrhea and malaria
  • Water, sanitation and hygiene services promoted through health centers and workers
  • Stocks management, human resources management
  • Training, testing and motivation/incentives of health service personnel.

This coordination framework should also be inclusive to help support other partners providing:

  • Emergency obstetric care (recognizing and tackling the 3 delays)
  • Reduction of maternal mortality (Rx for post-partum hemorrhage)
  • Access to contraception / family planning services

 

The communications (C4D) aspects for community based and health system service delivery are needed within the coordination umbrella to ensure messaging for improved demand for services is provided and repeated at the correct times, places and media channels. With the new innovations tools, there are greater opportunities to have more qualitative information with UReport and specific triangulated quantitative data through standardized program reporting tools.

With a coordination framework and generic tools for adaption in countries, it will be easier to develop monitoring and evaluation systems in a streamlined fashion. Emphasis is needed to ensure that the monitoring tool can work in areas with limited information availability and in regions with more robust reporting. The reporting tools should be based on national, regional and global protocols. Coordination should provide support both within and across program sections and the coordination work must be in conjunction with all counterparts and strategies in countries (for example with national mHealth policy). The coordination support also needs to provide clear guidelines on how to maintain privacy and anonymity of children and their families while ensuring that all collected data are protected from general release or misuse.

Bottleneck analysis

In countries with the highest rates of infant and maternal mortality, there are often barriers to actually using data for decision-making. National HMIS tools are are often incomplete, inconsistent, or a year out of date. Significant needs are evident not only structures for data collection but also the development of analysis skills and use of results for programmatic guidance. In these cases, the automation of programmatic management support is critical.

The bottleneck analysis framework is a well-defined tool easily grasped by most supervision staff, but sustainability of this analytical thinking is often a problem. Feedback mechanisms to respond to the identified constraints must be developed and supported. The innovations tools can support the regular collection of supply, demand, human resources and enabling environments but targeted messages to support the correct responses and regular reminders are needed to make the assessment, analysis and action cycle function.

Harmonization of data collected

Under the coordination structure, efforts to harmonize the collection of indicators are also needed. Significant efforts have been made by the global and regional offices through the NutriDash reporting tool and harmonization of management of SAM protocols across countries with high burdens of acute malnutrition. Other efforts such as those of David Brown UNICEF to improve reporting on coverage of vaccination rates have made progress. Still many mixed methods of reporting remain. There are needs to standardize reporting both within and across countries. With the introduction of new reporting forms and tools, we have the opportunity to make important improvements while improving the frequency and validity of data.

Equitable delivery of services

UNICEF and partners have demonstrated that integrating mobile technologies into health systems can help to identify problems of time, distance and coordination in the delivery of health services. The emergency responses for Polio and Ebola have contributed enormously to improving maps of areas where populations were typically underserved by health, nutrition and WASH services. Using and contributing to open-source mapping services provides opportunities to better link community health services to the formal health system and to directly target and interact with even the most isolated or often neglected beneficiaries.

To build this coordination framework, we will need three key inputs: country level facilitators of innovation projects, national coordinators to use innovations to improve monitoring and evaluation and global guidance for the design of tools and promotion best practices for implementation of the 1000 days programs.

A maintained supported platform of mHealth / mNutrition applications under the larger 1000 days coordination umbrella and active global support will decrease barriers to use technology for programming implementation, reduce duplication and costs, and promote replicability, accountability and commitment to improved service delivery.

Collection of IYCF Survey Data via SMS

Knowledge about  infant and young child feeding (IYCF) conditions is critical to address malnutrition.  Data collection on IYCF indicators is difficult. The analysis of results are often confounded by slight changes in questions, small sample size and seasonality.  The data collection can be biased by poor attention to detail in questionnaire design, training and limited rigor of interviewers.  Bananas and oranges are often erroneously included as vitamin A rich foods because they are mistaken for orange flesh fruits like mango.

If better methods were developed to facilitate collection of accurate and timely data on IYCF, then greater attention could be directed towards improving feeding and care behaviors.

Xiaozhen Du et al, 2013 compared IYCF survey data collected using SMS against survey data from personal interviews in about 600 children in rural China. Using a test–retest method, they compared a 24–hour recall data from normal survey methods collected during the day to text messaging collecting data later during the same day.  They asked the same five IYCF questions on how caregivers fed their children yesterday for both methods.

One criteria for participation in the study was caregiver’s easy access to cellphones. As households were selected by this point and not randomly, the sample was not representative of the general population. The caregivers interviewed were also paid a little less than 1$ for participation in the personal interview and again about 1$ if they completed the 2nd interview by SMS.

The response rate for the text messaging survey was poor compared to household surveys using personal interviews.  The response rate was 57% for the first SMS question and declined to 49% by the 5th SMS question.  This level of non-response would invalidate the results of any survey.  The data agreement between the two methods was considered “excellent” for the question on breastfeeding with a Kappa statistic of 0.81. I find this disagreement with two different answers on breastfeeding on the same day from the same respondent disconcerting.  The remaining specific questions on child feeding (grains, roots, tubers/ legumes and nuts / dairy products / flesh foods / eggs / vitamin A rich fruits and vegetables / other fruits and vegetables) were poor with Kappa ranging between 0.02 and 0.36.  This means that the results on the later questions could not be considered meaningful compared to the traditional face to face survey.

This was a very interesting and well written piece of research but unfortunately, this introduction of technology into survey methods does not allow us to make accurate or representative estimates on IYCF. We need to keep these results in mind when interpreting survey data collected with similar methods in the humanitarian context or development context.

 

 

The Fault in our Stores (of data)

Bench marking

The journalist Martin Plaut wrote an excellent article about the Trouble with African Statistics.   He identifies several cases of statistics reported as facts that are based on little more than thin air.  Imagine how difficult it is to track conditions in countries like Angola, which has not conducted a census in over 40 years and has not conducted a nutrition survey since 2008.  He makes it very difficult to miss the point by stating that “often African statistics are just politically driven guesses”.  The article was based on analysis by Good Governance Africa to be released on the 28th October.

The issues of poor quality or missing data also affect economic statistics. The recalculation of the GDP of Nigeria caused a great stir as overnight, the beehive of Africa became the largest economy. The new calculations of economic activity correctly included the Nollywood film industry, banking and the telecoms. Unfortunately they left out the value of breastmilk production which is most likely also huge.  These issues have been researched by economic historian Morten Jerven in his book Poor Numbers and reported as an editorial in the Lancet. The book explores the weaknesses in the National Statistical Offices and the effects on planning when administrative data for the day-to-day operation of governments do not exist.

Now that Ghana and Nigeria have recalculated their GDPs, most other countries in the region are considering to follow suit.  Ben Leo of the Center for Global Development reports that we need to “revisit what we think we really know” about these economies. An enormous number of econometric studies were done on data that has become redundant overnight.  The public spending in Nigeria on nutrition, health, education, etc…  was low to begin with.  Now with the readjustment of the economic condition, we see that social spending is closer to 1 percent of GDP compared to 2 percent formerly reported.

As Morton Jerven stated, “It quickly becomes evident that any statement about the size and direction of global poverty is relying on a lot of assumptions and extrapolations”.

And our data on the chronic and acute emergencies regularly battering the sub-Saharan populations are also problematic. The Active Learning Network for Accountability and Performance in Humanitarian Action (ALNAP) report on use of evidence in humanitarian action finds that are early warning data is often incomplete and monitoring is regularly very poor, inaccurate and non-representative.

We have tremendous opportunities with mobile technologies to collect more information, more quickly and with more granularity than ever before.  To address these weaknesses in data, there are many initiatives, but it clear that greater coordination, harmonization of critical indicators and more rigorous and regular data collection is needed

Apologies for the draft version sent out earlier. It is failure friday, and I am still learning how to work these high tech tools (blogs) for nutrition.  I tried to retract the first posting and ended up here (retractionwatch.com)