From Fail to Scale: catching up with the field workers

There is a misperception among many that tech or innovative developments are being introduced from the national level downwards.  When we started to scale-up the reporting with RapidPro in Northern Nigeria, it was immediately evident that the majority of implementation staff at health centers and hospitals were already using SMS to send their regular reports.

sms-on-phone

So consider this a Failure Friday posting on the wrong day.

Across west and central Africa, since the nutrition emergencies of 2010/11,  there has been tremendous pressure put on implmentation staff to send regular report on program activities.  As cellphones became more and more common across the region, it was only natural for fieldstaff to use their phones and their own credit to send the program data. In Nigeria an SMS costs 4 Naira.  This was about 2 cents before the recent decrease of value of Naira.

The formatting of the information in the SMS gently followed the order on the report forms.   Some entertaining reports reported 72 cases under ‘threat’  instead of under treatment, but overall the information was clear and immediately passed to the LGA or state level for input.

This system was reliant on someone to regularly request the data.  And on the receiving end, to have someone available to read, interpret and enter the data into an excel spreadsheet. At the receiving end, these efforts were tedious, time consuming and always at risk of introducing data entry errors.

Now with the roll-out and use of RapidPro, we are able to deliver on our side of the tech / innovative developments with the tools that allow us to catch their data and drop it immediately into analysis and response algorithms to trigger action.  So we changed this from a fail to scale as we begin to catch up with the fieldworkers and their technological capacities.

Over 10,000 weekly reports submitted – RapidPro Nutrition Nigeria

Today, we surpassed the mark of 10,000 weekly program and stock reports submitted for the management of severe acute malnutrition program in Northern Nigeria. While this number does not compare to the number of responses received by UReport, reporting program and stocks data are a different and more complicated responsibility. The initial registration involves completing five questions and correctly submitting a site ID. Weekly program data includes nine questions and stock report data has six questions.

RapidPro

Since July 2016, we have trained 1,580 persons at 651 outpatient and inpatient sites across nine states. We instruct the trainees to come to the training with their past two months of data to use for practice. When the reporters start to use the RapidPro tool, we are already beginning to receive valid data. The trainees are supervised rigorously by their direct supervisors and then by the trainers during the training to ensure that they understand how to report correctly.

As the number of questions to answer is significant and network problems could impede answering all the questions, we were concerned that the reporting burden was excessive. This was not the case. The rate of completion of all questions in the reporting flows is high (82% for program reporting and 88% for stocks reporting). These robust completion rates demonstrate the capacity of the health center staff to send complete data.

Out of the 10,000 submitted reports, there are errors. Some of these errors are caught through the built-in logic behind the question flows. Also, the trainees are trained to validate all of their submitted data and to send a confirmation. When they identify an error, then they can resend the data for that week with all the necessary corrections. The reporting question flows were designed in advance of the launch of the tool and follow the regional guidelines for reporting on management of severe acute malnutrition. These were piloted in the 2013 attempt with RapidSMS and meticulously tested in advance of the state level trainings.

Some RapidPro experts recommend to start very simple and increase the number of questions in the question flows over time. For the management of severe acute malnutrition program this produces incomplete data, which could lead to poor mismanagement or less than acceptable patient outcomes. By triangulating the complete data, it is possible to identify both strong and poor performance allowing coordination staff to target quickly supportive supervision to where it is needed and improve the quality of services.

RapidPro for Nutrition in Nigeria

Across Northern Nigeria, UNICEF is scaling up the use of RapidPro for the monitoring and evaluation of both the prevention and treatment of malnutrition. RapidPro is the UNICEF INNOVATION open-source tool that allows two way communication by text messaging. For this project, we are half way through our scale up plan and have trained over 600 implementation and supervision personnel.  Now we are able to collect program data in real-time. And through an agreement with three of the four national mobile phone providers, all text messages are sent at no cost.

success

The prevention of malnutrition is monitored through reporting on the promotion of optimal infant and young child feeding (IYCF) practices. Data are collected in a monthly summary on IYCF support group participation, supervision and referrals of mothers and children with feeding complications.  Participation and engagement will be strengthened with the introduction of tailor made instructional video content in local languages for use at IYCF support group meetings.

The Nigeria Community based Management of Acute Malnutrition (CMAM) program has quickly become one of the largest programs of its kind in the world.  Currently the programme is treating over 400,000 children a year. Before the introduction of RapidPro, program data took 3 weeks or longer to travel from the treatment site to the national level.  These delays often meant that late detection of stock-outs and other critical events provoked poor or inadequate responses reducing the effectiveness of the program to save lives.

With the introduction of RapidPro, data are sent from implementation sites directly to the national database. Stock-out alerts are sent immediately to program managers and comprehensive analyses shared with the supervision personnel help to detect all other critical events that could interrupt program implementation.  Implementation staff are reminded to send all missing reports weekly if reporting is not complete and immediate feedback is provided on data entry errors.

The health personnel appreciate that use RapidPro both reduces their reporting burden and presents information on the quantity and quality of services delivered. The health service personnel are already comfortable with sending text messages, they often express their delight as they are able to quickly learn how to send data and receive immediate feedback from the RapidPro tool.

Cellphone reporting for Nutrition in Mali

The UNICEF Mali office launched its mobile based Health Management Information System called SNISI in November 2014.  The pilot has successfully run in the Mopti Region of Mali collecting weekly and monthly data on malaria, the integrated management of acute malnutrition (IMAM), reproductive health, EPI, bednet distribution, Trachomatous Trichiasis and other nationally notifiable diseases,

The dashboard is not publicly accessible but is open to partners on request. It was built locally in Mali by the programming team – Yeleman.  The data entry is done on android phones with a purpose built java application and the SNISI dashboard is largely built with python and django. The dashboard is very data rich but works well on small screens of cell phones and large computers monitors.

For the IMAM program, the data comprehensively covers the activities of the three different types of sites URENAS – treating severe acute malnutrition (SAM) without complications, URENI – treating SAM with complications and URENAM – treating moderate acute malnutrition.

The tool collects data on cases treated from all age groups  < 6m, 6-59m, 59m+ for SAM and the above age groups of children and  pregnant and lactating women for moderate acute malnutrition.  For IMAM, stocks data are collected on  14 different items with specific cells to enter, the initial stocks, quantity received, used and lost during program implementation.  The stocks data are collected in the smallest unit, for example RUTF stocks are accounted for in sachets and not cartons. Tests for the consistency of the data are done on the phone during data collection to ensure data quality.

The dashboard provides analysis on how many reports were submitted in a timely, complete and accurate manner.  A system of validation and auto-validation of reports functions to assure that data are of high quality.  The dashboard produces automated monthly reports and weekly updates that are sent by via email.

The next steps with the SNISI tool are to expand its utilization across the country and further develop the mHealth tool for a pilot to improve maternal health in Sélingué health district Mali.

Stock Out

It’s failure friday – Taste my shiny apple

Open Data can be a national treasure – as demonstrated in the most recent elections in Burkina Faso and in their open data initiative.

For old school leaders like Tandja in Niger who was deposed after attempting to cover up a nutrition crisis and Al Bechir in Sudan who was indicted by the ICC while denying the nutrition crises in Darfur, Open Data can be a real political danger.

Since 2011, following the recommendations (commandments) of donors on Open Data, all nutrition survey reports and data were archived online. Colleagues from ECHO following the lead of aid transparency clearly defined the Open Data language required in their funding applications. The colleagues from OFDA did not insist on having the open data agreement in writing, but in 2014 the leadership in Washington DC launched the USAID open data policy. We explained to all government counterparts that without open access to survey data, we could not access donor funding. All government counterparts immediately gave oral consent as they realized the benefits of transparency.

We had repeated difficulties to figure out the best way to host the information on online. The available technical support was never adequate. Posting information online is something that many of us do everyday, but finding a practical method to put these data online was a nightmare. Then we were alerted that without written permission from individual countries we could not provide access to data and the Open Data initiative FAILED.

Pink Lady

In old English most all fruits were called apples (pineapples, custard apples, finger apples, yard apples, earth apples). For some, the most tasty is the fruit of the tree of knowledge. It is obvious that some information is still forbidden fruit. Wow, imagine a life without fruit.

Well now, you don’t have to. READ THIS !

It is a brilliant story of an academic who steals journal articles from the rich and gives free access to everyone.  We are not advocating for thievery, but we do appreciate more open access to scientific articles. We hope that she fairs better than Aaron Schwartz.

And clear efforts are underway to re-open access to nutrition data soon.

 

 

Video of Lancet Breastfeeding Series

EBF in WOLOF

Hey have you seen the launch Video of the Lancet Breastfeeding series already ? Click on the link and WATCH.  It is very inspiring.  Oh, you were sitting between Vicky Quinn and Bob Black in the audience ?

Our Werner Schultink stands out on the panel as a thoughtful leader among a panel of super(heroes), dedicated scientists and advocates.  Although Roger Thurow seemed to call him Banner repeatedly.  Maybe he was making a reference to Batman, I mean the Hulk and famous fitness clubs advocating for breastfeeding.

Some great points from the series:

  • Breastmilk does not make children smarter – formula makes children less intelligent.
  • The total sales value of breastmilk substitutes increases every year and is a growth industry. Apparently the previous executive director of UNICEF knew this when she immediately joined the board of Nestle at the end of her term.
  • Effective behavior change tools exist and many critical missed opportunities have been clearly identified to support best practices.  We have a lot of work to do.

The picture above is from advocacy efforts to promote exclusive breastfeeding in Wolof from Senegal. I would love to see these images and messages on all of the water bottles across the west Africa region.

We have identified previous efforts to use technology to protect, promote and support breastfeeding in Australia and Nigeria, but we need more innovation to support optimal infant and young child feeding.  If you have ideas, please share.

 

 

MUAC measurement – Innovative new tools needed

I was just talking to my friend who reminded me about failure and fridays. It’s friday. We never really like to talk about failure – but let’s do it anyhow.

MUAC strips are very cheap and perfectly tailored to identify of acute malnutrition in children in community and clinic based programs. Their use has allowed an enormous increase in scale of management of severe acute malnutrition programming that has saved the lives of hundreds of thousands of children or more over the past decades.

Ghana Standardization

In nutrition surveys and academic research where robust measurement tools are needed, these MUAC strips are disappointing. Repeated analyses from standardization exercises for nutrition surveys demonstrate that the MUAC strip measures have a variation of between 8 – 10 mm due to the lack of a standard tension applied to the strip at the moment of measure. The average of all MUAC measures from anthropometrists measuring 10 children twice is presented above. If there was no measurement error, then all results would have the same general value.

Research on the accuracy and precision of MUAC measures showed that in national surveys MUAC measures in had more measurement error / poorer precision than weight and height measures. Only the most experienced data collection staff can achieve an acceptable precision (+/- 2 mm) with MUAC measures.  The wider range of precision which we are forced to accept for reasons of expediency creates difficulties to achieve an acceptable accuracy with the MUAC measures.

Observer variability in MUAC measures is caused by setting the band too tight or too loose.  The lack of standard tension creates measurement error that is often not random.  In the training reported above, the anthropometrists with acceptable levels of precision pulled the strip less tightly than those who had poorer precision and less experience with measuring children.  This bias could significantly inflate the prevalence of acute malnutrition in survey results if not addressed properly before the start of the survey. In the past, poor and inaccurate survey results has lead to unacceptable misuse and waste of financial and human resources.

The Gates Global Challenge put out a call to develop innovative methods to measure MUAC.  They were also looking for other alternative approaches to measure child’s malnutrition status such as use of technologies to measure  fat deposits beneath the skin or other electronic technologies to create easy to use anthropometric assessments of malnutrition status. Hopefully something positive will come out of the results from this initiative.