Annex: SPRING Pathways to Better Nutrition Qualitative Methods - Uganda

SPRING has adapted guidance from several sources to develop a methodology for extracting nutrition-specific and nutrition-sensitive funding data from donor and government budget documents. This document is an annex to the Pathways to Better Nutrition (PBN) Case Study Series that outlines SPRING’s methodology and includes limitations of the data presented, as well as references for further information.

Background

Objectives for Uganda Qualitative Research:

• To explore the processes through which various nutrition stakeholder institutions in-country prioritize their activities to support the rollout of the Uganda National Nutrition Plan (UNAP)
• To identify views and opinions of these stakeholders regarding the various aspects of UNAP roll-out and to inform and strengthen the rollout
• To explore the perceptions of these stakeholder institutions on Scaling Up Nutrition (SUN) and UNAP
• To follow up with potential changes in the priorities over time, understand the rationales that support such changes, and analyze the potential impact of these changes that occur in the rollout process

Methods

The Pathways to Better Nutrition (PBN) case study is a mixed method, prospective study. As an integral part of the case study research, qualitative data were collected throughout the course of the case study. A Grounded Theory Approach was used to identify key themes (drivers of change) across all qualitative data (key informant interviews and focus groups discussions, weekly news content, and meeting notes).  Changes in these key themes, as well as changes in understanding of the UNAP, prioritization, planning, and financing were assessed using an innovative longitudinal grid analysis for each stakeholder group.. This approach was designed to obtain a thorough understanding of the issues stated in the aforementioned objectives, as well as the following key domains of inquiry of the overall case study (below) over time. These domains cut across the following objective areas:

• Learning, adaptation, and evidence on scale-up
• Adaptation of innovations/interventions to local context(s)
• Financing of nutrition-sensitive (sector level) and -specific (within sector) activities
• Long-term planning for sustainability

Sources

Data for the qualitative data stream primarily come from three sources:

• Key Informant Interviews (KIIs): in-depth interviews were conducted at the baseline and endline of the study, with shorter follow-up interviews occurring in between as prompted by current events. In the districts only, focus group discussions (FGDs) were used for follow up.
• News Content Analysis: news articles were collected from seven major Ugandan news outlets on a weekly basis.
• Meeting notes and reports: notes and/or reports were collected from most official UNAP meetings and other notable nutrition gatherings in Uganda.

Key Informant Interviews

Recruiting Key Informants

National Level

Individuals had to meet at least one or more of the following criteria to be considered KIs:

• They were involved in developing UNAP or were well versed in its objectives if not previously involved.
• They had designated roles in the rollout of UNAP within or beyond their specific institutional affiliations.
• They actively participated in or had significant influence in the implementation of UNAP.

In addition, potential KIs needed to be affiliated with the key nutrition stakeholder institutions in Uganda. SUN categorized these institutions into six groups (SUN 2010):

• Government
  • Office of the Prime Minister
  • National Planning Authority
  • Parliament
  • Ministry of Agriculture, Animal Industry and Fisheries
  • Ministry of Education, Science, Technology and Sports
  • Ministry of Finance, Planning and Economic Development
  • Ministry of Gender, Labor and Social Development
  • Ministry of Health
  • Ministry of Local Government
  • Ministry of Trade and Industry and Cooperatives
  • Ministry of Water and Environment (only included at endline)

• Donor agencies (bilateral and multilateral aid agencies)
• Civil society organizations (CSOs)
• Business/private sector
• United Nations (UN) groups
• Academic/research institutions

Sampling was purposive, and the sample for KIs included representatives of all six key stakeholder groups. The final list of KIs was determined through an iterative process between the case study team members in the Strengthening Partnerships, Results, and Innovations in Nutrition Globally (SPRING) Home Office (HO) and country office. 

SPRING case study country staff made phone calls and visits to the offices of the final list of KIs to schedule interviews. An introduction letter from SPRING and support letter from the Office of the Prime Minister (OPM) were delivered to the KIs to invite them to participate in the study.

District Level

The selection of district-level KIs aimed to have representatives from the six key stakeholder groups noted in the national section. A few modifications in the recruitment process were made to accommodate circumstances at the district level. First, all members of the District Nutrition Coordination Committee (DNCC) were included as KIs. Second, since few donor and UN agencies have representation at the district level, representatives of key projects funded by these agencies that operate in the selected case study districts were included as KIs for these groups. Third, community-level opinion leaders (religious leaders, elders, formal and informal practitioners and service providers and members of the private sector) as well as community-based organizations (CBOs) that may potentially influence district-level UNAP rollout were also included.

SPRING case study district research leads and short-term consultants paid an initial visit to the office of the Chief Administrative Officer (CAO) and his technical team in both case study districts (Kisoro and Lira) to present the support letter from OPM and introduce the case study. The district research leads also presented the proposed KI list to the CAO’s office and requested its support to approach and invite the respective KIs, especially those affiliated with the district government, to participate in the interviews. The CAO’s office and the DNCC focal person were also requested to advise on the existing nutrition partners/stakeholders in the district and how to contact them.

Basic Information of Key Informants

National Level

Continual data collection was used for national interviewing. A total of 23 national-level KII were conducted during the case study’s baseline, and the numbers of individuals tracked increased to 26 by the end of the study.  Follow-up interviews were triggered by information in the weekly news analysis and meeting notes. Due to changes of personnel in the government and other stakeholder groups, some KIs who had been interviewed in the baseline did not serve the same role throughout the course of this prospective study. Therefore, the case study followed the incumbents of the “positions,” not the individual KIs. The number of in-depth and follow-on interviews over the course of the study are shown in Table 1.

Table 1. Key Informant Interviews for PBN Case Study - National

District Level

Periodic data collection was used in the districts, due to human and financial resource limitations. A total of 31 and 16 interviews were conducted in the first round in Lira and Kisoro districts, respectively. The difference in the numbers was due to the large presence of partners in Lira compared to Kisoro district. Eleven KIIs and 3 FGDs (with district and subcounty NCCs) were conducted in the follow-up period. The questions followed the same themes as the baseline, with some adjustments to reflect current events.  Table 2 shows the breakdown by group and time in Kisoro district, and Table 3 shows this breakdown for Lira district.

Table 2. Key Informant Interviews for PBN Case Study - Kisoro District

Table 3. Key Informant Interviews for PBN Case Study - Lira District

Interview Tools

National Level

The national KII guide for baseline data collection was developed to capture information related to the objectives section above. The national KII guide included three sections: knowledge, understanding and current responsibilities related to UNAP rollout; processes and rationales to identify and budget for selected priority activities to implement UNAP; and perceptions of scaling up nutrition and its realization in Uganda through UNAP. Due to the differences in the functions that each stakeholder group assumes in support of UNAP, a core body of questions was developed that applies to all groups. Specific questions were developed to ask each group of stakeholders for their unique contribution to UNAP. Eventually six sets of the KII guide were developed following a similar structure.

The draft KII guide went through several rounds of deliberations and revisions among the case study team and the consultants. It also was pilot-tested with a national nutrition leader in-country to assess the clarity and appropriateness of the questions, as well as the time required to complete all the questions. The KII guide was then finalized after the inputs and feedback from the pilot test were incorporated.

For follow-up interviewing, we follow a semi-structured approach with usually 3–5 questions on a current event or budget activity.  The design of the KII tool at endline followed a reiterative process, similar to that in the baseline, to ensure that the questions asked are relevant to KIs from all stakeholder institutions and tightly built around the overall research questions of the case study; additional questions about specific events or themes identified throughout the case study for particular sectors were added to the KII guide at endline.

District Level

At the district level, the first round KII tool adopted the overall structure of and many questions in the national KII tool. Modifications were made to bring UNAP and SUN closer to the local context. The first section of the district KII tool asked KIs’ perceptions of the nutrition situation in their districts. Similarly, the last section of questions on the perceptions of scale-up and rollout of UNAP also focused KIs’ attention on the district where they reside and work. Because each UNAP “early riser” district is required to establish a DNCC to lead the rollout, the questions regarding the prioritization and funding of the key UNAP-related activities centered on how those decisions were made (or to be made) by the DNCC. Therefore, most questions in this section were designed in a way that can be applied to various stakeholder groups in the district. A set of questions was directed specifically to government-sector KIs concerning their relationships with their ministerial counterparts in terms of setting priorities and negotiating budgets. The intention is to establish critical linkages between the national and district levels in UNAP implementation. The tool developed for the second round followed the same themes as the baseline, with some adjustments to reflect current events. Both national- and district-levels KIIs are supported by the OPM. The research protocol and the national KII tool were approved by the Institutional Review Board (IRB) of John Snow, Inc. (JSI) in the USA and the IRB of Makerere University School of Public Health in Uganda.

News Content AnalysisA set of media/news sources were searched retrospectively each week using a set of search terms to follow the same stakeholder groups, related organizations, and events. Six primary news sources were searched:

• Daily Monitor
• New Vision
• Devex News
• Office of the Prime Minister (OPM) News Archive
• Google News
• Uganda National NGO Forum

The search terms used to find relevant articles were at minimum:Nutrition, Health, Population, Agriculture, Water and Sanitation, Education, Gender, Labour, Social Development, Local government, Office of Prime Minister (OPM), Budget, Finance, Donor, Trade, Lira District, Kisoro District. For global media sources these search terms were combined with “Uganda and.” If, in the weekly research meetings, specific projects or events were mentioned, those titles were also used in that weeks’ search.

News articles were captured by the qualitative analyst if they met specific inclusion criteria. These articles were summarized and presented for discussion with the entire case study team on a weekly basis. Based on inclusion criteria, context, and consensus from the study team news articles were included in the master NVivo file for analysis and/or marked for use to guide follow-up with KIs.

Inclusion criteria:

• Mentions anything about the UNAP
• Mentions nutrition in any of the official policy or annual/multi-year plan for one of the key study sectors
• Mentions nutrition-related initiatives in one of the sectors above, or a major shift in ministry priorities
• Mentions developments in national (government-wide) budgets/finances concerning each of the above areas by the key study sectors
• Mentions flow of funding from national level to regions/districts/localities
• Mentions any major event that may have an impact on the budget or priorities of one of the sectors listed above (examples include nationwide bandhs, rare natural events impacting food, agriculture, or access to public services)

Table 4 shows the tallies of the news articles included in the final analysis, broken down by month and related stakeholder group. A total of 262 articles were included. Staffing issues affected data collection in the first six months of the study.

Table 4. Summary of News Article Collection for PBN Case Study – Aggregated by Month, with Relevant Stakeholder Groups Starred.

Document Analysis

Three types of documents, in addition to published news, were collected and used to identify context changes regarding UNAP implementation and to provide guidance for follow-up with KIs.

• Meeting notes: Official meeting notes were collected for UNAP-related meetings, including regular technical working group meetings, stakeholder discussion meetings, and consultative meetings.
• Key policies/plans/guidance documents: New and/or modified documents of UNAP-related strategies and implementation plans were obtained.
• Events: Case study staff took personal notes at various events involving UNAP sectors, including workshops, conferences, discussion meetings, and fora held that discuss UNAP. At times, personal notes were used in lieu of official meeting notes when they were not able to be collected.

The SPRING in-country research team participated in these events, collected or took personal notes, and emailed the notes with their direct observations to the entire team. Documents were summarized and presented at weekly staff meetings; inclusion of documents was agreed upon using the same inclusion criteria as the news content listed above. In total, 22 documents were included from this data stream.

Data Collection, Processing, and Analysis

Collection

SPRING conducted the baseline national interviews in November 2013 in Kampala. The baseline district interviews were collected in Kisoro in May 2014 and in Lira in July–August 2014. Follow-up interviews were scheduled throughout the case study timeline at the national level; a second round of data collection was collected at the district level in February 2015 for both Kisoro and Lira. Endline interviews were conducted at the national level between July and October 2015.

All national- and district-level KIIs were scheduled to occur in the KI’s office and lasted for 30 to 80 minutes. The support letter from the OPM was presented to every KI at the initial meeting. Each KI was requested to sign a written informed consent form to give the case study team permission to ask the KI questions and record the conversation. All KIs agreed to be interviewed, and all but three declined to be recorded. All signed consent forms were kept in a safe place and submitted to the IRB of Makerere University upon the completion of the data collection.  

When permission was granted, the interviews were recorded with a Sony MP3 Portable Digital Voice Recorder (Model ICD-PX333 and 312) or an iPhone, when the digital voice recorder was not available. In addition, notes were taken by the case study team and consultants. All interviews were carried out and all notes were taken in English.

Processing

Notes from KIIs were reviewed on the same day of the interviews. All hand notes were typed up within days of the interviews. The full notes were then produced based on the transcripts of the digital recordings in the following months. Each KI was assigned a code in the full notes. The recordings were erased from the recorder once they had been transferred to a computer for transcription. The file was permanently deleted from the computer once the transcription of notes was completed. The codes and the full notes are stored in a folder on the SPRING project’s central portal, which is only accessible to authorized case study team members. The full notes were verbatim transcription and prepared in Microsoft Word documents.

News sources and documents were copied and pasted into Microsoft Word documents, when possible (if a PDF or PowerPoint document this was not possible). Each news source or document was identified by date published or of personal communication; and, if available, a URL link to the original source was provided.  All news sources and documents are stored in a folder on the SPRING project’s central portal.

Transcripts, news sources, and documents were uploaded and processed in NVivo 10 (QSR International, Australia).

Analysis

SPRING adopted a Grounded Theory Approach to allow the key themes to emerge from the interview notes (Lingard, Albert, and Levinson 2008). During the initial review of the notes, special attention was made to emerging themes that were most aligned with the case study’s primary concerns on the prioritization and funding of activities for the rollout of UNAP, as well as the key domains of the case study.

Initial codes (i.e., master nodes for coding in NVivo 10) were first identified after the baseline KI notes were reviewed. The case study co-leads then discussed and determined the key themes of the technical briefs for the case study, which informed the additional codes created for analysis, including sub-nodes. These additional codes and sub-nodes were developed according to either the existing conceptual framework or emerging themes from the data. Additional codes and sub-nodes were added based on emerging themes from the data; codes and sub-nodes were only added after discussion and consensus across the qualitative research team. All codes and sub-nodes were defined in the case study team’s code book.

All data sources were coded based on the defined codes in Nvivo 10. Due to the desire for preliminary analysis of the baseline data, some of the early coding was done in Microsoft Word; but, after all data collection concluded, all data was re-coded and cross-validated by a second analyst in Nvivo.  The coding was performed by a research analyst and reviewed by the qualitative lead of the case study team.

Strengths and Limitations

Qualitative research methods have unique strengths and are appropriate to study a small number of cases in depth. They are also appropriate to describe and explain a complex process, such as implementing a country’s scale-up nutrition plan. For a longitudinal case study, qualitative methods have the flexibility to be responsive to changes in the field contexts, improving the utility of the findings and recommendations through adaptation of data collection instruments and analysis. Asking the same questions to different individuals helps to detect discrepancies on accounts of facts and sequences, triangulate information to reach a consensus, and collect views and opinions, which could be potentially divergent, on the implementation of national rollout.  Such an exercise will reveal insights into why things are moving or not, and potential resolutions to challenges in the rollout.

Some general weaknesses of qualitative research also apply in this case study, namely that it has a small sample size and is relatively resource intensive. Related to samples size, while we were able to get a fairly representative sample of organizations at the beginning of the study, we lost some academic and private sector KIs to follow up, and were unable to interview one key government policy and oversight organization at the endline. In addition, across all three data streams, we had less than ideal saturation and data flow in the first six months of the study due to staffing issues.     

Strengths and limitations of the quantitative portion of this study are addressed in the Budget Methods Annex and Snapshot Methods Annex.