2.1 Data Cycles & Processes
In humanitarian action, effective data management is essential to ensure that information is collected, processed, shared, and stored in ways that are ethical, secure, and effective. This supports timely and informed decision-making, protects the rights and dignity of affected populations, and improves the quality and accountability of programmes.
Responsible data management follows a full data lifecycle, which reflects the journey of data from the identification of information needs to its final evaluation, archiving, or deletion. This chapter outlines nine core steps that make up this lifecycle. Each step must be approached with ethical considerations, technical rigour, and respect for humanitarian principles.
2.1 Applying the Data Lifecycle in Practice
The following nine steps represent the full lifecycle of programme data — from identifying needs to archiving — and should guide ethical, effective, and contextual data management:
Step 1: Define Data Needs
The first step is to define why data is being collected and what decisions or actions it will support. Clearly identify the objectives of the data collection, ensuring alignment with programme needs — whether for monitoring, evaluation, accountability, advocacy, or strategic planning.
This is also the stage to distinguish between:
Primary data: collected directly from people or communities
Secondary data: already available from reports, partners, or databases
Avoid unnecessary data collection by reviewing what already exists. This supports the principle of data minimization, particularly when past data has gone unused.
If personal or sensitive data is involved, a Data Protection Impact Assessment (DPIA) should be conducted to identify and mitigate potential risks. If a DPIA is not feasible, a risk-benefit assessment may be used instead.
Tool Tip: Sample Information Needs Matrix Clearly defining what data is needed, who it’s for, and how it will be used is critical to avoiding over-collection and ensuring data serves its intended purpose.
To support this, you can use an Information Needs Matrix — a practical tool that helps map out:
What data is required
Who needs it (e.g., programme, MEAL, donors)
For what purpose (e.g., targeting, reporting)
When it’s needed
How and how often it will be collected
Step 2: Design and Plan
Once data needs are identified, the next step is to define the methodology. This includes selecting qualitative, quantitative, or mixed methods based on the objectives, designing appropriate sampling strategies, and determining how both primary and secondary data will be collected, particularly from the targeted population, ensuring that the information supports effective eligibility screening, registration, and tailored service provision.
At this stage, teams must be trained not only on the use of tools and data collection techniques, but also on ethical data management practices. This includes obtaining informed consent, handling sensitive interactions with care, and applying data protection principles from the outset.
Planning also involves preparing the necessary tools — such as questionnaires, interview guides, or checklists — ensuring they are respectful, culturally appropriate, and aligned with the specific goals of the activity. In parallel, planners should consider the resources available (human, financial, technical), as well as the timeline and skill levels within the team.
Finally, it is important to define data access protocols. Assign user roles such as data collectors, data viewers, and data administrators according to responsibilities and levels of data sensitivity.
For example, in a mobile data collection platform like KoboToolbox, field staff may be granted access only to enter data, while data administrators at the coordination level manage form design, exports, and quality control.
Step 3: Collect Data
Data collection must be carried out in a way that respects the dignity, rights, and safety of individuals. Participation should always be voluntary, based on informed consent that clearly explains:
The purpose of the data collection
How the data will be used
Who will have access to the data
How long the data will be retained
Field teams should ensure confidentiality throughout the process — both during interactions and when recording data. This includes using secure digital tools (such as KoboToolbox or ODK) with password protection and offline capabilities, or ensuring proper handling and storage of paper-based forms.
It is also important to be aware of and mitigate potential biases — whether in sampling, targeting, or in the way questions are asked. Power dynamics (such as gender, age, or authority relationships) can influence how people respond and must be considered when designing and conducting interviews.
Step 4: Clean and Validate
After data is collected, it must be cleaned and validated to ensure accuracy, consistency, and readiness for analysis.
Cleaning involves reviewing the dataset to identify and correct errors, address inconsistencies, and resolve incomplete responses. This process is not about simply removing data, but rather amending, updating, or clarifying entries as needed — always in line with the original methodology and ethical standards.
Validation confirms that the data is internally consistent, accurate, and fit for the intended use.
When handling personal or sensitive information, apply anonymization or pseudonymization techniques as early as possible to protect identities, especially in protection-sensitive contexts.
Well-designed data collection tools can help minimize the need for extensive cleaning. For instance, if your programme targets individuals between the ages of 5 and 70, that range can be pre-defined within the tool to limit errors during collection.
It is best practice to document all changes made during the cleaning process. A Cleaning Log can be used to track edits, additions, or corrections — helping future teams understand what was changed, when, and why. This is especially important when revisiting data months later or in the case of staff turnover.
Tip: For detailed guidance on data cleaning workflows, see the CartONG Data Cleaning Guide. You can also use a simple Cleaning Log template to track and justify any modifications to your dataset.
Step 5: Store and Control Access
Once cleaned and validated, data must be stored securely and made accessible only to authorized personnel. This step is crucial for safeguarding sensitive information and maintaining the integrity of the data throughout its lifecycle.
Key practices include:
Use secure digital storage platforms with encryption, backups, and role-based access control.
Set access permissions based on roles and responsibilities. For example, data collectors may not need access to full datasets, while programme managers or analysts require access to clean, consolidated data for decision-making.
Restrict access to physical data (e.g., paper forms, consent sheets) by storing them in locked cabinets or other secure locations.
Maintain version control and centralized storage to prevent fragmentation of files across personal devices, email chains, or unprotected folders.
Assign a data protection or IM focal point to oversee access protocols, ensure compliance with policies, and respond to requests related to data subject rights.
Proper storage is not only a technical concern — it is an ethical and legal responsibility, especially when managing personal or sensitive information.
Reminder: Data protection is not just about avoiding breaches, it’s about creating a system where only the right people access the right data, for the right reasons, at the right time.
Step 6: Analyze and Use Data
Once data is stored and secured, the next step is to analyze it in line with the original objectives — whether for programme design, organizing service delivery, monitoring, adaptation, advocacy, accountability, or donor reporting. At this stage, the focus is on turning data into meaningful insights that can support evidence-based decisions and improve the relevance and effectiveness of programming.
Key considerations:
For Programme Service Delivery:
Ensure data is usable for service delivery: Structure and clean data in ways that allow project teams to efficiently register participants, track service completion, and coordinate follow-up.
Support frontline decision-making: Present key information (e.g., caseloads, delivery status, unmet needs) in simple formats for field and area teams to act on.
Maintain up-to-date participant data: Ensure records are current to avoid duplication, exclusion, or mis-targeting — especially in dynamic contexts.
Verify eligibility and coverage: Use data to check that assistance is reaching the intended population segments based on programme criteria.
Enable coordination and sequencing: Where multiple teams/sectors are delivering services, ensure analysis helps identify overlaps or gaps in service delivery.
For Other Functions (IM, MEAL, KM, and ICT)
Ensure inclusive and representative analysis: Check that data is disaggregated and analyzed across key dimensions (e.g., gender, age, disability, displacement status). Consider whether marginalized groups are adequately reflected.
Avoid bias and misinterpretation: Be mindful of sampling limitations, missing data, or potential skew introduced during collection. Document assumptions and methodological choices transparently.
Validate findings where possible: Share key results with communities, partners, or internal teams to confirm accuracy and relevance. Participatory validation can improve both data quality and trust.
Tailor outputs to the audience: Use dashboards, summary briefs, or maps to present insights in accessible formats. Ensure decision-makers receive actionable information — not just raw data.
Link insights to action: Integrate findings into programme review meetings, planning cycles, or coordination discussions to support real-time decisionmaking.
Good analysis doesn't just answer questions — it helps refine them, and informs better decisions at every level of the response.
Step 7: Share Data Responsibly
Sharing data can improve coordination, reduce duplication, and strengthen joint analysis; but it must always be done in a way that protects individuals and communities. It can occur both internally, between departments, teams, or geographic offices, and externally, with partners, clusters, donors, or government stakeholders.
Responsible data sharing means sharing only what is necessary, with the right people, and under conditions that minimize risk.
⚠️ Many of the practices listed below, such as clarifying the purpose of sharing, securing informed consent, and establishing Data Sharing Agreements (DSAs), should be defined during earlier steps in the data lifecycle. These include Step 2: Design and Plan and Step 3 (Collect Data). Step 7 is where these agreements and safeguards are put into action.
Key practices include:
Clarify the purpose of sharing: Only share data when it serves a specific, justifiable need (e.g., coordination with partners, advocacy, planning). Avoid sharing “just in case.”
Use Data Sharing Agreements (DSAs): A DSA outlines what data is being shared, for what purpose, with whom, and under what safeguards. It helps establish mutual accountability and clarity on how data will be used and protected.
Respect consent and privacy: Never share personally identifiable information (PII) unless it was explicitly consented to — and even then, assess whether sharing is necessary and proportionate.
Assess the recipient’s capacity: Ensure partners or stakeholders receiving data have adequate systems and practices to uphold protection standards.
Aggregate or anonymize data where possible: Especially when working with sensitive data, sharing should focus on trends, patterns, or summaries — not raw individual records.
Responsible sharing is not about restricting data, it’s about enabling collaboration without compromising ethics, privacy, or protection.
For guidance on data sharing tools and secure platforms, see 2.2 IM Tools and Data Management
Step 8: Archive or Delete
Decisions about how long to retain data — and whether to archive or delete it — should be guided by considerations made earlier in the data lifecycle, especially during Step 2: Design and Plan. Not all data serves the same purpose or duration: some may be needed only for short-term operational use, while others (such as analysis products, targeting data, or lessons learned) may have longer-term value for learning, accountability, or future programming.
At this step, those predefined retention strategies should be put into action.
Key practices include:
Archive data that has long-term relevance, such as baseline assessments, evaluation reports, cleaned datasets, or analytical summaries — particularly when they support institutional memory, audits, or longitudinal analysis.
Delete data that is no longer needed, especially sensitive or personal data, once its operational purpose or legal retention period has expired.
Ensure secure deletion by removing data from all systems, including backups and local devices, using appropriate digital wiping tools or physical destruction for paper records.
Document retention and disposal actions in your data management log, including the rationale for deleting or archiving datasets.
Apply anonymized archiving where data still holds long-term analytical or operational value but contains personal identifiers. Use advanced anonymization techniques to ensure individuals cannot be re-identified while retaining the dataset’s usefulness for trend analysis and planning.
Data archiving and deletion are not afterthoughts — they are an essential part of ethical and accountable information management.
Step 9: Evaluation of the Data Management Activity
Evaluation is an essential part of the data lifecycle — not just as a closing step, but as a way to reflect on and improve how data was managed across all stages. While some aspects of monitoring and quality assurance happen continuously, this step focuses on structured documentation, learning, and accountability once a data activity has concluded.
Key practices include:
Review what worked and what didn’t: Assess the effectiveness of each step in the data cycle — from planning to sharing — and identify areas for improvement.
Document how data consents were respected: Ensure that the handling, sharing, and retention of personal data was in line with what individuals originally agreed to.
Log incidents and corrective actions: Record any breaches, misuse, or unexpected risks that occurred, along with steps taken to address them.
Capture lessons learned: Include feedback from teams, partners, or communities about how data was used and how future processes can be strengthened.
Maintain a final data management record: This should summarize major decisions, risks, mitigation measures, and outcomes for institutional memory.
Data evaluation isn't just about checking boxes — it's about learning what worked, what didn’t, and how to do better next time.
REFERENCES & FURTHER READINGS:
Last updated