Data life cycle basics

Data is not a static pile of files. Software is the act of transforming state over time. Without values to read and write, there is nothing for the program to do. In any serious system that state is born, shaped, stored, moved, interpreted, retained, and eventually destroyed—or it lingers forever and becomes a liability. The data life cycle is the standard way computer science and information systems curricula name that path.

This note is a breakdown of the life cycle as a knowledge area: why data is not optional, then definitions, stages, cross-cutting concerns, and how the idea connects to databases, operating systems, security, and analytics.

One-sentence crystal

The data life cycle is the ordered story of information from intent and creation, through collection, processing, storage, use, and sharing, to long-term retention or disposal—with quality, security, privacy, and provenance binding every stage.

---

1. Why there is no program without data

Niklaus Wirth titled a 1976 book Algorithms + Data Structures = Programs. The equation is not decoration. An algorithm is a recipe for operating on operands. Strip the structures that hold those operands, and the recipe names operations that never meet a value.

That is why a life-cycle story belongs in CS, not only in “data admin.” If you cannot say what the bits are, you cannot say what the program is.

if ready:          # ready is a value
    for row in rows:   # rows is a collection of values
        emit(row)

No ready, no branch. No rows, no loop. Control flow is not a second substance floating above data; it is decisions about data.

Think of it like this: logic is pipe; data is water. Empty pipe is not a plumbing system. It is metal.

This note will not treat “data” as a pile that appears after the interesting engineering is done. The stages below are the biography of the only thing programs have to work on.

---

2. Data is a CS knowledge area

Computer science is not only algorithms and machines. A large fraction of real systems is moving bits that mean something under constraints of correctness, cost, latency, law, and trust.

If you can only “query a table” but cannot say where data came from, who may see it, how long it lives, and how it dies, the knowledge area is incomplete.

Learning outcomes

You should be able to:

1. Explain why algorithms without data are not programs (control flow, interfaces, stored-program hardware).
2. Name the main stages of a data life cycle and what happens in each.
3. Separate data, metadata, and derived data.
4. Point to cross-cutting concerns (quality, security, privacy, provenance) at any stage.
5. Map a familiar system (app, lab logger, firmware telemetry) onto the cycle.
6. Ask stage-appropriate questions in design or review.

---

3. Bird’s-eye cycle

Stages are often drawn as a ring because disposal can free capacity for new collection, and because “use” often feeds new creation (feedback). Linearity is still useful for teaching.

                    ┌──────────────┐
           ┌───────►│ 1. Plan /    │
           │        │    create    │
           │        └──────┬───────┘
           │               ▼
           │        ┌──────────────┐
           │        │ 2. Collect / │
           │        │    ingest    │
           │        └──────┬───────┘
           │               ▼
           │        ┌──────────────┐
           │        │ 3. Process / │
           │        │  transform   │
           │        └──────┬───────┘
           │               ▼
           │        ┌──────────────┐
           │        │ 4. Store /   │
           │        │   persist    │
           │        └──────┬───────┘
           │               ▼
           │        ┌──────────────┐
           │        │ 5. Share /   │
           │        │ use / analyze│
           │        └──────┬───────┘
           │               ▼
           │        ┌──────────────┐
           │        │ 6. Archive / │
           │        │   retain     │
           │        └──────┬───────┘
           │               ▼
           │        ┌──────────────┐
           └────────│ 7. Dispose / │
                    │   destroy    │
                    └──────────────┘

   Cross-cutting (every stage): quality · security · privacy · provenance · metadata

Some curricula insert “publish / distribute” as its own stage, or fold archive into storage. The seven stages below keep decisions visible.

---

4. Stage-by-stage breakdown

4.1 Plan and create

Before the first row exists, someone decides what the world is allowed to mean as data.

Create includes synthetic generation, form design, sensor selection, and log schema design. Bad planning cannot be fixed by a prettier dashboard later.

4.2 Collect and ingest

Data enters the system: sensors, user input, APIs, files, scrapers, message queues, lab instruments.

Ingest is the first place loss, duplication, and clock skew show up. Time stamps and source IDs belong here.

4.3 Process and transform

Raw inputs become usable representations: clean, join, normalize, encode, aggregate, feature-engineer, encrypt, anonymize.

  raw event  →  validate  →  enrich  →  transform  →  curated record

Rule of honesty: every transform can destroy information or inject bias. Document it; version it; test it.

4.4 Store and persist

Persistence is where data lives so later stages can find it: filesystems, object stores, databases, data lakes/warehouses, embedded flash logs, cold tape.

Storage is not neutral. Choosing a log file vs a transactional DB is a life-cycle decision.

4.5 Share, use, and analyze

The point of most data: decisions, products, models, human reading.

Share implies authorization, contracts, and often rate limits. “Everyone can SELECT *” is not a share strategy; it is an incident waiting for a calendar date.

4.6 Archive and retain

Not all data stays hot. Archive moves infrequently used data to cheaper, slower media with explicit retention policy.

Archive is still in the life cycle: forgotten archives are silent risk (breach surface, wrong historical truth).

4.7 Dispose and destroy

End of life: delete, shred, crypto-erase, or physically destroy media—according to sensitivity.

Disposal is a first-class stage. Curriculum that stops at “store and analyze” trains incomplete engineers.

---

5. Cross-cutting concerns

These are not “extra chapters after the cycle.” They are properties of the cycle.

5.1 Data quality

5.2 Security

Confidentiality, integrity, availability—and accountability—applied to data at rest, in transit, and in use (access control, encryption, logging, backup security).

5.3 Privacy

Not identical to security. Privacy asks whether collection and use respect people and law (minimization, purpose limitation, consent, subject rights). A well-encrypted hoard of unnecessary personal data can still be a privacy failure.

5.4 Provenance and lineage

Where did this value come from? Source system, transform version, time, operator. Lineage enables debug, audit, and scientific reproducibility. Without it, every dashboard is a rumor.

5.5 Metadata

Data about data: schemas, owners, sensitivity labels, retention tags, dictionary definitions. Metadata is what makes stages operable by people who did not create the data.

  data value  +  metadata  +  provenance  ≈  something you can trust or reject consciously

---

6. Data vs information vs knowledge

Curriculum often layers meanings:

The life cycle primarily moves data and metadata; information appears when use attaches purpose; knowledge often lives in models, docs, and trained people—and can feed new data (labels, features, decisions logged as events).

---

7. One example mapped end-to-end

To illustrate the data lifecycle concept, consider a smart fitness wearable tracking a user's heart rate and daily physical activity. Data is collected continuously from onboard sensors via Bluetooth sync, operating under a defined retention policy and managed by the health platform team. This raw telemetry is then processed to filter out motion artifacts, calculate aggregate metrics like daily active calories and sleep scores, and attach a secure user identifier and UTC timestamp before being stored in an encrypted cloud database with nightly backups. In the operational phase, the data powers the user's mobile dashboard, triggers real-time exertion alerts, and feeds weekly wellness trend reports. After one year of active accessibility, historical telemetry is rolled into compressed cold object storage for long-term multi-year trend comparisons. Eventually, automated lifecycle rules permanently delete the aged data upon expiration, backed by verified backup purges. Throughout this entire pipeline, cross-cutting requirements ensure data quality through outlier filtering, robust security via end-to-end encryption and tokenized authentication, strict privacy compliance for sensitive health metrics, and clear provenance tracked via firmware and schema versioning.

---

8. Design checklist

1. Purpose — Why does this data exist?
2. Birth — Who/what creates it; under what consent or right?
3. Shape — Schema, units, time, identifiers, versions.
4. Path — How does it move (protocols, batches, owners)?
5. Truth — How is quality measured and repaired?
6. Access — Who may read/write/share; how is that enforced?
7. Copies — Backups, replicas, exports—where are they?
8. Clock — Retention and legal hold.
9. Death — How is destruction defined and proven?
10. Feedback — Does use create new data that re-enters the cycle?

---

Closing

The data life cycle is a curriculum backbone for treating information as something that is planned, born, refined, kept, used, and ended—not as a spreadsheet that “just is.” Programs are already that story: they only ever run on state. The cycle is how we refuse to pretend the state has no history and no death.