Making source data usable without exposing private information

Before releasing a raw file, decide what detail a reader needs and what can be removed or changed. In Australia, the Privacy Act 1988 (Cth) and Australian Privacy Principles (APPs) govern the handling of personal information.

The OAIC (Office of the Australian Information Commissioner) publishes guidance titled “De-identification and the Privacy Act” and a De-identification Decision-Making Framework.

Separate the working and release files

Keep the original data in a controlled location. Build a release file with documented fields, units, definitions, period and transformation steps.

Remove or alter direct identifiers such as names, addresses, phone numbers and unique IDs. Then assess combinations such as postcode, age and a rare event, which may reveal someone indirectly.

The OAIC describes data as de-identified only when no reasonable steps could identify a person using the data and other information available. Assess the released fields together, not just one field at a time.

Where detailed data creates risk, publish aggregated values and calculation notes instead. Suppress or combine small groups when appropriate, and explain suppression so a reader does not mistake a blank for zero.

For third-party data, check that permissions and licence terms allow the intended public release.

Test usability after protection

Ask a colleague who has not seen the working file to interpret the release file. If they cannot, add a field dictionary or formula note rather than restoring risky records.

Give the file a version and date. Ensure any related chart or methodology note refers to the same version.