A practical, reproducible checklist
1. Protect the raw export
Keep an immutable copy of the original file. Record the instrument, collection date, software version and file format. Always write cleaned data into a different directory, not the instrument export folder.
2. Confirm encoding, delimiter and decimal convention
A laboratory CSV may separate columns with semicolons rather than commas. Decimal comma conventions can turn 2,5 into an invalid value if you blindly apply English-language settings. Check how the instrument encodes missing readings.
3. Normalize names without hiding them
Headers such as "Signal Intensity" and "signal-intensity" may collide after normalization. Rename them deterministically but record a one-to-one map of the original names and column positions.
4. Count missing observations
"NA", "N/A", "null" and blank cells are usually markers, not data values. Report how many are missing in every column. Missing data can be scientifically informative; do not silently fill or delete observations.
5. Recognize duplicates before removal
Identical rows might indicate an accidental repeated export or a valid repeated measurement. Investigate experimental identifiers, time stamps and original logs. An automated cleaner should never deduplicate scientific evidence without an explicit decision.
6. Flag anomalous values, not delete them
An interquartile-range rule is a simple diagnostic, not a measurement of truth. For example, a single signal of 200 beside observations near 10 may reflect an instrument glitch, a real biological event or a unit conversion error. Check the laboratory record.
7. Leave a reproducible audit trail
Export the cleaned CSV together with the input-to-output header map, count of missing fields, suspected duplicates, numerical summaries, and a readable list of assumptions. Keep this report with your analysis so a colleague can re-check it.
Example: tiny synthetic measurement table
| sample id | condition | signal intensity |
|---|---|---|
| S-001 | control | 11.2 |
| S-005 | treatment | NA |
| S-007 | control | 200.0 |
Synthetic values only. This preview is educational; it is not an analysis of client or participant information.
Need the checks as a repeatable script?
Scientific CSV Rescue Kit includes a runnable Python cleaner and JSON report, complete synthetic file, tests and full instructions. It runs locally and does not upload your research data.