The short answer
Decide what makes a record a duplicate before removing it. Separate identical parsed rows, repeated exact keys with different values, and different keys sharing a name. The inspector can count the first two categories; it cannot establish that two records describe the same person or decide which conflicting value is true.
What you’ll take away
- An extra row is not automatically an extra person.
- A shared display name is not a merge instruction.
- Preserve the source and document every removal or unresolved conflict.
Define what one row represents
An export might describe a connection, an observation or an interaction. Repeating a profile reference can be expected in an interaction log and suspicious in a one-row-per-connection table. Without a row definition, removing repeats can destroy valid evidence while making the table look cleaner.
Write the intended row unit and proposed key. If the source supplies no trustworthy unique key, do not invent person identity from a name. You can still flag candidate records for inspection. The goal is a defensible working table, not the smallest possible row count.
Keep three different duplicate questions separate
Identical parsed rows have equal field values in equal order. Their raw CSV bytes can still differ in quotation or line endings. Repeated keys may have conflicting company, name or date fields. Similar names with different keys are merely candidates for further contextual inspection.
The local inspector does exact text comparison for keys; it does not trim or normalize them into a match. Leading/trailing key whitespace is reported separately. This avoids silently turning a data-cleaning assumption into an identity decision. Width-mismatched records are counted separately and excluded from conditional key/email/row-duplicate checks.
| Case | Possible action | Required caution |
|---|---|---|
| Identical parsed row | Review an extra copy in a working derivative | A valid repeated observation may still need retention |
| Same key, conflicting values | Keep a conflict queue | Do not choose whichever row came last without a rule |
| Same name, different keys | Keep separate by default | Name resemblance does not establish identity |
Run the four-record exercise
Load the fictional example with heading record 2 and key column 3. Four matching-width data records produce one identical-row group with one extra record, and one repeated-key group with one extra record. These two totals overlap; adding them together would double-count the same repeated input.
The two Mira Demo references ending /a and /b remain different. The /a record occurs twice with identical parsed values. The inspector does not reduce four rows to three people or establish anyone’s real identity. It reports record structure and exact repetitions in an invented exercise.
Make one explicit decision per candidate
Give each candidate a local record reference and a reason for review. Compare the relevant source fields and ask whether the row unit permits repetition. For a conflicting key, preserve both values until a permitted authoritative source or a documented source-priority rule resolves the conflict.
A short decision log can contain source identity, candidate positions, rule, decision and unresolved reason. Keep names and email addresses out of a broadly shared log. If you cannot distinguish an accidental copy from a legitimate repeated observation, retain the candidate and disclose the limitation.
- Preserve the original file.
- Declare the row unit and exact comparison key.
- Separate identical rows, conflicting keys and name resemblance.
- Inspect candidates and record the decision rule.
- Create a separate derivative only for justified decisions.
Reconcile removals rather than celebrating a clean count
For a working table where the repeated /a row is confirmed to be an accidental copy, the constructed reconciliation is four input records minus one documented removal equals three retained records. That calculation is conditional on the decision; the browser tool does not perform or authorize the removal.
Recount unresolved keys and width mismatches after the transformation. Compare required field values before and after. Never claim a new distinct-person total just because a spreadsheet’s Remove Duplicates command produced fewer rows. The command applies selected column equality, not a real-world identity investigation.
Publish a limited result with a reversible trail
Keep the source, decision log and working derivative linked by private references or digests. When a later correction arrives, you should be able to recover the original candidate rather than reconstructing it from memory. Avoid overwriting the only copy while cleaning.
The downloaded report contains summary counts and numeric positions, with only the first 50 positions for applicable issue lists. It does not include a full private duplicate ledger. Its 1 MiB and 5,000-record limits are processing bounds, not recommendations for treating a large archive as a representative sample.
Sources & product references
Product details reflect the publication date. External features and interfaces may change.
