Remove Duplicate Lines – Delete Repeated Lines from a List

How to use the duplicate line remover

1. Paste your list into the box, one item per line, or press Try an example.
2. Choose your options:
   • Case-sensitive — treat “Apple” and “apple” as different.
   • Ignore spaces at the start and end of lines — treat “ apple” and “apple” as the same.
   • Remove empty lines — drop blank lines too.
3. Copy the result, download it as a text file, or press Use as input to keep editing.

The status line tells you how many lines were removed and how many remain.

Example

This list:

apple
banana
Apple
cherry
banana

date
cherry

becomes:

apple
banana
cherry
date

Four lines are removed: the second “Apple” (case ignored), the second “banana”, the empty line and the second “cherry”.

Common uses

• Email lists — remove repeated addresses before importing them into a mailing tool.
• Keyword research — combine keyword lists from several sources and remove overlaps.
• Product codes and SKUs — check a list for repeated entries.
• URLs and links — clean up crawled or exported link lists.
• Names and attendees — merge sign-up lists from different forms.
• Log files — reduce repeated error lines to see the unique ones.

Tips

• Normalise first for messy data. If some lines have extra spaces in the middle, remove extra spaces before removing duplicates.
• Sort afterwards if you want an alphabetical list — use sort lines.
• Email addresses are case-insensitive in practice, so leave “Case-sensitive” off when deduplicating emails.
• Check what was removed by comparing the before and after with text compare.

How duplicates are detected

Each line is compared with every line kept so far. With the default options, lines are trimmed of spaces at the ends and compared in lowercase, so “ Apple”, “apple” and “APPLE ” all count as the same item. Only the first one stays, in the position where it first appeared. Lines that differ in the middle — such as “New York” and “New York” with two spaces — are treated as different, which is why cleaning the text first can catch more duplicates.

If you need to know which lines were duplicated rather than simply delete them, compare the original and cleaned lists with text compare: every removed line is highlighted.

Duplicates in spreadsheets

When the duplicates are whole rows in a spreadsheet, export the sheet as CSV, paste it here and remove duplicate lines — each row is one line. To keep only the unique values of one column, paste just that column. Re-import the result into Excel or Google Sheets afterwards.

Privacy

Lists of emails, customers or phone numbers are personal data. This tool processes them in your browser without uploading anything, which helps you stay within data protection rules such as the GDPR.

For more involved cleaning, use find and replace to standardise text, or convert lists to and from spreadsheets with CSV to JSON.