Unicode-aware counts, reading time and platform limits
Duplicate Word Remover
Remove consecutive repeated words (“the the”) or every repetition from text, with case folding and an exclusion list; supports Arabic and lists what was removed.
Workspace
Result
How the result is calculated
The text is split into words and separators with a Unicode expression (letters, marks and digits with hyphen and apostrophe). In “Consecutive” mode a word is removed when it equals the word right before it; in “All” mode every occurrence after the first is removed. Words are compared after NFC normalization with optional case folding, and words in the exclusion list are kept. The whitespace before a removed word is dropped and spacing before punctuation is tidied.
Limits:
Removal is textual: the tool does not know the language and may remove intentional repetition (“very very”, “had had”). “All” mode also removes common function words (the, of) unless excluded. Arabic diacritics make a word differ from its unvocalized form.
FAQ
Can I see what was removed?
Yes, the summary lists removed words with how many times each was removed.
How do I keep intentional repetition?
Add the word to the exclusion list.
