Unicode normalizer

What is Unicode normalization?

Unicode often gives you more than one way to write the same thing. The letter é can be a single code point, U+00E9, or it can be the letter e followed by a combining acute accent. Both display identically and both are correct, but they are different sequences of bytes — so a naive comparison says they are different strings, a search for one does not find the other, and a database with a unique constraint happily accepts both.

Normalization is the fix. It rewrites text into one of four defined forms so that anything meaning the same thing ends up spelled the same way. The forms are named NFC, NFD, NFKC and NFKD, and choosing between them is the part that actually matters.

The names encode two independent decisions. C means composed and D means decomposed — whether é ends up as one code point or two. The K, confusingly, means compatibility, and that is a different axis entirely: it controls whether characters that merely look similar get folded together. Getting the first wrong costs you nothing a reader can see. Getting the second wrong destroys information.

How to use it

1. Paste any text into the box. All four forms are computed as you type. Each one is labelled canonical or compatibility, and marked when it differs from what you typed.
2. Compare the three counts underneath each form. UTF-16 units are what a string length reports, code points are what Unicode counts, and characters are what a reader counts. The same word can give three different answers, and NFC and NFD disagree about the first two while meaning the same text.
3. Look at what the accent-stripping snippet does. The bottom panel runs the standard remove-the-accents recipe and shows, character by character, which letters were folded to ASCII, which were left alone, and which vanished entirely.

Canonical is safe, compatibility is not

NFC and NFD are canonical forms. They rearrange without changing meaning: é as one code point and é as two are genuinely the same character, and converting between them is reversible in the sense that matters — NFC applied to the decomposed form gives exactly the composed one back. Applying either to arbitrary text is safe, and NFC is what you want almost always, because it is what the web, filenames and most databases expect.

NFKC and NFKD are compatibility forms, and they are a different kind of operation dressed in a similar name. They fold characters onto ones that merely resemble them. The superscript two in m² becomes an ordinary 2, so a square metre becomes m2. The ligature fi becomes two letters. The Roman numeral Ⅳ becomes the letters I and V. The trademark sign becomes TM. The fraction ½ becomes 1⁄2. Circled numbers become plain digits, and fullwidth Latin letters become ordinary ones.

None of that is a bug — it is the entire purpose. Compatibility folding exists so that a search for m2 finds m², and so that identifiers cannot be spoofed with lookalike characters. But it is one-way. Once you have folded a superscript into a digit you cannot tell whether the original was m² or m2, and if that distinction was carrying meaning it is gone. Use the compatibility forms for comparison keys and search indexes; do not store their output as though it were the user's text.

The measurement behind this is worth stating plainly. Over the 288,767 assigned code points, NFC changes 1,120 of them, NFD changes 13,233, NFKC changes 4,928 and NFKD changes 17,029. Every pair of forms disagrees with every other pair somewhere — NFC against NFKD on 15,930 code points alone. A single character can also grow dramatically: U+FDFA, an Arabic ligature, expands to eighteen characters under NFKD.

The accent-stripping snippet deletes real letters

Search for how to remove accents from a string and you will find the same three-step recipe everywhere: normalize to NFD, drop the combining marks, keep only ASCII. It works beautifully on the example everyone tests it with. café becomes cafe, naïve becomes naive, and the snippet gets copied into another codebase.

It is worth being precise about where it goes wrong, because the usual telling gets this backwards. The normalizing step deletes nothing at all — measured across every Latin letter, stripping combining marks after NFD removes exactly zero of them. The damage is done by the ASCII filter afterwards. A letter like ß, ø, ł, đ, ħ, þ or æ has no accent to strip in the first place; it survives normalization completely intact, and is then thrown away for the crime of not being in a to z.

Swept across all 907 Latin letters in Unicode, NFD followed by that filter folds 540 of them to ASCII and deletes 367 outright. Switching to NFKD improves it only slightly: 578 folded, 329 deleted, rescuing 38 letters that NFD loses because they have a compatibility decomposition where they have no canonical one.

In ordinary words the result is not subtle. Straße becomes Strae — not Strasse, which is the correct German transliteration, but a word with a letter simply missing. Łódź becomes odz. Đà Nẵng becomes aNang. And Ærø, a Danish island, becomes the single letter r, because two of its three letters are deleted and the survivor is an ASCII r that was never touched. If you need to fold text to ASCII, you need a transliteration table, not a normalization form.

Honest limits

The four forms are computed by your browser's own implementation of the standard, which is the same one your code will use. This page does not implement normalization itself; it shows you what the platform does and counts the consequences.

The accent-stripping demonstration reproduces the popular snippet deliberately, including its combining-mark ranges. It is there to show what that specific recipe does, not to be the best possible ASCII folder — a tool that quietly used a better algorithm would be arguing against something nobody wrote.

Normalization is not case folding, and it is not a security boundary on its own. NFKC is used as part of identifier comparison in several standards, but folding alone does not make two identifiers safe to treat as equal; confusable characters across scripts are a separate problem with its own Unicode data, and this page does not address it.

Finally, the figures quoted above are tied to a Unicode version. They were measured against the version Python's standard library ships with, cross-checked against your browser's implementation, and the script that produced them is committed alongside the site so the numbers can be regenerated rather than trusted.

Why is it free?

Because it costs nothing to run. Every form is computed in your browser as you type; nothing is uploaded, nothing is logged, and no server sees your text.

No account, no sign-up, nothing held back. The behaviour comes from the Unicode annex your browser already implements, and the measurements on this page are checked against a separate implementation in Python so the numbers are a comparison rather than a claim.