TL;DR: Use grapheme boundaries for many user-facing editing moves, the runtime's actual string indices for local APIs, and explicit units and document versions for stored positions. Keep normalization, layout and composition separate. A cursor position becomes reliable when both its unit and the source text it refers to are known.
A label builder presents a one-character counter beside a flag. Its database export says the flag used eight bytes. A browser script records an index change of four. The person editing the label presses Backspace once and removes the entire flag. Each observation answers a different question. A system that calls them all character counts will eventually split a selection across incompatible units.
The specimen cabinet below uses small illustrative strings. Unicode Standard Annex 29 supplies default text segmentation rules, including extended grapheme clusters. A grapheme cluster often corresponds to a unit a reader treats as one character. Languages, editing tasks and visual layout can call for further conventions. Unicode Standard Annex 15 specifies normalization forms, another choice with its own purpose. Unicode text segmentation, Unicode normalization.
Drawer one holds a flag
A rendered Canadian flag can be represented by two regional indicator code points. Each is encoded as two code units in UTF-16 and four bytes in UTF-8. Together they occupy four UTF-16 code units and eight UTF-8 bytes. Under the Unicode default segmentation rules this pair forms one extended grapheme cluster. Two code points can therefore be a faithful count as well as an unsuitable answer to how many backspaces a product should need.
The contract begins by naming the action. The label builder's caret can advance by a grapheme cluster during logical movement. A JavaScript substring operation uses UTF-16 code unit positions. A backend that reports source locations using UTF-8 offsets counts bytes. The displayed glyph depends on fonts and layout. An observer can reconcile all the numbers by looking at the exact string and the meaning of each boundary.
Starting with an empty label, put an ordinary letter before that flag and another letter after it. An editable range covering the flag starts after one UTF-16 unit and ends after five. Using byte offsets, it starts at one and ends at nine if the leading letter is ASCII. Its logical cluster positions go from one to two using zero-based boundaries. The terminating positions all describe the same boundary in that particular string. A stored integer without its unit could accidentally point inside a different element.
Imagine the software takes every UTF-16 unit as a caret stop. It can offer an index between the two halves of one regional indicator and another between the two indicators. A range API may represent those numbers even when the layout engine gives them no useful distinction. The editor should derive its intended movement from valid application boundaries and map them to the API's coordinate system. Clamping a numeric index to the string's total length will leave interior invalid positions possible.
Unicode's default regional indicator rules also care about context across a run. A greedy pairing beginning at an arbitrary chunk boundary can disagree with pairing for the complete string. A large editor using a piece table or rope needs segmentation that crosses chunks or preserves useful boundary state. Unit tests with a flag fully inside one short buffer will miss the defect that occurs when an insertion makes a pairing cross a storage split.
The flag also makes a clear acceptance example for selection. The editor may highlight an entire rendered cluster using a selection with two logical endpoints. Copying it should retain its constituent characters in the original sequence. A backend slicing raw bytes based on display counts can produce an invalid byte sequence or return part of the next item. Keeping logical positions and transport offsets related through the exact buffer prevents the copied selection from depending on accidental alignment.
Drawer two contains two ways to type a heading
One contributor enters a heading using a precomposed A with a ring. Another enters a Latin A followed by a combining ring above it. The sequences can represent canonically equivalent text, and each can form one extended grapheme cluster in that context. The precomposed form takes one UTF-16 code unit. The decomposed form takes two. Both facts matter before a document store chooses its normalized form.
An editing system might retain the originally entered form, or consistently normalize at declared boundaries. An importer that normalizes as it reads must check any incoming highlights, comments or diagnostics. An index marking the position after the decomposed pair is too far forward in the normalized representation unless it is translated. Normalization can preserve a human-readable word while changing the lengths that an API uses.
Suppose a selection covers the first two visible clusters in a decomposed four-letter title. A normalization routine may compact the first cluster. The second cluster's start moves in code units even though the title still looks the same. The editor can reproject remembered logical boundaries for that exact transformation, or store offsets only after normalization and retain a source mapping when importing annotations. A warning annotation from the original file should appear over the same intended word in its normalized view.
General replacement is more complex than normalization. When an edit deletes the entire selected phrase, there is no unique identical phrase to point to in the new revision. A comment anchor might move to the nearest surviving text according to a policy, or enter an unresolved state for the reviewer. Unit conversion cannot recover deleted context. The storage contract states what version matched the initial anchor and what happened to it afterward.
Even normalization at write time needs attention to the boundary of the write. Combining marks may interact with neighboring text when concatenated. An engineering design should follow the standard's rules for the selected normalization form and verify the behavior of the library it uses. Normalizing two source fragments independently and simply appending them does not always establish that their concatenation is normalized. When text operations preserve fragment identities, the resulting boundary deserves a test.
One product chooses to store source text as entered and offer a normalized view for comparison. Another adopts a specific normalized form for accepted editing buffers. Selection and revision history can work in either design. The decisions that matter are where a transformation happens, which representation an external report describes, and how mappings are retained for the transformations the product promises to support.
Drawer three reveals a cluster that crosses visible parts
A person technologist emoji can be formed by a person code point, a zero width joiner, and a laptop code point. In a supported display those three code points can appear as one image. The default Unicode rules can group the sequence into one extended grapheme cluster. The exact visual is shaped by platform font and fallback. Without a suitable combined glyph, a device could display recognizable components from the same underlying text.
A word processor should not segment based on the count of bitmaps a font happens to draw. A ligature can render adjacent letters with one combined shape even though navigation needs separate logical opportunities according to editing conventions. A complex script can shape clusters using context outside an individual code point. The text model owns logical positions and the renderer maps them to visual locations. Coordinate disagreements can be debugged with that separation intact.
Look at deletion near the joiner. Removing one raw code point may transform the cluster into another visible sequence and leave an unexpected character. For an ordinary user-facing backward delete, the product can choose to remove the preceding whole cluster. An advanced text inspection mode can expose individual code points with labels and let an expert edit them. These are different operations serving different users. The ordinary editor's approval tests should not inherit the inspector's raw indexing.
A caret can have more than one visual location at a logical boundary. Bidirectional text and a line wrap create cases where the displayed placement depends on the layout's affinity or movement convention. Grapheme clustering creates eligible logical units. It leaves the renderer and editing system responsible for choosing how left and right arrows move on screen. Testing an Arabic phrase next to a Latin numeral requires the actual displayed order and platform conventions, in addition to a simple count of clusters.
The line breaks provide a third dimension. If a caption wraps, the point where one line ends and the next begins can share a text index. Clicking near the end of the first line or the start of the next may yield the same logical position with different visual affiliation. Persisting such a selection for restoration can require extra information or an explicit default. A unit conversion table alone cannot preserve the appearance of a caret after the page width changes.
Some interfaces offer a delete word action. Word segmentation is another granularity with different rules, and a locale-aware segmenter for words may use dictionaries or other language knowledge. A desktop product decides whether to follow its native editor controls or define its own consistent cross-platform behavior. Treat these separately from Backspace and insertion. Both operations can consume characters while their user expectations come from different conventions.
The field guide labels counts with purposes
A design brief says the title accepts a maximum of twenty characters. Engineering asks whether that limit protects layout, storage, reading effort or a downstream system's schema. Each purpose leads to a different test. Display width depends on fonts and available space. A storage byte budget concerns an encoding or protocol. A grapheme limit may resemble a consumer's intuitive count while requiring a versioned segmentation policy.
For a byte budget of twenty in UTF-8, two sample flags of eight bytes each fit alongside four ordinary single-byte letters. A third such flag cannot fit. An interface could still describe the title as consisting of six user-facing units under the model above. If storage is the controlling constraint, a screen can show remaining capacity in a helpful way rather than guarantee three additional visible characters regardless of their representation.
A pasted value can exceed either limit. A product can reject it with an actionable message or propose a shortened value. Truncating an encoded buffer on an arbitrary byte boundary risks malformed text. Truncating at a code point boundary can still split an intended cluster. Choosing the last complete eligible cluster that fits both constraints gives a verifiable operation if that is the product's policy. When the very first cluster exceeds the byte budget, the user needs the real reason progress stopped.
A combined constraint can be stated without insisting that every editor implement all limits. Suppose a form allows ten logical units and 48 UTF-8 bytes because a backend needs that bound. Distinguish the progress counter from the server's encoded validation, and keep consistent boundary handling between both. If the server receives text from another client, its own validation remains necessary. A user who can edit and save the same title on different devices should not have to guess which interpretation will win.
Counting requires care in documents that include invisible formatting and controls. Their purpose may be essential to reading even though they produce no independent printed mark. A system that strips everything it cannot see can corrupt intended text. A supported editor displays the result and its boundaries according to the established representation. A diagnostic mode may reveal invisible information when needed, with clear descriptions of why editing it affects the surrounding string.
Unicode algorithm versions also matter for long-lived data. A later segmenter can recognize new sequences differently. Persisting indices in an old count may create disagreement after an upgrade. Offsets into an immutable encoded revision remain mechanically identifiable as long as the encoding and revision are known. A restored user-facing caret can then be adjusted to the current valid boundary under a documented policy. The system can preserve old range evidence without asking new users to adopt an obsolete display behavior.
Measure an edit before and after the input method commits
A typing session can contain intermediate composition text. An input method may assemble a character through several keystrokes, show a temporary candidate, and then commit the final string. An app that normalizes or resegments and replaces the whole input during composition can interrupt that sequence. One sound application policy applies acceptance rules at the appropriate commit point and interacts with the platform's composition lifecycle.
Consider a card entry showing three visible segments before editing. During a replacement the input method produces a new tentative string. A preview counter that attempts to enforce a final byte constraint on every transitional state can block valid completion. The UI can reserve editing capacity and give final feedback after commit, subject to the platform and product design. The stored document reflects a deliberate accepted value, not fragments of a user's candidate list.
The same issue appears for voice and handwriting input. The editor may receive provisional wording and then a longer final correction. An undo action needs an appropriate transaction boundary so it restores a user-meaningful edit. A grapheme cluster is a unit of text segmentation and does not by itself identify which input changes should be grouped into one undo step. The undo manager needs the intended editing event history.
During an undo, the original serialized sequence should remain recoverable if the product promised exact text preservation. If the editor normalizes on commit, saving and restoring the normalized result may be exactly the intended behavior. A test exercising both precomposed and decomposed specimen strings ties storage, counter and displayed selection together. The program can record one logical action while preserving the exact stored data that its policy warrants.
Framework APIs expose useful machinery that still needs unit labels. ECMAScript internationalization defines a segmenter that can return grapheme segments. Its segment records use an index into the original string. That makes the segmenter a helpful bridge to string slicing in JavaScript. It is also why the count of records and the numeric index of the next record diverge around these specimens. Test the supported runtime's implementation rather than assuming every device carries identical segmentation data. ECMAScript Internationalization API for segmenters.
The browser stores a route through the tree
A rendered passage can divide one logical text buffer into several text nodes for emphasis, links and editing decorations. The DOM defines range boundary points using a node and an offset. An offset in the range's node and an offset in the whole article therefore answer different questions. An editor that inserts marks during re-rendering needs to map its model positions back to the new node boundaries instead of keeping a reference to a removed text node. DOM Standard for ranges and boundary points.
Consider a highlighted flag in a paragraph where the first letter is in one text node and the flag begins a second. The string model owns the paragraph and its permitted clusters. The rendered node contributes a local offset at the start of that second node. After a style change, recreating the paragraph can merge the flag into a larger text node. The logical selection has the same text and the old local offset can now point to the preceding letter.
The interface may also preserve the direction of selection. A reader who drags backward across a word can select the same extent as someone dragging forward. An editing action such as extending the current selection may use the active end. A persisted range with numeric start and end captures extent. A user interface that promises restoration of continued manipulation may need an anchor and an active endpoint as well.
A diagnostic integration can test the invariant by applying a formatting change without changing the text, then restoring the selected cluster and extending the selection. The same apparent range should be available after new nodes render. An application mapping by the count of DOM nodes can fail when one node splits or merges. A versioned mapping between model and render keeps the browser's useful native behavior connected to the document's own semantics.
Put a revision number on the offset label
The storage contract can name a document id, revision, start, end, unit and encoding where needed. A range convention specifies whether the end is inclusive or excluded. The same numbers at a different revision can refer to entirely different words after a preceding insertion. Giving the receiving system enough evidence to reject stale or malformed positions prevents a visual highlight from quietly migrating.
A service can return annotations computed from imported UTF-8 source without forcing a browser to edit in bytes. Build a conversion for the exact accepted text. For a sparse set of annotations it can use indexed boundaries or compute local mappings. Large editors can maintain conversions as part of their text structure, with tests across chunks and changing prefix lengths. The interface should learn the correct conversion from documented contract values, rather than guessing based on the first test document.
When invalid data arrives, a useful status distinguishes an offset outside the source from one inside a valid encoded scalar but between accepted editing clusters. A repair operation might round endpoints outward for a purely visual diagnostic highlight. A destructive replacement can require exact eligible boundaries and the matched revision. These choices should follow the operation's promise. An automatic clamp that keeps the server happy may have changed the selection's meaning.
A product test can record the units at each layer for one complex string. It enters a flag, moves past it, selects it, copies it, exports a record, and restores a saved cursor into the matching revision. It also changes a preceding title while the annotation is stored. The expected results identify which offsets can be projected and which need a stale-state response. The test is small enough to diagnose and broad enough to find mismatched assumptions.
An operations team can reproduce the same issue without keeping sensitive text. A minimal published specimen with the relevant code points can exercise a bug reported from private content. An internal trace may record the encoding, revision state and positions without retaining a user's message. A Unicode update test has its own known specimens so the new behavior can be compared under both runtime and application policies.
At the end of the label exercise the browser still reports four units and the archive still writes eight bytes for the two code points. The text editor now handles the display as one logical segment for its chosen movement and deletion behavior. The document record tells a reader precisely how its saved positions are addressed. With those facts on the drawer labels, each team can reproduce the same user action without debating what an unlabeled character count meant.