Unicode character property#Bidirectional writing
{{Short description|Unicode code point property names and their uses}}
{{Use British English|date=January 2025}}
The Unicode Standard assigns various properties to each Unicode character and code point.{{cite web|url=https://www.unicode.org/versions/Unicode16.0.0/core-spec/chapter-4/|date=September 2024|title=The Unicode Standard Version 16 |publisher=The Unicode Consortium |access-date=2024-09-13}}
The properties can be used to handle characters (code points) in processes, like in line-breaking, script direction right-to-left or applying controls. Some "character properties" are also defined for code points that have no character assigned and code points that are labelled like "<not a character>". The character properties are described in Standard Annex #44.{{cite web |url=https://www.unicode.org/reports/tr44/ |title=Unicode Standard Annex #44: Unicode Character Database |website=Unicode |date=2024-08-27}}
Properties have levels of forcefulness: normative, informative, contributory, or provisional. For simplicity of specification, a character property can be assigned by specifying a continuous range of code points that have the same property.{{cite web|url=https://www.unicode.org/reports/tr44/#Code_Point_Ranges|title=Unicode Standard Annex #44: Unicode Character Database, 4.2.3 Code Point Ranges|website=Unicode |date=2024-08-27}}
Semantic elements
Properties are displayed in the following order:[https://unicode.org/reports/tr44/#UnicodeData.txt UnicodeData.txt]
[code];[name];[gc];[cc];[bc];[decomposition];[nv-dec];[nv-dig];[nv-num];[bm];[alias];;[upper case];[lower case];[title case]
alias
= corrected name. Obsolete. Now tracked with a separate database, but remains for Unicode 1.0 names.bc
= bidi (bidirectional) category [L, R etc]bm
= bidi mirrored [N or Y]cc
= combining class [position of diacritic]decomposition
type or= letter + diacritic, ligature X Y, superscript X, font X, initial X, medial X, final X, isolated X, vertical X, etc. gc
= general category [letter, symbol, digit, punctuation, case behaviour, etc.]nv
= numeric type and value [of a digit]. If numeric type is 'decimal', all 3 slots are filled. If 'digit', the first will be null. (This has been discontinued.) If 'numeric', then the first two will be null and only the last will be used.
The property between alias
and upper case
is obsolete and is now null for all Unicode characters.
{{anchor|Name}}Name and alias
A Unicode character is assigned a unique Name (na). The name is composed of uppercase letters A–Z, digits 0–9, hyphen-minus and space. Some sequences are excluded: names beginning with a space or hyphen, names ending with a space or hyphen, repeated spaces or hyphens, and space after hyphen are not allowed. The name is guaranteed to be unique within Unicode, and can be used to identify a code point and its character. Ideographic characters, of which there are tens of thousands, are named in the pattern "{{Smallcaps|{{lc:CJK UNIFIED IDEOGRAPH}}}}-hhhh". For example, {{unichar|4E00}}. Formatting characters also have names: {{unichar|00A0}}.
The following Unicode categories do not have a Name value assigned: Controls (General Category: Cc), Private use (Co), Surrogate (Cs), Non-characters (Cn) and Reserved (Cn). They may be referenced, informally, by a generic or specific meta-name, called "Code Point Labels": {{not a typo|
={{anchor|Version 1.0 names}}Unicode 1.0 names=
In version 2.0 of Unicode, many names were changed. From then on the rule "a name will never change" came into effect, including the strict (normative) use of alias names. Disused Unicode 1.0 names were moved to the property Alias, to provide backward compatibility.
For example, {{Unichar|264}} has the Unicode 1.0 name "LATIN SMALL LETTER BABY GAMMA".
=Character name alias=
{{main|Unicode alias names and abbreviations}}
Starting from Unicode 2.0, the published name for a code point will never change. Therefore, in the event of a character name being misspelled or if the character name is completely wrong or seriously misleading, a formal Character Name Alias may be assigned to the character, and this alias may be used by applications instead of the actual defective character name. For example, {{unichar|FE18}} has the character name alias "{{sc2|PRESENTATION FORM FOR VERTICAL RIGHT WHITE LENTICULAR BRACKET}}" in order to mitigate the misspelling of "bracket" as {{notatypo|"brakcet"}} {{sic}} in the actual character name; {{unichar|A015|YI SYLLABLE WU}} has the character name alias {{sc2|"YI SYLLABLE ITERATION MARK"}} because, contrary to the character name, it does not have a fixed syllabic value.
In addition to character name aliases which are corrections to defective character names, some characters are assigned aliases which are alternative names or abbreviations. Five types of character name aliases are defined in the Unicode Standard:
- Correction: corrections for misspelled or seriously incorrect character names;
- Control: ISO 6429 names for C0 and C1 control functions (which are not assigned character names in the Unicode Standard);
- Alternate: alternative names for some format characters (only {{unichar|FEFF}} which has the alias {{sc2|"BYTE ORDER MARK"}});
- Figment: Documented labels for some C1 control code functions which are not actual names in any standard;
- Abbreviation: Abbreviations or acronyms for control codes, format characters, spaces, and variation selectors.
All formal character name aliases follow the rules for permissible character names, and are guaranteed to be unique within both the character name alias and the character name namespaces (for this reason, the ISO 6429 name "BELL" is not defined as an alias for {{unichar|0007}} because U+1F514 is named "BELL"; U+0007 instead has the alias "ALERT").
As of Unicode 16.0, 35 formal character name aliases are defined as corrections for defective character names.{{Cite web | url=https://www.unicode.org/Public/UCD/latest/ucd/NameAliases.txt | title=UCD: Name Aliases|publisher=Unicode Consortium | work=Unicode Character Database | date=2024-04-24}}
Apart from these normative names, informal names may be shown in the Unicode code charts. These are other commonly used names for a character, and do not have the same character restriction. These informal names are not guaranteed to be unique, and may be changed or removed in later versions of the standard.
General Category
Each code point is assigned a value for General Category. This is one of the character properties that are also defined for unassigned code points and code points that are defined "not a character".
{{General Category (Unicode)|state=expanded}}
=Punctuation=
Characters have separate properties to denote they are a punctuation character. The properties all have a Yes/No values: Dash, Quotation_Mark, Sentence_Terminal, Terminal_Punctuation. The Punctuation property refers to characters that are used to divide or structure text, and these are classified into different types based on their roles. Unicode assigns these punctuation characters specific categories.
{{main|Dash|Quotation mark glyphs#Quotation marks in Unicode|Terminal punctuation}}
=Whitespace=
{{main|Whitespace character}}
Whitespace is a commonly used concept for a typographic effect. Basically it covers invisible characters that have a spacing effect in rendered text. It includes spaces, tabs, and new line formatting controls. In Unicode, such a character has the property set WSpace=yes
. In version {{Unicode version|version=16.0}}, there are 25 whitespace characters.
{{Whitespace (Unicode)|state=collapsed}}
=Casing=
The Case value is normative in Unicode. It pertains to those scripts with uppercase and lowercase letters. Case-difference occurs in Adlam, Armenian, Cherokee, Coptic, Cyrillic, Deseret, Garay, Glagolitic, Greek, Khutsuri and Mkhedruli Georgian, Latin, Medefaidrin, Old Hungarian, Osage, Vithkuqi and Warang Citi scripts.
Different languages have different case mapping rules.
In Turkish, {{Unichar|0069}} corresponds to {{Unichar|0130}} instead of {{Unichar|0049}}. Similarly, {{Unichar|0049}} when corresponds to {{Unichar|0131}} instead of {{Unichar|0069}}.
In Nawdm, the letter Ĥ corresponds to ɦ in lowercase instead of the usual case mappings being Ĥĥ and Ɦɦ.
In Greek, the letter sigma has different lowercase forms depending on where it is in a word. {{Unichar|03a3}} converts to {{Unichar|03c3}} if it is at the start or middle of a word, and converts to {{Unichar|03c2}} if it is at the end of a word.
In Lithuanian, the dot in lowercase i and j is preserved when followed by accents. For example: Í in lowercase is i̇́.[http://ftp.unicode.org/Public/UNIDATA/SpecialCasing.txt]
Despite the existence of {{Unichar|1E9E}}, {{Unichar|00DF}} corresponds to "SS".
Unicode encodes 31 titlecase characters.
- {{Unichar|01C5}}
- {{Unichar|01C8}}
- {{Unichar|01CB}}
- {{Unichar|01F2}}
- {{Unichar|1F88}}
- {{Unichar|1F89}}
- {{Unichar|1F8A}}
- {{Unichar|1F8B}}
- {{Unichar|1F8C}}
- {{Unichar|1F8D}}
- {{Unichar|1F8E}}
- {{Unichar|1F8F}}
- {{Unichar|1F98}}
- {{Unichar|1F99}}
- {{Unichar|1F9A}}
- {{Unichar|1F9B}}
- {{Unichar|1F9C}}
- {{Unichar|1F9D}}
- {{Unichar|1F9E}}
- {{Unichar|1F9F}}
- {{Unichar|1FA8}}
- {{Unichar|1FA9}}
- {{Unichar|1FAA}}
- {{Unichar|1FAB}}
- {{Unichar|1FAC}}
- {{Unichar|1FAD}}
- {{Unichar|1FAE}}
- {{Unichar|1FAF}}
- {{Unichar|1FBC}}
- {{Unichar|1FCC}}
- {{Unichar|1FFC}}
{{expand section|date=March 2022}}
=Other general characteristics=
Ideographic, alphabetic, noncharacter.
{{expand section|date=March 2022}}
Combining class
{{more|Combining character}}
Some common codes:
:0 = spacing letter, symbol or modifier (e.g. {{Char|a}}, {{Char|(}}, {{Char|ʰ}})
:1 = overlay
:6 = Han reading (CJK diacritic reading marks)
:7 = nukta (diacritic nukta in Brahmic scripts)
:8 = kana voicing marks
:9 = virama
10–199 = various fixed-position classes
Marks which attach to the base letter:
:200 = attached at bottom left
:202 = attached directly below (e.g. cedilla on ç)
:204 = attached at bottom right
:208 = attached to left
:210 = attached to right
:212 = attached to top left
:214 = attached directly above
:216 = attached at top right
Marks which do not attach to the base letter:
:218 = bottom left
:220 = directly below (e.g. ring on n̥)
:222 = below right
:224 = left
:226 = right
:228 = above left
:230 = above (e.g. acute accent on á)
:232 = above right
:233 = double below (subtends two bases)
:234 = double above (extends two bases)
:240 = iota subscript (only that Greek diacritic)
Bidirectional writing
Six character properties pertain to bi-directional writing: Bidi_Class, Bidi_Control, Bidi_Mirrored, Bidi_Mirroring_Glyph, Bidi_Paired_Bracket and Bidi_Paired_Bracket_Type.
One of Unicode's major features is support of bi-directional (Bidi) text display right-to-left (R-to-L) and left-to-right (L-to-R). The Unicode Bidirectional Algorithm UAX9{{cite web|url=https://www.unicode.org/reports/tr9/ |title=Unicode Standard Annex #9: Unicode Bidirectional Algorithm|work=The Unicode Standard|date=2024-09-02}} describes the process of presenting text with altering script directions. For example, it enables a Hebrew quote in an English text. The Bidi_Character_Type marks a character's behaviour in directional writing. To override a direction, Unicode has defined special formatting control characters (Bidi-Control characters). These characters can enforce a direction, and by definition only affect bi-directional writing.
Each code point has a property called Bidi_Class. It defines its behaviour in a bidirectional text as interpreted by the algorithm:
{{Bidi Class (Unicode)}}
In normal situations, the algorithm can determine the direction of a text by this character property. To control more complex Bidi situations, e.g. when an English text has a Hebrew quote, extra options are added to Unicode. 12 characters have the property {{code|1=Bidi_Control=Yes}}: ALM, FSI, LRE, LRI, LRM, LRO, PDF, PDI, RLE, RLI, RLM and RLO as named in the table. These are invisible formatting control characters, only used by the algorithm and with no effect outside of bidirectional formatting. Despite the name, they are formatting characters, not control characters, and have General category Other, format (Cf) in the Unicode definition.
Basically, the algorithm determines a sequence of characters with the same strong direction type (R-to-L or L-to-R), taking in account an overruling by the special Bidi-controls. Number strings (Weak types) are assigned a direction according to their strong environment, as are Neutral characters. Finally, the characters are displayed per a string's direction.
Two character properties are relevant to determining a mirror image of a glyph in bidirectional text: {{code|1=Bidi_Mirrored=Yes}} indicates that the glyph should be mirrored when written R-to-L. The property {{code|1=Bidi_Mirroring_Glyph=U+hhhh}} can then point to the mirrored character. For example, parentheses {{char|(}}, {{char|)}} are mirrored this way. Shaping cursive scripts such as Arabic, and mirroring glyphs that have a direction, is not part of the algorithm.
Numeric values and types
=Decimal=
Characters are classified with a Numeric type. Characters such as fractions, subscripts, superscripts, Roman numerals, currency numerators, encircled numbers, and script-specific digits are type Numeric. They have a numeric value that can be decimal, including zero and negatives, or a vulgar fraction. If there is not such a value, as with most of the characters, the numeric type is "None".
The characters that do have a numeric value are separated in three groups: Decimal (De), Digit (Di) and Numeric (Nu, i.e. all other). "Decimal" means the character is a straight decimal digit. Only characters that are part of a contiguous encoded range 0..9 have numeric type Decimal. Other digits, like superscripts, have numeric type Digit. All numeric characters like fractions and Roman numerals end up with the type "Numeric". The intended effect is that a simple parser can use these decimal numeric values, without being distracted by say a numeric superscript or a fraction. Eighty-three CJK Ideographs that represent a number, including those used for accounting, are typed Numeric.
On the other hand, characters that could have a numeric value as a second meaning are still marked Numeric type None, and have no numeric value. E.g. Latin letters can be used in paragraph numbering like "II.A.1.b", but the letters "I", "A" and "b" are not numeric (type None) and have no numeric value.
{{Numeric Type (Unicode)}}
=Hexadecimal digits=
Hexadecimal characters are those in the series with hexadecimal values 0123456789ABCDEF (sixteen characters, decimal value 0–15). The character property Hex_Digit is set to Yes when a character is in such a series:
{{Hexadecimal digit (Unicode)}}
Forty-four characters are marked as Hex_Digit. The ones in the Basic Latin block are also marked as ASCII_Hex_Digit.
Unicode has no separate characters for hexadecimal values. A consequence is, that when using regular characters it is not possible to determine whether hexadecimal value is intended, or even whether a value is intended at all. That should be determined at a higher level, e.g. by prepending 0x to a hexadecimal number or by context. The only feature is that Unicode can note that a sequence can or can not be a hexadecimal value.
Block
{{main|Unicode block}}
A block is a uniquely named, contiguous range of code points. It is identified by its first and last code point. Blocks do not overlap, nor do they extend across planes. The number of code points in each block must be a multiple of 16. A block may contain code points that are reserved, not-assigned, etc. Each character that is assigned, has a single "block name" value from the 338 names assigned as of Unicode version {{Unicode version|version=16.0}}. Unassigned code points outside of an existing block have the default value "No_block".
{{Unicode blocks|state=mw-collapsed}}
Script
{{main|Scripts in Unicode}}
Each assigned character can have a single value for its "Script" property, signifying to which script it belongs.{{cite web|url=https://www.unicode.org/reports/tr24/|title=Unicode Standard Annex #24: Unicode Script Property|work=The Unicode Standard|date=2024-07-31}} The value is a four-letter code in the range Aaaa-Zzzz, as available in ISO 15924, which is mapped to a writing system. Apart from when describing the background and usage of a script, Unicode does not use a connection between a script and languages that use that script. So "Hebrew" refers to the Hebrew script, not to the Hebrew language.
The special code Zyyy for "Common" allows a single value for a character that is used in multiple scripts. The code Zinh "Inherited script", used for combining characters and certain other special-purpose code points, indicates that a character "inherits" its script identity from the character with which it is combined. (Unicode formerly used the private code Qaai for this purpose.) The code Zzzz "Unknown" is used for all characters that do not belong to a script (i.e. the default value), such as symbols and formatting characters. Overall, characters of a single script can be scattered over multiple blocks, like Latin characters. And the other way around too: multiple scripts can be present is a single block, e.g. block Letterlike Symbols contains characters from the Latin, Greek and Common scripts.
When the Script is "" (blank), according to Unicode the character does not belong to a script. This pertains to symbols, because the existing ISO script codes "Zmth" (Mathematical notation), "Zsym" (Symbol), and "Zsye" (Symbol, emoji variant) are not used in Unicode. The "Script" property is also blank for code points that are not a typographic character like controls, substitutes, and private use code points.
If there is a specific script alias name in ISO 15924, it is used in the character name: {{unichar|0041|Latin capital letter A}}, and {{unichar|05D0|Hebrew letter alef}}.
{{ISO 15924 script codes and related Unicode data|state=mw-collapsed}}
Normalization properties
Decompositions, decomposition type, canonical combining class, composition exclusions, and more.
{{expand section|date=March 2022}}
Age
Age is the version of the standard in which the code point was first designated. The version number is shortened to the numbering major.minor', although there more detailed version numbers are used: versions 4.0.0 and 4.0.1 both are named 4.0 as Age. Given the releases, Age can be from the range: 1.1, 2.0, 2.1, 3.0, 3.1, 3.2, 4.0, 4.1, 5.0, 5.1, 5.2, 6.0, 6.1, 6.2, 6.3, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0, 12.1, 13.0, 14.0, 15.0, 15.1, and 16.0.{{Cite web|url=https://www.unicode.org/Public/UCD/latest/ucd/DerivedAge.txt|title=UCD: Derived Age|publisher=Unicode Consortium|work=Unicode Character Database|date=2024-04-30}} The long values for Age begin in a V and use an underscore instead of a dot: V1_1, for example. Codepoints without a specifically assigned age value have the value "NA", with the long form "Unassigned".
Deprecated
Once a character has been defined, it will not be removed or reassigned.{{cite web |url=https://www.unicode.org/policies/stability_policy.html |title=Unicode Character Encoding Stability Policies |website=Unicode |date=2024-01-09 |access-date=2024-01-13 |publisher=Unicode Consortium |quote=Once a character is encoded, it will not be moved or removed.}} However, a character may be deprecated, meaning its "use is strongly discouraged".{{cite web|title=The Unicode Standard, D13 Deprecated character |url=https://www.unicode.org/versions/Unicode16.0.0/core-spec/chapter-3/#G48383 |date=2024 |access-date=2024-09-13 }} As of Unicode version {{Unicode version|version=16.0}}, the following fifteen characters are deprecated:{{cite web |url=https://www.unicode.org/Public/16.0.0/ucd/PropList.txt |title=PropList-16.0.0.txt |website=Unicode |date=2024-05-31 |access-date=2024-09-13 |publisher=Unicode Consortium}}
class="wikitable sortable mw-collapsible {{{state|mw-uncollapsed}}}" style="margin:0"
! colspan=5 style="border:1px solid grey; line-height:1.75em;" |Deprecated characters in Unicode |
Codepoint
!Character name !colspan=2|Recommended alternative !Remarks |
---|
U+0149
|{{unichar/name|na=LATIN SMALL LETTER N PRECEDED BY APOSTROPHE}} |U+02BC U+006E |ʼn | |
U+0673
|{{unichar/name|na=ARABIC LETTER ALEF WITH WAVY HAMZA BELOW}} |U+0627 U+065F |اٟ | |
U+0F77
|{{unichar/name|na=TIBETAN VOWEL SIGN VOCALIC RR}} |U+0FB2 U+0F81{{efn|name=Depr01|{{unichar|0F81|TIBETAN VOWEL SIGN REVERSED II}} is itself discouraged (but not deprecated), and is canonically equivalent to the sequence U+0F71 U+0F80.}} |ྲཱྀ | |
U+0F79
|{{unichar/name|na=TIBETAN VOWEL SIGN VOCALIC LL}} |U+0FB3 U+0F81{{efn|name=Depr01}} |ླཱྀ | |
U+17A3
|{{unichar/name|na=KHMER INDEPENDENT VOWEL QAQ}} |U+17A2 |អ | |
U+17A4
|{{unichar/name|na=KHMER INDEPENDENT VOWEL QAA}} |U+17A2 U+17B6 |អា | |
U+206A
|{{unichar/name|na=INHIBIT SYMMETRIC SWAPPING}} |colspan=2|None{{efn|name=Depr02|Rather than using this control character to indicate the appropriate appearance for text, appropriate character codes with the correct state should be used.{{cite web|url=https://www.unicode.org/versions/Unicode16.0.0/core-spec/chapter-23/#G19593 |title=Chapter 23.3: Deprecated Format Characters |date=2024 |access-date=2024-09-13}}}} | |
U+206B
|{{unichar/name|na=ACTIVATE SYMMETRIC SWAPPING}} |colspan=2|None{{efn|name=Depr02}} | |
U+206C
|{{unichar/name|na=INHIBIT ARABIC FORM SHAPING}} |colspan=2|None{{efn|name=Depr02}} | |
U+206D
|{{unichar/name|na=ACTIVATE ARABIC FORM SHAPING}} |colspan=2|None{{efn|name=Depr02}} | |
U+206E
|{{unichar/name|na=NATIONAL DIGIT SHAPES}} |colspan=2|None{{efn|name=Depr02}} | |
U+206F
|{{unichar/name|na=NOMINAL DIGIT SHAPES}} |colspan=2|None{{efn|name=Depr02}} | |
U+2329
|{{unichar/name|na=LEFT-POINTING ANGLE BRACKET}} |U+3008{{efn|name=Depr03|This alternative character is in the CJK Symbols and Punctuation block, and is not suitable for mathematical or technical use}} |〈 |{{unichar|27E8|MATHEMATICAL LEFT ANGLE BRACKET}} is recommended for mathematical and other technical use |
U+232A
|{{unichar/name|na=RIGHT-POINTING ANGLE BRACKET}} |U+3009{{efn|name=Depr03}} |〉 |{{unichar|27E9|MATHEMATICAL RIGHT ANGLE BRACKET}} is recommended for mathematical and other technical use |
U+E0001
|{{unichar/name|na=LANGUAGE TAG}} |colspan=2|None{{efn|name=Depr04|Alternative means of language tagging should be used instead.{{cite web |url=https://www.unicode.org/versions/Unicode16.0.0/core-spec/chapter-23/#G30427 |title=23.9: Tag Characters, Deprecated Use for Language Tagging |date=2024 |access-date=2024-09-13}}}} | |
class="sortbottom"
|colspan=5|{{notelist}} |
Boundaries
The Unicode Standard specifies the following boundary-related properties:
- Grapheme cluster
- Word
- Line
- Sentence
{{expand section|date=January 2025}}
Alias name
{{main|Unicode alias names and abbreviations}}Unicode can assign alias names to code points. These names are unique over all names (including regular ones), so they can be used as identifier. There are five possible reasons to add an alias:
;1. Abbreviation
:Commonly occurring abbreviations or acronyms for control codes, format characters, spaces, and variation selectors.
:For example, {{unichar|00A0|no-break space}} has alias {{smallcaps|NBSP}}. Sometimes presented in a box: {{Unicode alias/abbrbox|abbr=NBSP|title=no-break space}}.
;2. Control
:ISO 6429 names for C0 and C1 control functions and similar commonly occurring names, are added as an alias to the character.
:For example, {{unichar|0008}} has the alias {{smallcaps2|BACKSPACE}}.
;3. Correction
:This is a correction for a "serious problem" in the primary character name, usually an error.
:For example, {{unichar|2118}} is actually a lowercase p, and so is given alias name {{smallcaps2|1=WEIERSTRASS ELLIPTIC FUNCTION}}: "actually this has the form of a lowercase calligraphic p, despite its name, and through the alias the correct spelling is added." In descriptions, with preceding symbol ※.
;4. Alternate
:A widely used alternate name for a character.
:Example: {{unichar|FEFF|ZERO WIDTH NO-BREAK SPACE}} has the alternate alias {{smallcaps2|1=BYTE ORDER MARK}}.
;5. Figment
:Several documented labels for C1 control code points which were never actually approved in any standard (figment meaning "feigned, in fiction").
:For example, {{unichar|0099}} has the figment alias {{smallcaps2|1=SINGLE GRAPHIC CHARACTER INTRODUCER}}. This name is an architectural concept from early drafts of ISO/IEC 10646-1, but it was never approved or standardized.
External links
- [https://www.unicode.org/reports/tr44/ Unicode Character Database], annex #44, explaining the different properties
- [https://www.unicode.org/Public/UCD/latest/ucd/UnicodeData.txt UnicodeData.txt] – a list of all Unicode characters, with their properties