EBCDIC

File:Blue-punch-card-front-horiz top-char-contrast-stretched.png with the Hollerith encoding of the 1964 EBCDIC character set. Contrast at the top is enhanced to show the printed characters. The "number" punches (0–9) directly translate to the lower 4 bits of EBCDIC, though the upper 4 bits of EBCDIC are more complex.]]

EBCDIC was devised in 1963 and 1964 by IBM and was announced with the release of the IBM System/360 line of mainframe computers. It is an eight-bit character encoding, developed separately from the seven-bit ASCII encoding scheme. It was created to extend the existing Binary-Coded Decimal (BCD) Interchange Code, or BCDIC, which itself was devised as an efficient means of encoding the two zone and number punches on punched cards into six bits. The distinct encoding of 's' and 'S' (using position 2 instead of 1) was maintained from punched cards where it was desirable not to have hole punches too close to each other to ensure the integrity of the physical card.{{Cite web |title=Doug Jones's punched card codes |url=http://homepage.cs.uiowa.edu/~jones/cards/codes.html |access-date=2023-01-14 |website=homepage.cs.uiowa.edu}}{{failed verification|date=January 2023}}

While IBM was a chief proponent of the ASCII standardization committee, the company did not have time to prepare ASCII peripherals (such as card punch machines) to ship with its System/360 computers, so the company settled on EBCDIC. The System/360 became wildly successful, together with clones such as RCA Spectra 70, ICL System 4, and Fujitsu FACOM, thus so did EBCDIC.

All IBM's mainframe operating systems, and its IBM i operating system for midrange computers, use EBCDIC as their inherent encoding (with toleration for ASCII, for example, ISPF in z/OS can browse and edit both EBCDIC and ASCII encoded files). Software can translate to and from encodings, and modern mainframes (such as IBM Z) include processor instructions, at the hardware level, to accelerate translation between character sets. Modern z/OS compilers for the C and C++ languages on IBM Z mainframes, and earlier OS/390 C and C++ compilers on IBM System/390 mainframes, support a POSIX-compatible execution environment that makes use of ASCII by default.{{cite web|url=https://www.ibm.com/docs/en/zos/3.1.0?topic=pages-enhanced-ascii|title=Enhanced ASCII|work=z/OS UNIX System Services Planning|date=2024-08-28}}

Not all operating systems running on IBM hardware use EBCDIC; IBM AIX, Linux on IBM Z, and Linux on Power all use ASCII, as do all operating systems that run on the IBM Personal Computer and its successors.

{{More citations needed section|date=November 2022}}

There were numerous difficulties to writing software that would work in both ASCII and EBCDIC.

• The gaps between letters made simple code that worked in ASCII fail on EBCDIC. For example {{code|1=for (c = 'A'; c <= 'Z'; ++c) putchar(c);|lang=c}} would print the alphabet from A to Z if ASCII is used, but print 41 characters (including a number of unassigned ones) in EBCDIC.
• Sorting EBCDIC put lowercase letters before uppercase letters and letters before numbers, exactly the opposite of ASCII.
• Most programming languages and file formats and network protocols designed for ASCII used available punctuation marks (such as the curly braces {{char|{{(}}}} and {{char|{{)}}}}) that did not exist in EBCDIC, making translation to EBCDIC systems difficult. Workarounds such as trigraphs were used.{{cite web|title=Rationale for International Standard – Programming Languages – C|version=Revision 5.10|date=April 2003|url=http://www.open-std.org/jtc1/sc22/wg14/www/C99RationaleV5.10.pdf#page=204|at=§ MSE.4: Support for invariant ISO/IEC 646|access-date=2022-11-24 |url-status=live|archive-url=https://web.archive.org/web/20160606072228/http://www.open-std.org/jtc1/sc22/wg14/www/C99RationaleV5.10.pdf|archive-date=2016-06-06}} Conversely EBCDIC had a few characters such as {{char|¢}} (US cent) that were used on IBM systems and could not be translated to ASCII.
• The most common newline convention used with EBCDIC is to use a NEL (NEXT LINE) code between lines. Converters to other encodings often replace NEL with LF or CR/LF, even if there is a NEL in the target encoding. This causes the LF and NEL to translate to the same character and be unable to be distinguished.
• If seven-bit ASCII was used, there was an "unused" high bit in 8-bit bytes, and many pieces of software stored other information there. Software would also pack the seven bits and discard the eighth, such as packing five seven-bit ASCII characters in a 36-bit word.{{cite book|url=http://www.bitsavers.org/pdf/dec/pdp10/1970_PDP-10_Ref/1970PDP10Ref_Part2.pdf|title=PDP-10 Reference Handbook, Book 2: Assembling the Source Program|page=221|publisher=Digital Equipment Corporation}} On the PDP-11, bytes with the high bit set were treated as negative numbers, behavior that was copied to C, causing unexpected problems if the high bit was set. These all made it difficult to switch from ASCII to the 8-bit EBCDIC (and also made it difficult to switch to 8-bit extended ASCII encodings).

{{Further|Code page#EBCDIC-based code_pages|l1=EBCDIC code pages}}

There are hundreds of EBCDIC code pages based on the original EBCDIC character encoding; there are a variety of EBCDIC code pages intended for use in different parts of the world, including code pages for non-Latin scripts such as Chinese, Japanese (e.g., EBCDIC 930, JEF, and KEIS), Korean, and Greek (EBCDIC 875). There is also a huge number of variations with the letters swapped around for no discernible reason.{{citation needed|date=December 2022}}

The table below shows the "invariant subset"{{cite web| url = https://www.ibm.com/support/knowledgecenter/en/ssw_ibm_i_71/nls/rbagsinvariantcharset.htm| website = IBM Knowledge Center | title = Invariant character set| date = 14 August 2018 }} of EBCDIC, which are characters that should have the same assignments on all EBCDIC code pages that use the Latin alphabet. (This includes most of the ISO/IEC 646 invariant repertoire, except the exclamation mark.) It also shows (in gray) missing ASCII and EBCDIC punctuation, located where they are in Code Page 37 (one of the code page variants of EBCDIC). The blank cells are filled with region-specific characters in the variants, but the characters in gray are often swapped around or replaced as well. Like ASCII, the invariant subset works only for languages using only the ISO basic Latin alphabet, such as English.

|{{chset-cell1|U+004A LATIN CAPITAL LETTER J|J}}

|{{chset-cell1|U+004B LATIN CAPITAL LETTER K|K}}

|{{chset-cell1|U+004C LATIN CAPITAL LETTER L|L}}

|{{chset-cell1|U+004D LATIN CAPITAL LETTER M|M}}

|{{chset-cell1|U+004E LATIN CAPITAL LETTER N|N}}

|{{chset-cell1|U+004F LATIN CAPITAL LETTER O|O}}

|{{chset-cell1|U+0050 LATIN CAPITAL LETTER P|P}}

|{{chset-cell1|U+0051 LATIN CAPITAL LETTER Q|Q}}

|{{chset-cell1|U+0052 LATIN CAPITAL LETTER R|R}}

|{{chset-cell1|||style=background:#DDD}}

|{{chset-cell1|||style=background:#DDD}}

|{{chset-cell1|||style=background:#DDD}}

|{{chset-cell1|||style=background:#DDD}}

|{{chset-cell1|||style=background:#DDD}}

|{{chset-cell1|||style=background:#DDD}}

|-

|{{chset-left1|Ex}}

|{{chset-cell1|U+005C REVERSE SOLIDUS|\|224|style=background:#DDD}}

|{{chset-cell1|||style=background:#DDD}}

|{{chset-cell1|U+0053 LATIN CAPITAL LETTER S|S}}

|{{chset-cell1|U+0054 LATIN CAPITAL LETTER T|T}}

|{{chset-cell1|U+0055 LATIN CAPITAL LETTER U|U}}

|{{chset-cell1|U+0056 LATIN CAPITAL LETTER V|V}}

|{{chset-cell1|U+0057 LATIN CAPITAL LETTER W|W}}

|{{chset-cell1|U+0058 LATIN CAPITAL LETTER X|X}}

|{{chset-cell1|U+0059 LATIN CAPITAL LETTER Y|Y}}

|{{chset-cell1|U+005A LATIN CAPITAL LETTER Z|Z}}

|{{chset-cell1|||style=background:#DDD}}

|{{chset-cell1|||style=background:#DDD}}

|{{chset-cell1|||style=background:#DDD}}

|{{chset-cell1|||style=background:#DDD}}

|{{chset-cell1|||style=background:#DDD}}

|{{chset-cell1|||style=background:#DDD}}

|-

|{{chset-left1|Fx}}

|{{chset-cell1|U+0030 DIGIT ZERO|0}}

|{{chset-cell1|U+0031 DIGIT ONE|1}}

|{{chset-cell1|U+0032 DIGIT TWO|2}}

|{{chset-cell1|U+0033 DIGIT THREE|3}}

|{{chset-cell1|U+0034 DIGIT FOUR|4}}

|{{chset-cell1|U+0035 DIGIT FIVE|5}}

|{{chset-cell1|U+0036 DIGIT SIX|6}}

|{{chset-cell1|U+0037 DIGIT SEVEN|7}}

|{{chset-cell1|U+0038 DIGIT EIGHT|8}}

|{{chset-cell1|U+0039 DIGIT NINE|9}}

|{{chset-cell1|||style=background:#DDD}}

|{{chset-cell1|||style=background:#DDD}}

|{{chset-cell1|||style=background:#DDD}}

|{{chset-cell1|||style=background:#DDD}}

|{{chset-cell1|||style=background:#DDD}}

|{{chset-ctrl1| |{{Control code link|EO}}}}

|}

Following are the definitions of EBCDIC control characters which either do not map onto the ASCII control characters, or have additional uses. When mapped to Unicode, these are mostly mapped to C1 control character codepoints in a manner specified by IBM's Character Data Representation Architecture (CDRA).{{cite web|url=https://www.unicode.org/reports/tr16/tr16-6.html#Step%202 |title=3.3 Step 2: Byte Conversion |work=UTF-EBCDIC |id=Unicode Technical Report #16 |last1=Umamaheswaran |first1=V.S. |publisher=Unicode Consortium |date=1999-11-08 |quotation=The 64 control characters...the ASCII DELETE character (U+007F)...are mapped respecting EBCDIC conventions, as defined in IBM Character Data Representation Architecture, CDRA, with one exception -- the pairing of EBCDIC Line Feed and New Line control characters are swapped from their CDRA default pairings to ISO/IEC 6429 Line Feed (U+000A) and Next Line (U+0085) control characters}}{{cite web|url=https://www.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/EBCDIC/CP037.TXT |title=cp037_IBMUSCanada to Unicode table |last1=Steele |first1=Shawn |publisher=Microsoft/Unicode Consortium |date=1996-04-24}}

Although the default mapping of New Line (NL) corresponds to the ISO/IEC 6429 Next Line (NEL) character (the behaviour of which is also specified, but not required, in Unicode Annex 14),{{cite web|url=http://www.unicode.org/reports/tr14/tr14-43.html#NL |version=Revision 43 |title=NL: Next Line (A) (Non-tailorable) |work=Unicode Line Breaking Algorithm |id=Unicode Standard Annex #14 |date=2019-02-15 |first1=Andy |last1=Heninger |publisher=Unicode Consortium}} most of these C1-mapped controls match neither those in the ISO/IEC 6429 C1 set, nor those in other registered C1 control sets such as ISO 6630.{{cite iso-ir|number=124 |title=Additional Control Functions for Bibliographic Use according to International Standard ISO 6630 |sponsor=ISO/TC 46 |date=1986-02-01}} Although this effectively makes the non-ASCII EBCDIC controls a unique C1 control set, they are not among the C1 control sets registered in the ISO-IR registry,{{citation|title=ISO/IEC International Register of Coded Character Sets To Be Used With Escape Sequences |id=ISO-IR |publisher=ITSCJ/IPSJ |url=https://itscj.ipsj.or.jp/english/vbcqpr00000004qn-att/ISO-IR.pdf}} meaning that they do not have an assigned control set designation sequence (as specified by ISO/IEC 2022, and optionally permitted in ISO/IEC 10646 (Unicode)).{{citation|mode=cs1 |id=ISO/IEC 10646 |title=Information technology — Universal Coded Character Set (UCS) |author=ISO/IEC JTC 1/SC 2 |author-link=ISO/IEC JTC 1/SC 2 |publisher=ISO |edition=5th |year=2017 |url=https://standards.iso.org/ittf/PubliclyAvailableStandards/c069119_ISO_IEC_10646_2017.zip |section=12.4: Identification of control function set |pages=19–20 |quotation=For other C0 or C1 sets, the final octet F shall be obtained from the International Register of Coded Character Sets....If such an escape sequence appears within a code unit sequence conforming to this International Standard, it shall be padded in accordance with Clause 11.}}

Besides U+0085 (Next Line), the Unicode Standard does not prescribe an interpretation of C1 control characters, leaving their interpretation to higher level protocols (it suggests, but does not require, their ISO/IEC 6429 interpretations in the absence of use for other purposes),{{cite book|url=https://www.unicode.org/versions/Unicode12.0.0/ch23.pdf#page=3 |chapter=23.1: Control Codes |title=The Unicode Standard |edition=12.0.0 |date=2019 |author=Unicode Consortium |author-link=Unicode Consortium |isbn=978-1-936213-22-1 |pages=868–870}} so this mapping is permissible in, but not specified by, Unicode.

The following code pages have the full Latin-1 character set (ISO/IEC 8859-1). The first column gives the original code page number. The second column gives the number of the code page updated with the euro sign (€) replacing the universal currency sign (¤) (or in the case of EBCDIC 924, with the set changed to match ISO 8859-15)

Different countries have different code pages because these code pages originated as code pages with country-specific character repertoires, and were later expanded to contain the entire ISO 8859-1 repertoire, meaning that a given ISO 8859-1 character may have different code point values in different code pages. They are known as Country Extended Code Pages (CECPs).{{cite web |url=http://www.columbia.edu/kermit/ftp/charsets/iso8859.txt |title=iso8859.txt |publisher=Kermit project / Columbia University}}

{{Self-contradictory|othersection|date=May 2024}}

Open-source software advocate and software developer Eric S. Raymond writes in his Jargon File that EBCDIC was loathed by hackers, by which he meant{{cite web|url=http://manybooks.net/titles/anonetext02jarg422.html|title=The New Hacker's Dictionary|last=Raymond|first=Eric S.|year=1997|author-link=Eric S. Raymond|page=310}} members of a subculture of enthusiastic programmers. The Jargon File 4.4.7 gives the following definition:

{{Blockquote|text = EBCDIC: /eb´s@·dik/, /eb´see`dik/, /eb´k@·dik/, n.

[abbreviation, Extended Binary Coded Decimal Interchange Code] An alleged character set used on IBM dinosaurs. It exists in at least six mutually incompatible versions, all featuring such delights as non-contiguous letter sequences and the absence of several ASCII punctuation characters fairly important for modern computer languages (exactly which characters are absent varies according to which version of EBCDIC you're looking at). IBM adapted EBCDIC from punched card code in the early 1960s and promulgated it as a customer-control tactic (see connector conspiracy), spurning the already established ASCII standard. Today, IBM claims to be an open-systems company, but IBM's own description of the EBCDIC variants and how to convert between them is still internally classified top-secret, burn-before-reading. Hackers blanch at the very name of EBCDIC and consider it a manifestation of purest evil.

|source=The Jargon file 4.4.7}}

EBCDIC design was also the source of many jokes. One such joke, found in the Unix fortune file of 4.3BSD Reno (1990){{citation|url=https://minnie.tuhs.org/cgi-bin/utree.pl?file=4.3BSD-Reno/share/games/fortune/fortunes|title=4.3BSD-Reno/share/games/fortune/fortunes}} went:

{{Blockquote|text = Professor: "So the American government went to IBM to come up with an encryption standard, and they came up with—"
Student: "EBCDIC!"}}

References to the EBCDIC character set are made in the 1979 computer game series Zork. In the "Machine Room" in Zork II, EBCDIC is used to imply an incomprehensible language:

{{Blockquote|text = This is a large room full of assorted heavy machinery, whirring noisily. The room smells of burned resistors. Along one wall are three buttons which are, respectively, round, triangular, and square. Naturally, above these buttons are instructions written in EBCDIC...}}

In 2021, it became public that a Belgian bank was still using EBCDIC internally in 2019. A customer insisted that the correct spelling of his surname included an umlaut, which the bank omitted, and the customer filed a complaint citing the guarantee in the General Data Protection Regulation of the right to timely "rectification of inaccurate personal data." The bank's argument included the fact that their system used EBCDIC, as well as that it did not support letters with diacritics (or lower case, for that matter). The appeals court ruled in favor of the customer.{{cite web| url = https://gdprhub.eu/index.php?title=Court_of_Appeal_of_Brussels_-_2019/AR/1006| title = Court of Appeal of Brussels - 2019/AR/1006 - GDPRhub}}{{cite web |last=Eden |first=Terence |author-link=Terence Eden |date=25 October 2021 |title=EBCDIC is incompatible with GDPR – Terence Eden's Blog |url=https://shkspr.mobi/blog/2021/10/ebcdic-is-incompatible-with-gdpr/}}

• UTF-EBCDIC

{{Reflist|refs=

{{cite book|title=Coded Character Sets, History and Development |series=The Systems Programming Series |author-last=Mackenzie |author-first=Charles E. |year=1980 |edition=1 |publisher=Addison-Wesley Publishing Company, Inc. |isbn=0-201-14460-3 |lccn=77-90165 |url=https://textfiles.meulie.net/bitsaved/Books/Mackenzie_CodedCharSets.pdf |access-date=2022-04-06}}

{{cite web|author-last=Bemer |author-first=Bob |author-link=Bob Bemer |title=EBCDIC and the P-Bit (The Biggest Computer Goof Ever) - Computer History Vignettes |url=http://www.bobbemer.com/P-BIT.HTM |access-date=2013-07-02 |url-status=dead |archive-url=https://web.archive.org/web/20180513204153/http://www.bobbemer.com/P-BIT.HTM |archive-date=13 May 2018 |quote=...but their printers and punches were not ready to handle ASCII, and IBM just HAD to announce.}}

{{cite web|url=https://www.sr-ix.com/Archive/CharCodeHist/X3.4-1963/page4.JPG |title=X3.4-1963 |page=4 |date=1963}} (NB. IBM had four staff members on the final 21-member ASA X3.2 sub-committee.)

{{cite web|url=http://publib.boulder.ibm.com/infocenter/zos/v1r9/index.jsp?topic=%2Fcom.ibm.zos.r9.adms700%2Fadms7a05158.htm |title=IBM confirms the use of EBCDIC in their mainframes as a default practice |date=2008 |access-date=2008-06-16 |author=IBMnt |archive-date=2013-01-03 |archive-url=https://archive.today/20130103091717/http://publib.boulder.ibm.com/infocenter/zos/v1r9/index.jsp?topic=/com.ibm.zos.r9.adms700/adms7a05158.htm |url-status=dead}}

{{cite web|url=http://catb.org/jargon/html/E/EBCDIC.html |access-date=2018-05-13 |url-status=live |title=EBCDIC |work=Jargon File |archive-url=https://web.archive.org/web/20180513205203/http://catb.org/jargon/html/E/EBCDIC.html |archive-date=2018-05-13}}

{{cite web|publisher=IBM Corporation |title=Appendix G-1. EBCDIC control character definitions |url=https://www-01.ibm.com/software/globalization/cdra/appendix_g1.html |work=Character Data Representation Architecture |archive-url=https://web.archive.org/web/20180911044845/https://www-01.ibm.com/software/globalization/cdra/appendix_g1.html |archive-date=2018-09-11|url-status=dead}}

}}

• IBM-related:
• {{webarchive |url=https://web.archive.org/web/20180513205355/https://www-01.ibm.com/software/globalization/g11n-res.html |date=2018-05-13 |title=Character Data Representation Architecture (CDRA) from IBM}}. Contains IBM's official information on code pages and character sets.
• {{webarchive |url=https://web.archive.org/web/20150619065737/http://www-01.ibm.com/software/globalization/cp/cp00037.html |title=Code page 37 |date=2015-06-19}}
• {{webarchive |url=https://web.archive.org/web/20150607033635/http://www-01.ibm.com/software/globalization/cp/cp01047.html |title=Code page 1047 |date=2015-06-07}}
• [https://www.ibm.com/support/knowledgecenter/SSEQ5Y_5.9.0/com.ibm.pcomm.doc/reference/pdf/hcp_referenceV58.pdf Host Code Page Reference] from IBM, shows code charts for several single-byte IBM EBCDIC pages.
• [https://icu4c-demos.unicode.org/icu-bin/convexp ICU Converter Explorer] Contains more information about EBCDIC derived from IBM's CDRA, including DBCS EBCDIC (Double Byte Character Set EBCDIC)
• {{cite web|url=https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_74/nls/rbagscodepages.htm|title=Code Pages|website=IBM }} from {{cite web|url=https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_74/nls/rbagsglobalmain.htm|title=IBM i globalization|website=IBM }}
• [https://bs2manuals.ts.fujitsu.com/download/manual/3085.5 XHCS V2.0 manual], shows code charts for several single-byte Siemens/Fujitsu (as opposed to IBM) EBCDIC pages used on the BS2000.
• {{Webarchive|url=https://web.archive.org/web/20160303222819/http://www.hansenb.pdx.edu/DMKB/dict/tutorials/ebcdic.php |title= EBCDIC character list, including decimal and hex values, symbolic name, and character/function |date=2016-03-03}}
• {{Webarchive|url=https://web.archive.org/web/20180418175635/http://wzwz.de/prog/ebcdic/cc_en.htm |title=EBCDIC-code pages with Latin-1-charset (JavaScript) |date=2018-04-18}}
• {{webarchive|url=https://web.archive.org/web/20160827044130/http://mainframe.wiki/ebcdicTbl.php|title=All EBCDIC code pages and 3270 graphics escape codes|date=August 27, 2016}}
• [https://icu.unicode.org/charts/charset ICU Character Set Mapping Tables] Contains computer readable Unicode mapping tables for EBCDIC and many other character sets

{{Character encodings}}

Category:IBM mainframe operating systems