UTF-EBCDIC

UTF-EBCDIC

Created by	IBM
Definitions	Unicode Technical Report #16
Based on	UTF-8
Transforms / Encodes	Unicode

UTF-EBCDIC is a character encoding capable of encoding all 1,112,064 valid character code points in Unicode using 1 to 5 bytes (in contrast to a maximum of 4 for UTF-8). It is meant to be EBCDIC-friendly, so that legacy EBCDIC applications on mainframes may process the characters without much difficulty. Its advantages for existing EBCDIC-based systems are similar to UTF-8's advantages for existing ASCII-based systems. Details on UTF-EBCDIC are defined in Unicode Technical Report #16.

To produce the UTF-EBCDIC encoded version of a series of Unicode code points, an encoding based on UTF-8 (known in the specification as UTF-8-Mod) is applied first (creating what the specification calls an I8 sequence). The main difference between this encoding and UTF-8 is that it allows Unicode code points U+0080 through U+009F (the C1 control codes) to be represented as a single byte and therefore later mapped to corresponding EBCDIC control codes. In order to achieve this, UTF-8-Mod uses 101xxxxx instead of 10xxxxxx as the format for trailing bytes in a multi-byte sequence. As this can only hold 5 bits rather than 6, the UTF-8-Mod encoding of codepoints above U+03FF are larger than the UTF-8 encoding.

The UTF-8-Mod transformation leaves the data in an ASCII-based format (for example, U+0041 "A" is still encoded as 0x41), so each byte is fed through a reversible (one-to-one) lookup table to produce the final UTF-EBCDIC encoding. For example, 0x41 in this table maps to 0xC1; thus the UTF-EBCDIC encoding of U+0041 (Unicode's "A") is 0xC1 (EBCDIC's "A").

UTF-EBCDIC is rarely used, even on the EBCDIC-based mainframes for which it was designed. IBM EBCDIC-based mainframe operating systems, such as z/OS, usually use UTF-16 for complete Unicode support. For example, IBM Db2, COBOL, PL/I, Java and the IBM XML toolkit support UTF-16 on IBM mainframes.

Codepage layout

There are 160 characters with single-byte encodings in UTF-EBCDIC (compared to 128 in UTF-8). As can be seen, the single-byte portion is similar to IBM-1047 instead of IBM-37 due to the location of the square brackets. CCSID 37 has [] at hex BA and BB instead of at hex AD and BD respectively.

UTF-EBCDIC

0

1

2

3

4

5

6

7

8

9

A

B

C

D

E

F

0x

NUL

SOH

STX

ETX

ST

HT

SSA

DEL

EPA

RI

SS2

VT

FF

CR

SO

SI

1x

DLE

DC1

DC2

DC3

OSC

LF

BS

ESA

CAN

EM

PU2

SS3

FS

GS

RS

US

2x

PAD

HOP

BPH

NBH

IND

NEL

ETB

ESC

HTS

HTJ

VTS

PLD

PLU

ENQ

ACK

BEL

3x

DCS

PU1

SYN

STS

CCH

MW

SPA

EOT

SOS

SGCI

SCI

CSI

DC4

NAK

PM

SUB

4x

SP

•

•

•

•

•

•

•

•

•

•

.

<

(

+

|

5x

&

•

•

•

•

•

•

•

•

•

!

$

*

)

;

^

6x

-

/

•

•

•

•

•

•

•

•

•

,

%

_

>

?

7x

•

•

•

•

2

2

2

2

2

`

:

#

@

'

=

"

8x

2

a

b

c

d

e

f

g

h

i

2

2

2

2

2

2

9x

2

j

k

l

m

n

o

p

q

r

2

2

2

2

2

2

Ax

2

~

s

t

u

v

w

x

y

z

2

2

2

[

2

2

Bx

2

2

2

2

2

2

2

3

3

3

3

3

3

]

3

3

Cx

{

A

B

C

D

E

F

G

H

I

3

3

3

3

3

3

Dx

}

J

K

L

M

N

O

P

Q

R

3

3

4

4

4

4

Ex

\

4

S

T

U

V

W

X

Y

Z

4

4

4

5

5

Fx

0

1

2

3

4

5

6

7

8

9

APC

  Start bytes for a sequence of that many bytes. Tooltip shows the lowest code point encoded using that start byte.

  Start byte where not all combinations of continuation bytes are valid, either because it is an invalid overlong form (the tooltip shows the code point of the first valid sequence), or because it encodes a code point greater than U+10FFFF.

  Continuation bytes. Tooltip shows the hexadecimal value of the 5 bits they add.

  Unused, including lead bytes that can only start an invalid overlong form. For example, 0x76 because even 0x76 0x73 (which maps to the UTF-8-Mod sequence 0xC2 0xBF) would merely be an overlong encoding of U+005F (properly encoded as UTF-8-Mod 0x5F, UTF-EBCDIC 0x6D).

Oracle UTFE

Oracle UTFE is a Unicode 3.0 UTF-8 Oracle database variation, similar to the CESU-8 variant of UTF-8, where supplementary characters are encoded as two 4-byte characters rather than a single 4- or 5-byte character. It is used only on EBCDIC platforms.

See also

• UTF-1
• UTF-8
• BOCU-1

External links

• V.S. Umamaheswaran, Unicode Technical Report #16: the definition of UTF-EBCDIC (2002-04-16)