Tcl_FreeEncoding(3tcl)
Tcl_GetEncoding(3) Tcl Library Procedures Tcl_GetEncoding(3)
_________________________________________________________________
NAME
Tcl_GetEncoding, Tcl_FreeEncoding, Tcl_GetEncodingFromObj,
Tcl_ExternalToUtfDString, Tcl_ExternalToUtf,
Tcl_UtfToExternalDString, Tcl_UtfToExternal,
Tcl_WinTCharToUtf, Tcl_WinUtfToTChar, Tcl_GetEncodingName,
Tcl_SetSystemEncoding, Tcl_GetEncodingNameFromEnvironment,
Tcl_GetEncodingNames, Tcl_CreateEncoding,
Tcl_GetEncodingSearchPath, Tcl_SetEncodingSearchPath,
Tcl_GetDefaultEncodingDir, Tcl_SetDefaultEncodingDir - pro-
cedures for creating and using encodings
SYNOPSIS
#include <tcl.h>
Tcl_Encoding
Tcl_GetEncoding(interp, name)
void
Tcl_FreeEncoding(encoding)
int |
Tcl_GetEncodingFromObj(interp, objPtr, encodingPtr) |
char *
Tcl_ExternalToUtfDString(encoding, src, srcLen, dstPtr)
char *
Tcl_UtfToExternalDString(encoding, src, srcLen, dstPtr)
int
Tcl_ExternalToUtf(interp, encoding, src, srcLen, flags, statePtr,
dst, dstLen, srcReadPtr, dstWrotePtr, dstCharsPtr)
int
Tcl_UtfToExternal(interp, encoding, src, srcLen, flags, statePtr,
dst, dstLen, srcReadPtr, dstWrotePtr, dstCharsPtr)
char *
Tcl_WinTCharToUtf(tsrc, srcLen, dstPtr)
TCHAR *
Tcl_WinUtfToTChar(src, srcLen, dstPtr)
const char *
Tcl_GetEncodingName(encoding)
int
Tcl_SetSystemEncoding(interp, name)
const char * |
Tcl Last change: 8.1 1
Tcl_GetEncoding(3) Tcl Library Procedures Tcl_GetEncoding(3)
Tcl_GetEncodingNameFromEnvironment(bufPtr) |
void
Tcl_GetEncodingNames(interp)
Tcl_Encoding
Tcl_CreateEncoding(typePtr)
Tcl_Obj * |
Tcl_GetEncodingSearchPath() |
int |
Tcl_SetEncodingSearchPath(searchPath) |
const char *
Tcl_GetDefaultEncodingDir(void)
void
Tcl_SetDefaultEncodingDir(path)
ARGUMENTS
Tcl_Interp *interp (in) Interpreter
to use for
error
reporting,
or NULL if
no error
reporting
is
desired.
const char *name (in) Name of
encoding
to load.
Tcl_Encoding encoding (in) The encod-
ing to
query,
free, or
use for
converting
text. If
encoding
is NULL,
the
current
system
encoding
is used.
Tcl_Obj *objPtr (in) Name of |
encoding |
Tcl Last change: 8.1 2
Tcl_GetEncoding(3) Tcl Library Procedures Tcl_GetEncoding(3)
to get |
token for.
Tcl_Encoding *encodingPtr (out) Points to |
storage |
where |
encoding |
token is |
to be |
written.
const char *src (in) For the
Tcl_ExternalToUtf
functions,
an array
of bytes
in the
specified
encoding
that are
to be con-
verted to
UTF-8.
For the
Tcl_UtfToExternal
and
Tcl_WinUtfToTChar
functions,
an array
of UTF-8
characters
to be con-
verted to
the speci-
fied
encoding.
const TCHAR *tsrc (in) An array
of Windows
TCHAR
characters
to convert
to UTF-8.
int srcLen (in) Length of
src or
tsrc in
bytes. If
the length
is nega-
tive, the
encoding-
Tcl Last change: 8.1 3
Tcl_GetEncoding(3) Tcl Library Procedures Tcl_GetEncoding(3)
specific
length of
the string
is used.
Tcl_DString *dstPtr (out) Pointer to
an unini-
tialized
or free
Tcl_DString
in which
the con-
verted
result
will be
stored.
int flags (in) Various
flag bits
OR-ed
together.
TCL_ENCODING_START
signifies
that the
source
buffer is
the first
block in a
(poten-
tially
multi-
block)
input
stream,
telling
the
conversion
routine to
reset to
an initial
state and
perform
any ini-
tializa-
tion that
needs to
occur
before the
first byte
is con-
verted.
TCL_ENCODING_END
Tcl Last change: 8.1 4
Tcl_GetEncoding(3) Tcl Library Procedures Tcl_GetEncoding(3)
signifies
that the
source
buffer is
the last
block in a
(poten-
tially
multi-
block)
input
stream,
telling
the
conversion
routine to
perform
any final-
ization
that needs
to occur
after the
last byte
is con-
verted and
then to
reset to
an initial
state.
TCL_ENCODING_STOPONERROR
signifies
that the
conversion
routine
should
return
immedi-
ately upon
reading a
source
character
that does
not exist
in the
target
encoding;
otherwise
a default
fallback
character
will
automati-
Tcl Last change: 8.1 5
Tcl_GetEncoding(3) Tcl Library Procedures Tcl_GetEncoding(3)
cally be
substi-
tuted.
Tcl_EncodingState *statePtr (in/out) Used when
converting
a (gen-
erally
long or
indefinite
length)
byte
stream in
a piece-
by-piece
fashion.
The
conversion
routine
stores its
current
state in
*statePtr
after src
(the
buffer
containing
the
current
piece) has
been con-
verted;
that state
informa-
tion must
be passed
back when
converting
the next
piece of
the stream
so the
conversion
routine
knows what
state it
was in
when it
left off
at the end
of the
last
Tcl Last change: 8.1 6
Tcl_GetEncoding(3) Tcl Library Procedures Tcl_GetEncoding(3)
piece.
May be
NULL, in
which case
the value
specified
for flags
is ignored
and the
source
buffer is
assumed to
contain
the com-
plete
string to
convert.
char *dst (out) Buffer in
which the
converted
result
will be
stored.
No more
than
dstLen
bytes will
be stored
in dst.
int dstLen (in) The max-
imum
length of
the output
buffer dst
in bytes.
int *srcReadPtr (out) Filled
with the
number of
bytes from
src that
were actu-
ally con-
verted.
This may
be less
than the
original
source
length if
Tcl Last change: 8.1 7
Tcl_GetEncoding(3) Tcl Library Procedures Tcl_GetEncoding(3)
there was
a problem
converting
some
source
charac-
ters. May
be NULL.
int *dstWrotePtr (out) Filled
with the
number of
bytes that
were actu-
ally
stored in
the output
buffer as
a result
of the
conver-
sion. May
be NULL.
int *dstCharsPtr (out) Filled
with the
number of
characters
that
correspond
to the
number of
bytes
stored in
the output
buffer.
May be
NULL.
Tcl_DString *bufPtr (out) Storage |
for the |
prescribed |
system |
encoding |
name.
const Tcl_EncodingType *typePtr (in) Structure
that
defines a
new type
of encod-
ing.
Tcl Last change: 8.1 8
Tcl_GetEncoding(3) Tcl Library Procedures Tcl_GetEncoding(3)
Tcl_Obj *searchPath (in) List of |
filesystem |
direc- |
tories in |
which to |
search for |
encoding |
data |
files.
const char *path (in) A path to
the loca-
tion of
the encod-
ing file.
_________________________________________________________________
INTRODUCTION
These routines convert between Tcl's internal character
representation, UTF-8, and character representations used by
various operating systems or file systems, such as Unicode,
ASCII, or Shift-JIS. When operating on strings, such as
such as obtaining the names of files or displaying charac-
ters using international fonts, the strings must be
translated into one or possibly multiple formats that the
various system calls can use. For instance, on a Japanese
Unix workstation, a user might obtain a filename represented
in the EUC-JP file encoding and then translate the charac-
ters to the jisx0208 font encoding in order to display the
filename in a Tk widget. The purpose of the encoding pack-
age is to help bridge the translation gap. UTF-8 provides
an intermediate staging ground for all the various encod-
ings. In the example above, text would be translated into
UTF-8 from whatever file encoding the operating system is
using. Then it would be translated from UTF-8 into whatever
font encoding the display routines require.
Some basic encodings are compiled into Tcl. Others can be
defined by the user or dynamically loaded from encoding
files in a platform-independent manner.
DESCRIPTION
Tcl_GetEncoding finds an encoding given its name. The name
may refer to a built-in Tcl encoding, a user-defined encod-
ing registered by calling Tcl_CreateEncoding, or a
dynamically-loadable encoding file. The return value is a
token that represents the encoding and can be used in subse-
quent calls to procedures such as Tcl_GetEncodingName,
Tcl_FreeEncoding, and Tcl_UtfToExternal. If the name did
not refer to any known or loadable encoding, NULL is
returned and an error message is returned in interp.
Tcl Last change: 8.1 9
Tcl_GetEncoding(3) Tcl Library Procedures Tcl_GetEncoding(3)
The encoding package maintains a database of all encodings
currently in use. The first time name is seen,
Tcl_GetEncoding returns an encoding with a reference count
of 1. If the same name is requested further times, then the
reference count for that encoding is incremented without the
overhead of allocating a new encoding and all its associated
data structures.
When an encoding is no longer needed, Tcl_FreeEncoding
should be called to release it. When an encoding is no
longer in use anywhere (i.e., it has been freed as many
times as it has been gotten) Tcl_FreeEncoding will release
all storage the encoding was using and delete it from the
database.
Tcl_GetEncodingFromObj treats the string representation of |
objPtr as an encoding name, and finds an encoding with that |
name, just as Tcl_GetEncoding does. When an encoding is |
found, it is cached within the objPtr value for future |
reference, the Tcl_Encoding token is written to the storage |
pointed to by encodingPtr, and the value TCL_OK is returned. |
If no such encoding is found, the value TCL_ERROR is |
returned, and no writing to *encodingPtr takes place. Just |
as with Tcl_GetEncoding, the caller should call |
Tcl_FreeEncoding on the resulting encoding token when that |
token will no longer be used.
Tcl_ExternalToUtfDString converts a source buffer src from
the specified encoding into UTF-8. The converted bytes are
stored in dstPtr, which is then null-terminated. The caller
should eventually call Tcl_DStringFree to free any informa-
tion stored in dstPtr. When converting, if any of the char-
acters in the source buffer cannot be represented in the
target encoding, a default fallback character will be used.
The return value is a pointer to the value stored in the
DString.
Tcl_ExternalToUtf converts a source buffer src from the
specified encoding into UTF-8. Up to srcLen bytes are con-
verted from the source buffer and up to dstLen converted
bytes are stored in dst. In all cases, *srcReadPtr is
filled with the number of bytes that were successfully con-
verted from src and *dstWrotePtr is filled with the
corresponding number of bytes that were stored in dst. The
return value is one of the following:
TCL_OK All bytes of src were con-
verted.
TCL_CONVERT_NOSPACE The destination buffer was
not large enough for all
of the converted data; as
Tcl Last change: 8.1 10
Tcl_GetEncoding(3) Tcl Library Procedures Tcl_GetEncoding(3)
many characters as could
fit were converted though.
TCL_CONVERT_MULTIBYTE The last few bytes in the
source buffer were the
beginning of a multibyte
sequence, but more bytes
were needed to complete
this sequence. A subse-
quent call to the conver-
sion routine should pass a
buffer containing the
unconverted bytes that
remained in src plus some
further bytes from the
source stream to properly
convert the formerly
split-up multibyte
sequence.
TCL_CONVERT_SYNTAX The source buffer con-
tained an invalid charac-
ter sequence. This may
occur if the input stream
has been damaged or if the
input encoding method was
misidentified.
TCL_CONVERT_UNKNOWN The source buffer con-
tained a character that
could not be represented
in the target encoding and
TCL_ENCODING_STOPONERROR
was specified.
Tcl_UtfToExternalDString converts a source buffer src from
UTF-8 into the specified encoding. The converted bytes are
stored in dstPtr, which is then terminated with the
appropriate encoding-specific null. The caller should even-
tually call Tcl_DStringFree to free any information stored
in dstPtr. When converting, if any of the characters in the
source buffer cannot be represented in the target encoding,
a default fallback character will be used. The return value
is a pointer to the value stored in the DString.
Tcl_UtfToExternal converts a source buffer src from UTF-8
into the specified encoding. Up to srcLen bytes are con-
verted from the source buffer and up to dstLen converted
bytes are stored in dst. In all cases, *srcReadPtr is
filled with the number of bytes that were successfully con-
verted from src and *dstWrotePtr is filled with the
corresponding number of bytes that were stored in dst. The
Tcl Last change: 8.1 11
Tcl_GetEncoding(3) Tcl Library Procedures Tcl_GetEncoding(3)
return values are the same as the return values for
Tcl_ExternalToUtf.
Tcl_WinUtfToTChar and Tcl_WinTCharToUtf are Windows-only
convenience functions for converting between UTF-8 and Win-
dows strings. On Windows 95 (as with the Unix operating
system), all strings exchanged between Tcl and the operating
system are "char" based. On Windows NT, some strings
exchanged between Tcl and the operating system are "char"
oriented while others are in Unicode. By convention, in
Windows a TCHAR is a character in the ANSI code page on Win-
dows 95 and a Unicode character on Windows NT.
If you planned to use the same "char" based interfaces on
both Windows 95 and Windows NT, you could use
Tcl_UtfToExternal and Tcl_ExternalToUtf (or their
Tcl_DString equivalents) with an encoding of NULL (the
current system encoding). On the other hand, if you planned
to use the Unicode interface when running on Windows NT and
the "char" interfaces when running on Windows 95, you would
have to perform the following type of test over and over in
your program (as represented in pseudo-code):
if (running NT) {
encoding <- Tcl_GetEncoding("unicode");
nativeBuffer <- Tcl_UtfToExternal(encoding, utfBuffer);
Tcl_FreeEncoding(encoding);
} else {
nativeBuffer <- Tcl_UtfToExternal(NULL, utfBuffer);
}
Tcl_WinUtfToTChar and Tcl_WinTCharToUtf automatically handle
this test and use the proper encoding based on the current
operating system. Tcl_WinUtfToTChar returns a pointer to a
TCHAR string, and Tcl_WinTCharToUtf expects a TCHAR string
pointer as the src string. Otherwise, these functions
behave identically to Tcl_UtfToExternalDString and
Tcl_ExternalToUtfDString.
Tcl_GetEncodingName is roughly the inverse of
Tcl_GetEncoding. Given an encoding, the return value is the
name argument that was used to create the encoding. The
string returned by Tcl_GetEncodingName is only guaranteed to
persist until the encoding is deleted. The caller must not
modify this string.
Tcl_SetSystemEncoding sets the default encoding that should
be used whenever the user passes a NULL value for the encod-
ing argument to any of the other encoding functions. If
name is NULL, the system encoding is reset to the default
system encoding, binary. If the name did not refer to any
known or loadable encoding, TCL_ERROR is returned and an
error message is left in interp. Otherwise, this procedure
increments the reference count of the new system encoding,
Tcl Last change: 8.1 12
Tcl_GetEncoding(3) Tcl Library Procedures Tcl_GetEncoding(3)
decrements the reference count of the old system encoding,
and returns TCL_OK.
Tcl_GetEncodingNameFromEnvironment provides a means for the |
Tcl library to report the encoding name it believes to be |
the correct one to use as the system encoding, based on sys- |
tem calls and examination of the environment suitable for |
the platform. It accepts bufPtr, a pointer to an uninitial- |
ized or freed Tcl_DString and writes the encoding name to |
it. The Tcl_DStringValue is returned.
Tcl_GetEncodingNames sets the interp result to a list con-
sisting of the names of all the encodings that are currently
defined or can be dynamically loaded, searching the encoding
path specified by Tcl_SetDefaultEncodingDir. This procedure
does not ensure that the dynamically-loadable encoding files
contain valid data, but merely that they exist.
Tcl_CreateEncoding defines a new encoding and registers the
C procedures that are called back to convert between the
encoding and UTF-8. Encodings created by Tcl_CreateEncoding
are thereafter visible in the database used by
Tcl_GetEncoding. Just as with the Tcl_GetEncoding pro-
cedure, the return value is a token that represents the
encoding and can be used in subsequent calls to other encod-
ing functions. Tcl_CreateEncoding returns an encoding with
a reference count of 1. If an encoding with the specified
name already exists, then its entry in the database is
replaced with the new encoding; the token for the old encod-
ing will remain valid and continue to behave as before, but
users of the new token will now call the new encoding pro-
cedures.
The typePtr argument to Tcl_CreateEncoding contains informa-
tion about the name of the encoding and the procedures that
will be called to convert between this encoding and UTF-8.
It is defined as follows:
typedef struct Tcl_EncodingType {
const char *encodingName;
Tcl_EncodingConvertProc *toUtfProc;
Tcl_EncodingConvertProc *fromUtfProc;
Tcl_EncodingFreeProc *freeProc;
ClientData clientData;
int nullSize;
} Tcl_EncodingType;
The encodingName provides a string name for the encoding, by
which it can be referred in other procedures such as
Tcl_GetEncoding. The toUtfProc refers to a callback pro-
cedure to invoke to convert text from this encoding into
UTF-8. The fromUtfProc refers to a callback procedure to
Tcl Last change: 8.1 13
Tcl_GetEncoding(3) Tcl Library Procedures Tcl_GetEncoding(3)
invoke to convert text from UTF-8 into this encoding. The
freeProc refers to a callback procedure to invoke when this
encoding is deleted. The freeProc field may be NULL. The
clientData contains an arbitrary one-word value passed to
toUtfProc, fromUtfProc, and freeProc whenever they are
called. Typically, this is a pointer to a data structure
containing encoding-specific information that can be used by
the callback procedures. For instance, two very similar
encodings such as ascii and macRoman may use the same call-
back procedure, but use different values of clientData to
control its behavior. The nullSize specifies the number of
zero bytes that signify end-of-string in this encoding. It
must be 1 (for single-byte or multi-byte encodings like
ASCII or Shift-JIS) or 2 (for double-byte encodings like
Unicode). Constant-sized encodings with 3 or more bytes per
character (such as CNS11643) are not accepted.
The callback procedures toUtfProc and fromUtfProc should
match the type Tcl_EncodingConvertProc:
typedef int Tcl_EncodingConvertProc(
ClientData clientData,
const char *src,
int srcLen,
int flags,
Tcl_EncodingState *statePtr,
char *dst,
int dstLen,
int *srcReadPtr,
int *dstWrotePtr,
int *dstCharsPtr);
The toUtfProc and fromUtfProc procedures are called by the
Tcl_ExternalToUtf or Tcl_UtfToExternal family of functions
to perform the actual conversion. The clientData parameter
to these procedures is the same as the clientData field
specified to Tcl_CreateEncoding when the encoding was
created. The remaining arguments to the callback procedures
are the same as the arguments, documented at the top, to
Tcl_ExternalToUtf or Tcl_UtfToExternal, with the following
exceptions. If the srcLen argument to one of those high-
level functions is negative, the value passed to the call-
back procedure will be the appropriate encoding-specific
string length of src. If any of the srcReadPtr, dstWro-
tePtr, or dstCharsPtr arguments to one of the high-level
functions is NULL, the corresponding value passed to the
callback procedure will be a non-NULL location.
The callback procedure freeProc, if non-NULL, should match
the type Tcl_EncodingFreeProc:
typedef void Tcl_EncodingFreeProc(
ClientData clientData);
Tcl Last change: 8.1 14
Tcl_GetEncoding(3) Tcl Library Procedures Tcl_GetEncoding(3)
This freeProc function is called when the encoding is
deleted. The clientData parameter is the same as the
clientData field specified to Tcl_CreateEncoding when the
encoding was created.
Tcl_GetEncodingSearchPath and Tcl_SetEncodingSearchPath are |
called to access and set the list of filesystem directories |
searched for encoding data files. |
The value returned by Tcl_GetEncodingSearchPath is the value |
stored by the last successful call to |
Tcl_SetEncodingSearchPath. If no calls to |
Tcl_SetEncodingSearchPath have occurred, Tcl will compute an |
initial value based on the environment. There is one encod- |
ing search path for the entire process, shared by all |
threads in the process. |
Tcl_SetEncodingSearchPath stores searchPath and returns |
TCL_OK, unless searchPath is not a valid Tcl list, which |
causes TCL_ERROR to be returned. The elements of searchPath |
are not verified as existing readable filesystem direc- |
tories. When searching for encoding data files takes place, |
and non-existent or non-readable filesystem directories on |
the searchPath are silently ignored. |
Tcl_GetDefaultEncodingDir and Tcl_SetDefaultEncodingDir are |
obsolete interfaces best replaced with calls to |
Tcl_GetEncodingSearchPath and Tcl_SetEncodingSearchPath. |
They are called to access and set the first element of the |
searchPath list. Since Tcl searches searchPath for encoding |
data files in list order, these routines establish the |
"default" directory in which to find encoding data files.
ENCODING FILES
Space would prohibit precompiling into Tcl every possible
encoding algorithm, so many encodings are stored on disk as
dynamically-loadable encoding files. This behavior also
allows the user to create additional encoding files that can
be loaded using the same mechanism. These encoding files
contain information about the tables and/or escape sequences
used to map between an external encoding and Unicode. The
external encoding may consist of single-byte, multi-byte, or
double-byte characters.
Each dynamically-loadable encoding is represented as a text
file. The initial line of the file, beginning with a "#"
symbol, is a comment that provides a human-readable descrip-
tion of the file. The next line identifies the type of
encoding file. It can be one of the following letters:
[1] S
A single-byte encoding, where one character is always
Tcl Last change: 8.1 15
Tcl_GetEncoding(3) Tcl Library Procedures Tcl_GetEncoding(3)
one byte long in the encoding. An example is iso8859-
1, used by many European languages.
[2] D
A double-byte encoding, where one character is always
two bytes long in the encoding. An example is big5,
used for Chinese text.
[3] M
A multi-byte encoding, where one character may be
either one or two bytes long. Certain bytes are lead
bytes, indicating that another byte must follow and
that together the two bytes represent one character.
Other bytes are not lead bytes and represent them-
selves. An example is shiftjis, used by many Japanese
computers.
[4] E
An escape-sequence encoding, specifying that certain
sequences of bytes do not represent characters, but
commands that describe how following bytes should be
interpreted.
The rest of the lines in the file depend on the type.
Cases [1], [2], and [3] are collectively referred to as
table-based encoding files. The lines in a table-based
encoding file are in the same format as this example taken
from the shiftjis encoding (this is not the complete file):
# Encoding file: shiftjis, multi-byte
M
003F 0 40
00
0000000100020003000400050006000700080009000A000B000C000D000E000F
0010001100120013001400150016001700180019001A001B001C001D001E001F
0020002100220023002400250026002700280029002A002B002C002D002E002F
0030003100320033003400350036003700380039003A003B003C003D003E003F
0040004100420043004400450046004700480049004A004B004C004D004E004F
0050005100520053005400550056005700580059005A005B005C005D005E005F
0060006100620063006400650066006700680069006A006B006C006D006E006F
0070007100720073007400750076007700780079007A007B007C007D203E007F
0080000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000FF61FF62FF63FF64FF65FF66FF67FF68FF69FF6AFF6BFF6CFF6DFF6EFF6F
FF70FF71FF72FF73FF74FF75FF76FF77FF78FF79FF7AFF7BFF7CFF7DFF7EFF7F
FF80FF81FF82FF83FF84FF85FF86FF87FF88FF89FF8AFF8BFF8CFF8DFF8EFF8F
FF90FF91FF92FF93FF94FF95FF96FF97FF98FF99FF9AFF9BFF9CFF9DFF9EFF9F
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
81
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
Tcl Last change: 8.1 16
Tcl_GetEncoding(3) Tcl Library Procedures Tcl_GetEncoding(3)
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
300030013002FF0CFF0E30FBFF1AFF1BFF1FFF01309B309C00B4FF4000A8FF3E
FFE3FF3F30FD30FE309D309E30034EDD30053006300730FC20152010FF0F005C
301C2016FF5C2026202520182019201C201DFF08FF0930143015FF3BFF3DFF5B
FF5D30083009300A300B300C300D300E300F30103011FF0B221200B100D70000
00F7FF1D2260FF1CFF1E22662267221E22342642264000B0203220332103FFE5
FF0400A200A3FF05FF03FF06FF0AFF2000A72606260525CB25CF25CE25C725C6
25A125A025B325B225BD25BC203B301221922190219121933013000000000000
000000000000000000000000000000002208220B2286228722822283222A2229
000000000000000000000000000000002227222800AC21D221D4220022030000
0000000000000000000000000000000000000000222022A52312220222072261
2252226A226B221A223D221D2235222B222C0000000000000000000000000000
212B2030266F266D266A2020202100B6000000000000000025EF000000000000
The third line of the file is three numbers. The first
number is the fallback character (in base 16) to use when
converting from UTF-8 to this encoding. The second number
is a 1 if this file represents the encoding for a symbol
font, or 0 otherwise. The last number (in base 10) is how
many pages of data follow.
Subsequent lines in the example above are pages that
describe how to map from the encoding into 2-byte Unicode.
The first line in a page identifies the page number. Fol-
lowing it are 256 double-byte numbers, arranged as 16 rows
of 16 numbers. Given a character in the encoding, the high
byte of that character is used to select which page, and the
low byte of that character is used as an index to select one
of the double-byte numbers in that page - the value obtained
being the corresponding Unicode character. By examination
of the example above, one can see that the characters 0x7E
and 0x8163 in shiftjis map to 203E and 2026 in Unicode,
respectively.
Following the first page will be all the other pages, each
in the same format as the first: one number identifying the
page followed by 256 double-byte Unicode characters. If a
character in the encoding maps to the Unicode character
0000, it means that the character does not actually exist.
If all characters on a page would map to 0000, that page can
be omitted.
Case [4] is the escape-sequence encoding file. The lines in
an this type of file are in the same format as this example
taken from the iso2022-jp encoding:
# Encoding file: iso2022-jp, escape-driven
E
init {}
final {}
iso8859-1 \x1b(B
jis0201 \x1b(J
Tcl Last change: 8.1 17
Tcl_GetEncoding(3) Tcl Library Procedures Tcl_GetEncoding(3)
jis0208 \x1b$@
jis0208 \x1b$B
jis0212 \x1b$(D
gb2312 \x1b$A
ksc5601 \x1b$(C
In the file, the first column represents an option and the
second column is the associated value. init is a string to
emit or expect before the first character is converted,
while final is a string to emit or expect after the last
character. All other options are names of table-based
encodings; the associated value is the escape-sequence that
marks that encoding. Tcl syntax is used for the values; in
the above example, for instance, "{}" represents the empty
string and "\x1b" represents character 27.
When Tcl_GetEncoding encounters an encoding name that has
not been loaded, it attempts to load an encoding file called
name.enc from the encoding subdirectory of each directory
that Tcl searches for its script library. If the encoding
file exists, but is malformed, an error message will be left
in interp.
KEYWORDS
utf, encoding, convert
Tcl Last change: 8.1 18
Man(1) output converted with
man2html