js/globals/encoding
js/globals/encoding.ts
TextEncoder / TextDecoder, base64, DOMException, and structuredClone globals.
This module implements the encoding-adjacent web globals that userland code
expects to find on globalThis: TextEncoder and TextDecoder for
text/byte conversion, btoa and atob for base64, DOMException and
QuotaExceededError for platform-style errors, and structuredClone for
deep copies. Everything here is installed globally at startup, so no import
is needed to use any of it.
TextEncoder.encoding is always "utf-8". TextDecoder accepts UTF-8
label aliases plus the UTF-16 labels (utf-16, utf-16le, utf-16be) and
rejects every other encoding with a RangeError. stream: true buffers
incomplete trailing multi-byte sequences and prepends them to the next
decode() call.
Why no legacy encodings?
The WHATWG Encoding Standard requires implementations to support all legacy encodings (Latin-1, Shift-JIS, etc.) for decoding. We intentionally limit this to UTF-8 and UTF-16. The vast majority of modern text is UTF-8, and adding full encoding support would require a large lookup table or a C library dependency (libiconv). Contributors who need legacy encoding support can import a pure-JS library via the module loader.
Encoder implementation
TextEncoder.encode() processes each JS code unit. JS strings are UTF-16
internally, so surrogate pairs (U+D800..U+DBFF followed by U+DC00..U+DFFF)
are recombined into a 32-bit code point before encoding as a 4-byte UTF-8
sequence. Lone surrogates are encoded as U+FFFD, matching web behavior.
Decoder implementation
TextDecoder.decode() validates each multi-byte sequence against three
classes of errors:
- Invalid lead byte: a byte whose high bits don't match any UTF-8 sequence prefix (e.g. 0xFF).
- Truncated sequence: a lead byte promises N continuation bytes but fewer are available.
- Overlong / surrogate / out-of-range: a sequence that encodes a code point that could have been expressed with fewer bytes (overlong), a code point in the surrogate range (U+D800–U+DFFF, which UTF-8 cannot encode in valid form), or a code point above U+10FFFF.
In non-fatal mode (default), all three replace the offending byte(s) with U+FFFD (the REPLACEMENT CHARACTER). In fatal mode, a TypeError is thrown.
The optional BOM strip (U+FEFF at offset 0) is on by default to match
browser behavior; set ignoreBOM: true on TextDecoder to preserve it.
encodeInto
TextEncoder.encodeInto(input, destination) writes directly into an
existing Uint8Array without allocating. It returns { read, written }:
read is the number of JS code units consumed (note: a surrogate pair
counts as 2 code units), written is the number of bytes written. Stops
early if the destination would overflow.
structuredClone
structuredClone() deep-copies a value using V8's native structured
serialization — the same machinery realm messaging uses — and falls back to
a JS clone pass only for platform objects V8 cannot reconstruct on its own
(Blob, File, CryptoKey, DOMException). See the function docs for the
full support matrix and transfer semantics.
Example
const bytes = new TextEncoder().encode('hello');
const text = new TextDecoder().decode(bytes); // "hello"
const b64 = btoa('hi'); // "aGk="
const raw = atob(b64); // "hi"
const copy = structuredClone({ nested: new Map([['x', 1]]) });
WHATWG Encoding Standard: https://encoding.spec.whatwg.org/ base64 utilities and structuredClone: https://html.spec.whatwg.org/multipage/ DOMException: https://webidl.spec.whatwg.org/#idl-DOMException
Functions
function btoa(data: string): string
Encode a Latin-1 binary string to base64 (web btoa).
Each character of the input is treated as one byte, so the input must be a
"binary string" whose character codes are all in [0, 255. Throws a
TypeError if any character code is greater than 255 — encode real text to
UTF-8 bytes with TextEncoder first if it may contain non-Latin-1
characters. The output is standard base64 with = padding.
btoa('hi'); // "aGk="
// Base64-encode arbitrary bytes:
const bytes = new TextEncoder().encode('héllo');
const b64 = btoa(String.fromCharCode(...bytes));
function atob(encodedData: string): string
Decode a base64 string to a Latin-1 binary string (web atob).
Each character of the result holds one decoded byte. ASCII whitespace
(space, tab, LF, CR, FF) in the input is ignored, and missing = padding
is accepted for valid 2- and 3-character remainders. Throws a TypeError on
any other malformed input: characters outside the base64 alphabet,
misplaced padding, or a whitespace-stripped length of % 4 == 1 (which no
padding can make valid).
atob('aGk='); // "hi"
atob('aGk'); // "hi" — missing padding accepted
// Recover bytes from base64:
const bytes = Uint8Array.from(atob('aGk='), (c) => c.charCodeAt(0));
function structuredClone<T>(value: T, options?: {
transfer?: ArrayBuffer[];
}): T
Deep-clone a value using V8's structured serialization machinery.
Fino routes ordinary JavaScript values through the runtime's V8
ValueSerializer / ValueDeserializer binding, the same native machinery
used by realm messaging. JS-defined platform objects that V8 cannot
reconstruct by itself, such as Blob, File, and CryptoKey, are cloned
through narrow Fino host-object helpers.
structured-clone support matrix
| Category | Global structuredClone() support |
|---|---|
| Primitives | undefined, null, boolean, number, string, and bigint clone by value. Symbols throw DataCloneError. |
| Plain objects | Plain objects and null-prototype objects clone recursively. Objects with custom prototypes throw DataCloneError. |
| Arrays | Dense and sparse arrays clone recursively while preserving holes. |
| Dates and regexps | Date clones preserve time values. RegExp clones preserve source and flags, with lastIndex reset. |
| Wrapper objects | Boolean, Number, and String wrapper objects clone with their primitive value. |
| Maps and sets | Map and Set clone entries recursively, including cyclic references. |
| Errors | Error, AggregateError, and DOMException clone their supported name/message/cause/error details. |
| URL types | URL and URLSearchParams throw DataCloneError. |
| File API | Blob and File clone through Fino host-object helpers. |
| Crypto | CryptoKey clones through the WebCrypto module's key-material helper. |
| Binary data | ArrayBuffer, typed arrays, BigInt64Array, BigUint64Array, and DataView clone with copied backing bytes. |
| Cycles | Object, array, map, and set cycles are preserved in the cloned graph. |
Transfer lists support only ArrayBuffer. V8 transfers the backing data into
the clone and detaches the source buffer, so the source buffer's byteLength
becomes zero. Supplying the same buffer more than once in the transfer list
throws DataCloneError.
Functions, symbols, weak collections, objects with custom prototypes,
streams, direct MessagePort values, stream transfer entries, and
MessagePort transfer entries throw DataCloneError.
const original: any = { nested: new Map([['x', 1]]) };
original.self = original;
const copy = structuredClone(original);
copy.self === copy; // true
// Transfer instead of copy — detaches the source buffer:
const buf = new ArrayBuffer(1024);
const moved = structuredClone(buf, { transfer: [buf] });
buf.byteLength; // 0
Classes
class DOMException extends Error {
Web DOMException class used by platform APIs and structuredClone errors.
The name, message, and legacy numeric code properties follow the DOM
standard names used by browsers. Unknown names receive code 0. The legacy
INDEX_SIZE_ERR-style numeric constants are defined on both the
constructor and the prototype, matching the WebIDL surface.
Fino's own APIs throw plain Error subclasses; this class exists for
web-platform compatibility — primarily structuredClone()'s
DataCloneError — and for userland code that expects the global.
try {
structuredClone(() => {});
} catch (err) {
if (err instanceof DOMException && err.name === 'DataCloneError') {
console.log(err.code); // 25
}
}
Constructors
constructor(message = '', name = 'Error')
Create a DOMException with a message and a standard error name.
Both arguments are string-coerced. name defaults to 'Error' and
selects the legacy code value when it matches one of the DOM standard
error names.
const err = new DOMException('operation was aborted', 'AbortError');
Getters
get name()
Standard error name supplied at construction, such as 'AbortError' or
'DataCloneError'.
get code()
Legacy numeric code matching name, or 0 for names without one.
new DOMException('', 'AbortError').code; // 20
new DOMException('', 'SomethingElse').code; // 0
class QuotaExceededError extends DOMException {
Web quota exceeded error.
Some web APIs throw this specialized DOMException subclass so tests and
userland code can check the constructor as well as the QuotaExceededError
name and legacy code.
throw new QuotaExceededError('storage quota exceeded');
Readonly Properties
readonly requested: number | null
Amount the failed operation asked for, or null when unknown.
readonly quota: number | null
Quota limit that was exceeded, or null when unknown.
Constructors
constructor(message = '', options: {
requested?: number | null;
quota?: number | null;
} = {})
Create a quota exceeded error.
requested and quota default to null, matching APIs that expose no
numeric quota details.
const err = new QuotaExceededError('too much data', { requested: null, quota: null });
class TextEncoder {
WHATWG TextEncoder — encodes strings to UTF-8 bytes.
Per the spec, TextEncoder supports only UTF-8; there is no label
argument. The constructor takes no options and instances are stateless, so
one shared encoder can serve a whole module. Lone surrogates in the input
are replaced with U+FFFD rather than throwing, matching browsers.
const encoder = new TextEncoder();
const bytes = encoder.encode('héllo'); // Uint8Array of UTF-8 bytes
// Zero-allocation encoding into an existing buffer:
const dest = new Uint8Array(64);
const { read, written } = encoder.encodeInto('héllo', dest);
https://encoding.spec.whatwg.org/#interface-textencoder
Getters
get encoding()
Encoding label for this encoder.
TextEncoder only supports UTF-8, so this always returns "utf-8".
new TextEncoder().encoding; // "utf-8"
Methods
encode(input: string = ''): Uint8Array
Encode input to a new Uint8Array of UTF-8 bytes.
The input is string-coerced and defaults to the empty string. Lone surrogates are encoded as U+FFFD; the method never throws on any string.
const bytes = new TextEncoder().encode('ok');
bytes[0]; // 111 ('o')
encodeInto(input: string, destination: Uint8Array): {
read: number;
written: number;
}
Encode as much of input as fits into destination without allocating.
Returns { read, written }: read is the number of UTF-16 code units
consumed (a surrogate pair counts as 2) and written is the number of
UTF-8 bytes stored. Encoding stops before a character whose full UTF-8
sequence would not fit, so destination never receives a partial
sequence — resume by re-calling with input.slice(read). Throws if
destination is not a Uint8Array.
const dest = new Uint8Array(2);
const result = new TextEncoder().encodeInto('abc', dest);
result.read; // 2
result.written; // 2
class TextDecoder {
WHATWG TextDecoder — decodes UTF-8, UTF-16LE, and UTF-16BE bytes to strings.
The constructor accepts the UTF-8 label aliases from the WHATWG Encoding
spec plus utf-16, utf-16le, and utf-16be; any other label throws a
RangeError. Malformed input is replaced with U+FFFD by default, or throws a
TypeError when constructed with fatal: true. A leading BOM is stripped
unless ignoreBOM: true.
Passing { stream: true } to decode() buffers an incomplete trailing
multi-byte sequence (or a trailing UTF-16 lead surrogate) instead of
replacing it, and prepends the buffered bytes to the next call — so a byte
stream can be decoded chunk by chunk without splitting characters. A
decoder used for streaming is stateful; use one instance per stream.
new TextDecoder().decode(new Uint8Array([0x68, 0x69])); // "hi"
// Chunked decoding — "€" (0xE2 0x82 0xAC) split across two reads:
const decoder = new TextDecoder();
decoder.decode(new Uint8Array([0xe2, 0x82]), { stream: true }); // ""
decoder.decode(new Uint8Array([0xac])); // "€"
https://encoding.spec.whatwg.org/#interface-textdecoder
Constructors
constructor(label: string = 'utf-8', options: {
fatal?: boolean;
ignoreBOM?: boolean;
} = {})
Create a TextDecoder for a supported encoding label.
label is matched case-insensitively after trimming ASCII whitespace and
defaults to 'utf-8'; unsupported labels throw a RangeError. fatal
makes malformed input throw a TypeError instead of emitting U+FFFD, and
ignoreBOM preserves an initial BOM instead of stripping it.
const decoder = new TextDecoder('utf8', { fatal: true });
decoder.encoding; // "utf-8"
new TextDecoder('shift-jis'); // throws RangeError
Getters
get encoding()
Normalized encoding name: "utf-8", "utf-16le", or "utf-16be".
Label aliases normalize to their canonical form, so 'utf8' and
'unicode-1-1-utf-8' both report "utf-8", and 'utf-16' reports
"utf-16le".
new TextDecoder('unicode-1-1-utf-8').encoding; // "utf-8"
new TextDecoder('utf-16').encoding; // "utf-16le"
get fatal()
Whether malformed input throws a TypeError instead of emitting U+FFFD.
new TextDecoder('utf-8', { fatal: true }).fatal; // true
get ignoreBOM()
Whether an initial BOM is preserved in decoded output.
false means the leading BOM is skipped, matching browser defaults.
new TextDecoder('utf-8', { ignoreBOM: true }).ignoreBOM; // true
Methods
decode(input?: ArrayBuffer | ArrayBufferView | null, options?: {
stream?: boolean;
}): string
Decode input to a string.
input may be an ArrayBuffer or any ArrayBufferView (the view's byte
range is decoded); omitting it or passing null decodes an empty chunk,
which is how a streaming sequence is flushed. Any other input type throws
a TypeError.
With stream: true, an incomplete multi-byte sequence at the end of the
chunk is buffered and prepended to the next call instead of being
replaced, enabling chunk-by-chunk decoding of a byte stream. Finish the
stream with a final non-streaming call (typically decode() with no
argument): leftover incomplete bytes then become U+FFFD, or throw a
TypeError in fatal mode.
In fatal mode a TypeError is thrown on any malformed sequence, and the decoder's buffered streaming state is reset.
const decoder = new TextDecoder();
for await (const chunk of byteStream) {
process(decoder.decode(chunk, { stream: true }));
}
process(decoder.decode()); // flush