js/data/arrow

const Type

Arrow Type union tags (from Schema.fbs). Used as the typeId on every DataType and as the union discriminant in IPC metadata.

Prefer matching on a type's kind string in JS code; the numeric ids exist for IPC and C Data Interface interop.

const TimeUnit

Time/timestamp/duration units (Arrow TimeUnit), from whole seconds down to nanoseconds. Passed to time32/time64/timestamp/duration and stored on the resulting type's unit property.

const DateUnit

Date units (Arrow DateUnit): DAY for date32 (days since the Unix epoch) and MILLISECOND for date64 (milliseconds since the Unix epoch).

const IntervalUnit

Interval units (Arrow IntervalUnit). Selects the physical encoding of an interval(...) type: a 4-byte month count (YEAR_MONTH), an 8-byte day + millisecond pair (DAY_TIME), or a 16-byte month/day/nanosecond triple (MONTH_DAY_NANO).

const UnionMode

Union modes (Arrow UnionMode). Sparse unions store every child column at the union's full length; Dense unions add a 32-bit offsets buffer so each child stores only its own values.

const Precision

Floating-point precisions (Arrow Precision): half (16-bit), single (32-bit), and double (64-bit). Stored on FloatType.precision.

const nullType

Builds the null type. Named nullType because null is a reserved word.

import { nullType } from 'fino:data/arrow';
const t = nullType();

const bool

Builds the bit-packed boolean type.

import { bool } from 'fino:data/arrow';
const t = bool();

const int8

Builds a signed 8-bit integer type.

import { int8 } from 'fino:data/arrow';
const t = int8();

const int16

Builds a signed 16-bit integer type.

import { int16 } from 'fino:data/arrow';
const t = int16();

const int32

Builds a signed 32-bit integer type.

import { int32 } from 'fino:data/arrow';
const t = int32();

const int64

Builds a signed 64-bit integer type.

import { int64 } from 'fino:data/arrow';
const t = int64();

const uint8

Builds an unsigned 8-bit integer type.

import { uint8 } from 'fino:data/arrow';
const t = uint8();

const uint16

Builds an unsigned 16-bit integer type.

import { uint16 } from 'fino:data/arrow';
const t = uint16();

const uint32

Builds an unsigned 32-bit integer type.

import { uint32 } from 'fino:data/arrow';
const t = uint32();

const uint64

Builds an unsigned 64-bit integer type.

import { uint64 } from 'fino:data/arrow';
const t = uint64();

const float16

Builds a half-precision (16-bit) floating-point type.

import { float16 } from 'fino:data/arrow';
const t = float16();

const float32

Builds a single-precision (32-bit) floating-point type.

import { float32 } from 'fino:data/arrow';
const t = float32();

const float64

Builds a double-precision (64-bit) floating-point type.

import { float64 } from 'fino:data/arrow';
const t = float64();

const decimal

Builds a fixed-point decimal type with precision total digits and scale fractional digits. Storage defaults to 128 bits, the common width for SQL-style decimals up to 38 digits.

import { decimal } from 'fino:data/arrow';
const price = decimal(10, 2);        // 128-bit
const tiny = decimal(7, 3, 32);      // 32-bit storage

const date32

Builds a day-precision date type (32-bit days since the Unix epoch).

import { date32 } from 'fino:data/arrow';
const t = date32();

const date64

Builds a millisecond-precision date type (64-bit milliseconds since the Unix epoch).

import { date64 } from 'fino:data/arrow';
const t = date64();

const time32

Builds a 32-bit time-of-day type; defaults to milliseconds. The Arrow spec pairs 32-bit times with SECOND or MILLISECOND units — the unit is not validated here, but other tooling will reject finer units at this width.

import { time32, TimeUnit } from 'fino:data/arrow';
const t = time32(TimeUnit.SECOND);

const time64

Builds a 64-bit time-of-day type; defaults to microseconds. The Arrow spec pairs 64-bit times with MICROSECOND or NANOSECOND units.

import { time64, TimeUnit } from 'fino:data/arrow';
const t = time64(TimeUnit.NANOSECOND);

const timestamp

Builds a timestamp type; defaults to zone-naive microseconds. Pass a timezone string to mark values as absolute instants in that zone.

import { timestamp, TimeUnit } from 'fino:data/arrow';
const created = timestamp(TimeUnit.MILLISECOND, 'UTC');

const duration

Builds a duration type (64-bit elapsed-time counts); defaults to microseconds.

import { duration, TimeUnit } from 'fino:data/arrow';
const t = duration(TimeUnit.NANOSECOND);

const interval

Builds a calendar-interval type. The unit is required because it changes the physical width (4, 8, or 16 bytes per value).

import { interval, IntervalUnit } from 'fino:data/arrow';
const t = interval(IntervalUnit.DAY_TIME);

const utf8

Builds the UTF-8 string type with 32-bit offsets.

import { utf8 } from 'fino:data/arrow';
const t = utf8();

const largeUtf8

Builds the UTF-8 string type with 64-bit offsets.

import { largeUtf8 } from 'fino:data/arrow';
const t = largeUtf8();

const binary

Builds the variable-length binary type with 32-bit offsets.

import { binary } from 'fino:data/arrow';
const t = binary();

const largeBinary

Builds the variable-length binary type with 64-bit offsets.

import { largeBinary } from 'fino:data/arrow';
const t = largeBinary();

const utf8View

Builds the view-layout UTF-8 string type (16-byte views plus variadic data buffers).

import { utf8View } from 'fino:data/arrow';
const t = utf8View();

const binaryView

Builds the view-layout binary type (16-byte views plus variadic data buffers).

import { binaryView } from 'fino:data/arrow';
const t = binaryView();

const fixedSizeBinary

Builds a fixed-size binary type of exactly byteWidth bytes per value.

import { fixedSizeBinary } from 'fino:data/arrow';
const sha256 = fixedSizeBinary(32);

const list

Builds a list type with 32-bit offsets over the given element field.

import { list, utf8, Field } from 'fino:data/arrow';
const tags = list(new Field('item', utf8()));

const largeList

Builds a list type with 64-bit offsets over the given element field.

import { largeList, binary, Field } from 'fino:data/arrow';
const blobs = largeList(new Field('item', binary()));

const listView

Builds a list-view type (32-bit offsets and sizes) over the given element field.

import { listView, int32, Field } from 'fino:data/arrow';
const t = listView(new Field('item', int32()));

const largeListView

Builds a large list-view type (64-bit offsets and sizes) over the given element field.

import { largeListView, int32, Field } from 'fino:data/arrow';
const t = largeListView(new Field('item', int32()));

const fixedSizeList

Builds a fixed-size list type: every value holds exactly listSize elements of the given field.

import { fixedSizeList, float32, Field } from 'fino:data/arrow';
const embedding = fixedSizeList(768, new Field('item', float32(), false));

const struct

Builds a struct type from its member fields.

import { struct, utf8, int32, Field } from 'fino:data/arrow';
const person = struct([
  new Field('name', utf8(), false),
  new Field('age', int32()),
]);

const map

Builds a map type over an entries struct field. The child should be a non-nullable struct field with a non-nullable key field and a value field; pass keysSorted: true only when keys are sorted within each map value.

import { map, struct, utf8, int32, Field } from 'fino:data/arrow';
const t = map(new Field('entries', struct([
  new Field('key', utf8(), false),
  new Field('value', int32()),
]), false));

const union

Builds a union type. typeIds[i] is the tag written to the typeIds buffer for values of child i; tags need not be contiguous or start at zero.

import { union, UnionMode, int32, utf8, Field } from 'fino:data/arrow';
const t = union(UnionMode.Dense, [0, 1], [
  new Field('num', int32()),
  new Field('str', utf8()),
]);

const dictionary

Builds a dictionary-encoded type: indexType codes referencing a dictionary of valueType values. The id ties the field to its dictionary batch in IPC streams; distinct dictionaries in one schema need distinct ids. The resulting typeId mirrors valueType.typeId.

import { dictionary, int8, utf8 } from 'fino:data/arrow';
const color = dictionary(0, int8(), utf8());

const runEndEncoded

Builds a run-end encoded type from its two child fields: runEnds (a non-nullable integer field of exclusive run end indices) and values (one value per run).

import { runEndEncoded, int32, utf8, Field } from 'fino:data/arrow';
const t = runEndEncoded(
  new Field('run_ends', int32(), false),
  new Field('values', utf8()),
);

type BufferKind = 'validity' | 'offset32' | 'offset64' | 'size32' | 'size64' | 'data' | 'typeIds' | 'views'

A logical component of a type's physical buffer layout, in Arrow order.

validity is the null bitmap, offset32/offset64 are value offsets, size32/size64 are list-view sizes, data is the values buffer, typeIds is the union type-id buffer, and views is the 16-byte view buffer used by the Utf8View/BinaryView types.

type DataType = NullType | BoolType | IntType | FloatType | DecimalType | DateType | TimeType | TimestampType | IntervalType | DurationType | Utf8Type | LargeUtf8Type | BinaryType | LargeBinaryType | Utf8ViewType | BinaryViewType | FixedSizeBinaryType | ListType | LargeListType | ListViewType | LargeListViewType | FixedSizeListType | StructType | MapType | UnionType | DictionaryType | RunEndEncodedType

Any Arrow logical type. Narrow on the kind discriminant in a switch to recover the concrete member.

interface BaseType {

Base shape shared by every DataType.

Readonly Properties

interface NullType extends BaseType {

The null type: every value is null, and no buffers are allocated at all.

import { nullType, bufferLayout } from 'fino:data/arrow';

const t = nullType();
bufferLayout(t); // []

Properties

interface BoolType extends BaseType {

Boolean type. Values are bit-packed in the data buffer, eight per byte, so fixedWidthBytes reports 0 for it despite the fixed layout.

import { bool } from 'fino:data/arrow';

const t = bool(); // { kind: 'bool', typeId: 6 }

Properties

interface IntType extends BaseType {

Fixed-width integer of 8 to 64 bits, signed or unsigned. Built by the int8..int64 and uint8..uint64 factories.

import { int32, uint64 } from 'fino:data/arrow';

const a = int32();  // bitWidth 32, signed true
const b = uint64(); // bitWidth 64, signed false

Properties

interface FloatType extends BaseType {

IEEE 754 floating-point number in half, single, or double precision.

import { float64, Precision } from 'fino:data/arrow';

const t = float64();
t.precision === Precision.DOUBLE; // true

Properties

interface DecimalType extends BaseType {

Fixed-point decimal: precision total significant digits with scale digits after the decimal point, stored as a little-endian two's-complement integer of bitWidth bits.

import { decimal } from 'fino:data/arrow';

const money = decimal(38, 9); // 128-bit storage by default

Properties

interface DateType extends BaseType {

Calendar date. DateUnit.DAY stores 32-bit days since the Unix epoch (date32); DateUnit.MILLISECOND stores 64-bit milliseconds since the epoch (date64).

import { date32, DateUnit } from 'fino:data/arrow';

const t = date32();
t.unit === DateUnit.DAY; // true

Properties

interface TimeType extends BaseType {

Time of day since midnight, as a 32- or 64-bit integer in unit resolution. The Arrow spec pairs 32-bit storage with second/millisecond units and 64-bit storage with microsecond/nanosecond units.

import { time64, TimeUnit } from 'fino:data/arrow';

const t = time64(TimeUnit.NANOSECOND); // bitWidth 64

Properties

interface TimestampType extends BaseType {

An instant: a 64-bit count of unit intervals since the Unix epoch. With a timezone the values are absolute instants displayed in that zone; with timezone: null they are zone-naive wall-clock readings.

import { timestamp, TimeUnit } from 'fino:data/arrow';

const utc = timestamp(TimeUnit.MILLISECOND, 'UTC');
const naive = timestamp(); // microseconds, no timezone

Properties

interface IntervalType extends BaseType {

Calendar interval. The unit selects the physical encoding: months only (YEAR_MONTH), days plus milliseconds (DAY_TIME), or months, days, and nanoseconds (MONTH_DAY_NANO).

import { interval, IntervalUnit } from 'fino:data/arrow';

const t = interval(IntervalUnit.MONTH_DAY_NANO); // 16 bytes per value

Properties

interface DurationType extends BaseType {

Elapsed time as a 64-bit integer count of unit intervals, with no calendar semantics (unlike IntervalType).

import { duration, TimeUnit } from 'fino:data/arrow';

const t = duration(TimeUnit.NANOSECOND);

Properties

interface Utf8Type extends BaseType {

Variable-length UTF-8 string with 32-bit offsets, which caps a single column chunk's string data at 2 GiB. Use LargeUtf8Type beyond that.

import { utf8, bufferLayout } from 'fino:data/arrow';

bufferLayout(utf8()); // ['validity', 'offset32', 'data']

Properties

interface LargeUtf8Type extends BaseType {

Variable-length UTF-8 string with 64-bit offsets, for columns whose string data may exceed the 2 GiB limit of Utf8Type.

import { largeUtf8, bufferLayout } from 'fino:data/arrow';

bufferLayout(largeUtf8()); // ['validity', 'offset64', 'data']

Properties

interface BinaryType extends BaseType {

Variable-length byte string with 32-bit offsets. Same layout as Utf8Type but with no UTF-8 validity guarantee.

import { binary } from 'fino:data/arrow';

const t = binary(); // { kind: 'binary', typeId: 4 }

Properties

interface LargeBinaryType extends BaseType {

Variable-length byte string with 64-bit offsets, for columns whose data may exceed the 2 GiB limit of BinaryType.

import { largeBinary } from 'fino:data/arrow';

const t = largeBinary(); // { kind: 'largebinary', typeId: 19 }

Properties

interface Utf8ViewType extends BaseType {

UTF-8 string in the view ("German string") layout: each value is a 16-byte view that inlines strings of 12 bytes or fewer and otherwise points into one of a variadic set of data buffers. Views make take/filter cheap because only the fixed-size view buffer is rewritten.

import { utf8View, hasVariadicBuffers } from 'fino:data/arrow';

hasVariadicBuffers(utf8View()); // true

Properties

interface BinaryViewType extends BaseType {

Byte string in the view layout: 16-byte views with short values inlined and long values referencing variadic data buffers. The binary counterpart of Utf8ViewType.

import { binaryView, bufferLayout } from 'fino:data/arrow';

bufferLayout(binaryView()); // ['validity', 'views']

Properties

interface FixedSizeBinaryType extends BaseType {

Byte string of exactly byteWidth bytes per value, stored contiguously with no offsets buffer. Common for hashes, UUIDs, and fixed-size codes.

import { fixedSizeBinary } from 'fino:data/arrow';

const uuid = fixedSizeBinary(16);

Properties

interface ListType extends BaseType {

Variable-length list with 32-bit offsets. The element type, name, and nullability come from the child field.

import { list, int32, Field } from 'fino:data/arrow';

const scores = list(new Field('item', int32()));

Properties

interface LargeListType extends BaseType {

Variable-length list with 64-bit offsets, for lists whose flattened element count may exceed the 32-bit range of ListType.

import { largeList, utf8, Field } from 'fino:data/arrow';

const t = largeList(new Field('item', utf8()));

Properties

interface ListViewType extends BaseType {

List in the view layout: separate 32-bit offsets and sizes buffers, so lists may be stored out of order or share element ranges.

import { listView, int32, Field, bufferLayout } from 'fino:data/arrow';

const t = listView(new Field('item', int32()));
bufferLayout(t); // ['validity', 'offset32', 'size32']

Properties

interface LargeListViewType extends BaseType {

List in the view layout with 64-bit offsets and sizes; the large-offset counterpart of ListViewType.

import { largeListView, int32, Field } from 'fino:data/arrow';

const t = largeListView(new Field('item', int32()));

Properties

interface FixedSizeListType extends BaseType {

List of exactly listSize elements per value. No offsets buffer is needed; element i of list j lives at child index j * listSize + i.

import { fixedSizeList, float32, Field } from 'fino:data/arrow';

const vec3 = fixedSizeList(3, new Field('item', float32(), false));

Properties

interface StructType extends BaseType {

Named tuple of child columns. The struct itself only carries a validity bitmap; each child field stores its own buffers at the same length.

import { struct, int32, utf8, Field } from 'fino:data/arrow';

const person = struct([
  new Field('name', utf8(), false),
  new Field('age', int32()),
]);

Properties

interface MapType extends BaseType {

Map from keys to values, physically a list of entries structs. The child field must be a non-nullable struct with a non-nullable key field and a value field. keysSorted asserts keys are sorted within each map.

import { map, struct, utf8, int32, Field } from 'fino:data/arrow';

const entries = new Field('entries', struct([
  new Field('key', utf8(), false),
  new Field('value', int32()),
]), false);
const t = map(entries);

Properties

interface UnionType extends BaseType {

Tagged union of child types. Each slot's typeIds-buffer entry selects a child; typeIds[i] gives the tag value for child i. In Dense mode an extra 32-bit offsets buffer indexes into per-child columns; in Sparse mode every child spans the full length.

import { union, UnionMode, int32, utf8, Field } from 'fino:data/arrow';

const t = union(UnionMode.Dense, [0, 1], [
  new Field('num', int32()),
  new Field('str', utf8()),
]);

Properties

interface DictionaryType extends BaseType {

Dictionary-encoded column: values are indices of indexType into a shared dictionary of valueType values. Following Arrow convention, typeId mirrors the value type's id — dictionary encoding is a physical encoding recorded on the field, not a distinct logical type — so always check kind === 'dictionary' rather than the numeric id.

import { dictionary, int8, utf8 } from 'fino:data/arrow';

// Low-cardinality strings: int8 codes into a utf8 dictionary.
const color = dictionary(0, int8(), utf8());

Properties

interface RunEndEncodedType extends BaseType {

Run-end encoded column: the runEnds child holds the exclusive end index of each run and the values child holds one value per run. The type itself has no buffers — all data lives in the two children.

import { runEndEncoded, int32, utf8, Field } from 'fino:data/arrow';

const t = runEndEncoded(
  new Field('run_ends', int32(), false),
  new Field('values', utf8()),
);

Properties

interface VectorData {

Raw column data used to construct a Vector via makeVector.

Only the buffers relevant to the layout of type are consulted; the rest may be omitted. Buffers are raw little-endian bytes (Uint8Array) exactly as they appear in Arrow IPC — e.g. valueOffsets for a utf8 column is the byte image of an Int32Array, not the typed array itself.

import { makeVector, int32 } from 'fino:data/arrow';

const values = new Uint8Array(new Int32Array([10, 20, 30]).buffer);
const v = makeVector({ type: int32(), length: 3, values });
v.get(2); // 30

Properties

interface IPCWriteOptions {

Options controlling IPC serialization.

Accepted by every writer in this module. Compression applies to record batch and dictionary batch bodies only — message metadata is never compressed — and readers on the other end must support the chosen codec.

import { RecordBatch, tableToIPC, type IPCWriteOptions } from 'fino:data/arrow';

const options: IPCWriteOptions = { compression: 'lz4' };
const bytes = tableToIPC(RecordBatch.from({ id: [1, 2, 3] }), options);

Properties

function fixedWidthBytes(type: DataType): number

Byte size of a fixed-width type's element, or 0 for variable-width types.

Note that bool also reports 0: values are bit-packed, so no whole-byte element size exists. Nested and variable-length types (lists, structs, strings, views) all return 0.

import { fixedWidthBytes, int64, utf8, bool } from 'fino:data/arrow';

fixedWidthBytes(int64()); // 8
fixedWidthBytes(utf8());  // 0 (variable width)
fixedWidthBytes(bool());  // 0 (bit-packed)

function bufferLayout(type: DataType): BufferKind[]

The physical buffer layout of a type, in Arrow pre-order. Does not include child-column buffers (those follow the parent's own node in IPC order).

Dictionary types report the layout of their index type, since only the codes live in the column itself. Null and run-end-encoded types own no buffers and return an empty array. View types report views but not their variadic data buffers — check hasVariadicBuffers for those.

import { bufferLayout, utf8, list, int32, Field } from 'fino:data/arrow';

bufferLayout(utf8());                          // ['validity', 'offset32', 'data']
bufferLayout(list(new Field('item', int32()))); // ['validity', 'offset32']

function childFields(type: DataType): Field[]

Child fields of a nested type, in Arrow order. Empty for leaf types.

Run-end-encoded types return [runEnds, values]; dictionaries return nothing here because the dictionary column travels separately.

import { childFields, struct, int32, Field } from 'fino:data/arrow';

const t = struct([new Field('x', int32(), false)]);
childFields(t).map((f) => f.name); // ['x']

function hasVariadicBuffers(type: DataType): boolean

Whether a type carries variadic data buffers (the View types).

IPC record batches list a per-column buffer count; view columns append a variable number of data buffers after the fixed layout, so writers and readers use this to know when to expect them.

import { hasVariadicBuffers, utf8View, utf8 } from 'fino:data/arrow';

hasVariadicBuffers(utf8View()); // true
hasVariadicBuffers(utf8());     // false

function formatType(type: DataType): string

A human-readable label for error messages. Currently the kind string.

import { formatType, timestamp } from 'fino:data/arrow';

throw new Error(`unsupported type: ${formatType(timestamp())}`);

function typeEquals(a: DataType, b: DataType): boolean

Structural equality of two types.

Compares every own parameter (bit width, unit, timezone, precision, ...) and recurses through child fields, requiring matching child names and nullability as well as equal child types. Dictionary types additionally compare index and value types, though the dictionary id and other non-child parameters must also match.

import { typeEquals, timestamp, TimeUnit } from 'fino:data/arrow';

typeEquals(timestamp(TimeUnit.MICROSECOND, 'UTC'),
           timestamp(TimeUnit.MICROSECOND, 'UTC')); // true
typeEquals(timestamp(TimeUnit.MICROSECOND, 'UTC'),
           timestamp(TimeUnit.MICROSECOND, null));  // false

function makeVector(data: VectorData): Vector

Construct a vector from raw column buffers.

Selects the layout subclass from data.type.kind and wraps the provided buffers without copying. This is the hot path used by IPC decoding and native producers, so no validation is performed — buffers are trusted to match the layout of type, and missing or undersized buffers surface as errors (or garbage reads) only when elements are accessed.

import { makeVector, vectorFromArray, dictionary, int8, utf8 } from 'fino:data/arrow';

// A dictionary-encoded column: int8 indices into ['red', 'green', 'blue'].
const v = makeVector({
  type: dictionary(0, int8(), utf8()),
  length: 4,
  values: new Uint8Array([0, 1, 0, 2]),
  dictionary: vectorFromArray(['red', 'green', 'blue'], utf8()),
});
v.toArray(); // ['red', 'green', 'red', 'blue']

function vectorFromArray(values: unknown[], type?: DataType): Vector

Build a vector from plain JS values, inferring the type when not given.

Inference looks at the first non-null value: boolean → bool, number → float64, bigint → int64, string → utf8, arrays → list, other objects → struct. An all-null input infers int32. Passing an explicit type skips inference and encodes the values for that layout — supported kinds are bool, the fixed-width primitives (int, float, date, time, timestamp, duration), utf8/binary and their large* variants, list/largelist, struct, map, decimal, and fixedsizebinary. Both null and undefined become null slots.

Throws ArrowError for kinds it cannot build (views, unions, dictionaries, run-end encoding, intervals, fixed-size lists) — construct those with makeVector from raw buffers instead.

Intended for convenience and tests; makeVector is the performance path.

import { vectorFromArray, timestamp, TimeUnit } from 'fino:data/arrow';

vectorFromArray(['a', null, 'c']).toArray();  // ['a', null, 'c']
vectorFromArray([{ x: 1 }, { x: 2 }]).get(1); // { x: 2 }

const ts = vectorFromArray([1720000000000n, null], timestamp(TimeUnit.MILLISECOND));
ts.get(0); // 1720000000000n

function tableFromIPC(bytes: Uint8Array | ArrayBuffer): Table

Decode an Arrow IPC buffer (stream or file format) into a Table.

The one-call counterpart to tableToIPC: format detection, dictionary resolution, and decompression all happen internally. Equivalent to RecordBatchReader.from(bytes).toTable(); use the reader classes directly when per-batch access matters. Throws ArrowParseError on malformed input or when no schema message is present.

import { tableFromIPC, tableToIPC, RecordBatch } from 'fino:data/arrow';

const bytes = tableToIPC(RecordBatch.from({ id: [1, 2, 3] }));
const table = tableFromIPC(bytes);
console.log(table.numRows); // 3

function tableToIPC(source: Table | RecordBatch, options?: IPCWriteOptions & { format?: 'stream' | 'file'; }): Uint8Array

Serialize a table or a single record batch to Arrow IPC bytes.

One-shot convenience over the writer classes: a Table writes each of its batches in order, a lone RecordBatch writes just itself. format selects the streaming format (the default) or the random-access file format; tableFromIPC is the inverse and auto-detects which one it was given. Throws ArrowError if the source contains no batches.

import { RecordBatch, Table, tableToIPC, tableFromIPC } from 'fino:data/arrow';

const b1 = RecordBatch.from({ id: [1, 2], name: ['a', 'b'] });
const b2 = RecordBatch.from({ id: [3, 4], name: ['c', 'd'] });
const bytes = tableToIPC(Table.from([b1, b2]), { format: 'file' });
tableFromIPC(bytes).numRows; // 4

class Field {

A named, nullable column type with optional key/value metadata.

Fields compose recursively to describe nested types: list, fixedSizeList, map, union, and runEndEncoded types each carry child fields, and struct carries one field per member. nullable defaults to true, matching Arrow's convention that a column admits nulls unless declared otherwise.

Instances are immutable — to "change" a field, construct a new one.

import { Field, list, float64, struct, utf8 } from 'fino:data/arrow';

const readings = new Field('readings', list(new Field('item', float64(), false)));
const meta = new Field('meta', struct([new Field('tag', utf8())]), true,
  new Map([['origin', 'sensor-7']]));

Readonly Properties

Constructors

Static Methods

class Schema {

An ordered list of fields plus schema-level metadata: the type of a RecordBatch or Table.

Column order is significant — the vectors of a RecordBatch align positionally with fields. The constructor performs no validation: duplicate field names are permitted, and field() simply returns the first match.

import { Schema, Field, int32, utf8 } from 'fino:data/arrow';

const inferred = Schema.from({ id: int32(), name: utf8() });
const strict = new Schema(
  [new Field('id', int32(), false), new Field('name', utf8())],
  new Map([['version', '2']]),
);
strict.field('id')?.nullable; // false

Readonly Properties

Constructors

Static Methods

import { Schema, int64, utf8, float64 } from 'fino:data/arrow';

const schema = Schema.from({ id: int64(), name: utf8(), score: float64() });
schema.fields.map((f) => f.name); // ['id', 'name', 'score']

Methods

abstract class Vector {

Abstract base for every Arrow vector — one typed, immutable column.

The concrete subclasses (one per physical layout) are private; obtain instances through makeVector, vectorFromArray, or by decoding IPC data. All element access flows through get, which applies the validity bitmap and the logical offset before dispatching to the layout-specific read. Vectors are iterable, yielding get(0) through get(length - 1).

import { vectorFromArray } from 'fino:data/arrow';

const v = vectorFromArray([1.5, null, 3]); // inferred float64
v.isValid(1); // false
v.get(2);     // 3
[...v];       // [1.5, null, 3]

Readonly Properties

Constructors

Getters

Methods

class RecordBatch {

A batch of equal-length columns described by a Schema.

Construct directly when the schema and typed vectors are already in hand (the IPC reader and C Data importer do this), or use RecordBatch.from to build one from plain JS arrays with inferred types. All fields are read-only after construction.

import { RecordBatch, Schema, vectorFromArray, int32, utf8 } from 'fino:data/arrow';

const batch = new RecordBatch(
  Schema.from({ id: int32(), name: utf8() }),
  [vectorFromArray([1, 2], int32()), vectorFromArray(['a', 'b'], utf8())],
);
batch.row(0); // { id: 1, name: 'a' }

Readonly Properties

Constructors

Static Methods

import { RecordBatch } from 'fino:data/arrow';

const batch = RecordBatch.from({
  id: [1n, 2n, 3n],           // bigint → Int64
  name: ['ada', 'lin', null], // string → Utf8, marked nullable
});
batch.schema.fields.map((f) => f.type.kind); // ['int', 'utf8']

Getters

Methods

class Table {

A schema plus zero or more equally-shaped record batches.

The table is a thin view over its batches: nothing is copied, and batches keep their identity (batches is the array passed in). Row iteration and toArray walk the batches in order, yielding each row as a plain { name: value } object of raw physical values. Column access via getChild/columnAt returns chunked Column views instead of flattened arrays, so a multi-gigabyte IPC stream can be scanned without re-materializing its columns.

Throws if any batch's column count differs from the table schema's field count; deeper type agreement between batches is the producer's responsibility.

import { RecordBatch, Table, tableFromIPC } from 'fino:data/arrow';

// From batches produced locally...
const table = Table.from([
  RecordBatch.from({ id: [1, 2], name: ['ada', 'grace'] }),
  RecordBatch.from({ id: [3], name: ['edsger'] }),
]);

// ...or from Arrow IPC bytes (pyarrow, polars, DuckDB, ...).
// const table = tableFromIPC(bytes);

table.numRows;                    // 3
table.getChild('name')!.get(2);   // 'edsger'
for (const row of table) {
  console.log(row.id, row.name);
}

Readonly Properties

Constructors

Static Methods

Getters

Methods

class Column {

A logical column spanning a table's batches, addressed as one sequence.

A column never copies data: it holds one Vector chunk per batch and translates a table-wide row index into a (chunk, local index) pair on each access. Columns are produced by Table.getChild and Table.columnAt; each call builds a fresh view over the table's batches.

Values come back as the underlying vectors' raw physical values: null for nulls, number for widths up to 32 bits, bigint for 64-bit integers and timestamps, string, Uint8Array for binary, and arrays/objects for nested types.

import { RecordBatch, Table } from 'fino:data/arrow';

const table = Table.from([
  RecordBatch.from({ n: [1, 2] }),
  RecordBatch.from({ n: [3, 4, 5] }),
]);

const col = table.getChild('n')!;
col.length;   // 5
col.get(3);   // 4 — crosses the chunk boundary transparently
[...col];     // [1, 2, 3, 4, 5]

Readonly Properties

Constructors

Methods

class ArrowError extends Error {

Error thrown for invalid Arrow usage: bad buffer lengths, unsupported types passed to a builder, or malformed values.

This is the "caller error" class — it signals a problem with the arguments or data handed to the Arrow API rather than with decoded bytes. Typical sources: constructing a RecordBatch whose columns differ in length or count from the schema, combining batches with mismatched schemas into a Table, asking vectorFromArray to build an unsupported type, or writing a sliced column through the IPC writer. Decoding failures throw ArrowParseError instead.

import { ArrowError, RecordBatch, Schema, Field, int32, vectorFromArray } from 'fino:data/arrow';

const schema = new Schema([new Field('a', int32()), new Field('b', int32())]);
try {
  new RecordBatch(schema, [vectorFromArray([1, 2, 3], int32())]);
} catch (err) {
  err instanceof ArrowError; // true: 2 fields, 1 column
}

class ArrowParseError extends ParseError {

Error thrown when decoding an Arrow IPC stream or file that is malformed or uses an unsupported feature. Extends ParseError for hex-dump diagnostics.

Raised by tableFromIPC, RecordBatchReader, and RecordBatchFileReader for truncated buffers, missing magic bytes or continuation markers, streams with no schema message, and legacy pre-0.15 encapsulated messages. The inherited offset property locates the failing byte in the input, and render() formats a hex dump around it for logging.

import { ArrowParseError, tableFromIPC } from 'fino:data/arrow';

try {
  tableFromIPC(new Uint8Array([0xde, 0xad, 0xbe, 0xef]));
} catch (err) {
  if (err instanceof ArrowParseError) {
    console.error(`bad IPC data at byte ${err.offset}`);
    console.error(err.render());
  }
}

class RecordBatchReader {

Reads record batches from an Arrow IPC stream or file.

Construct via the static factories: from for in-memory bytes, fromAsync for an async byte source. Both formats are detected automatically — a buffer starting with the ARROW1 magic is treated as a file, anything else as a stream. All batches are decoded eagerly at construction; the instance is a plain container afterwards, iterable batch by batch or collapsible into a Table.

Construction throws ArrowParseError when the input holds no schema message or uses the legacy pre-0.15 framing, and ArrowError when a dictionary-encoded column references a dictionary that never arrived or a delta dictionary batch is encountered.

import { RecordBatchReader } from 'fino:data/arrow';

const reader = RecordBatchReader.from(bytes);
console.log(reader.schema.fields.map((f) => f.name));
for (const batch of reader) {
  console.log(batch.numRows);
}

Readonly Properties

Static Methods

import { RecordBatchReader } from 'fino:data/arrow';
import { DiskFileSystem } from 'fino:file';

const fs = new DiskFileSystem();
const file = await fs.open('/data/events.arrows');
const reader = await RecordBatchReader.fromAsync(file.reader());
await file.close();

Methods

class RecordBatchFileReader {

Reader for the Arrow IPC file format with indexed batch access.

Unlike RecordBatchReader, construction insists on the ARROW1 magic and throws ArrowParseError if it is absent — use this class when the input must be a well-formed Arrow file rather than a bare stream. All batches are decoded eagerly (the footer only bounds the message scan), after which batch(i) and numBatches give positional access.

import { RecordBatchFileReader } from 'fino:data/arrow';

const reader = RecordBatchFileReader.from(bytes);
const last = reader.batch(reader.numBatches - 1);
console.log(last.numRows);

Readonly Properties

Static Methods

Getters

Methods

class RecordBatchStreamWriter {

Writer for the Arrow IPC streaming format.

Produces the sequential wire form — schema message, dictionary batches, record batches, end-of-stream marker — meant for transports where the consumer reads messages in order (sockets, pipes, HTTP bodies) and for any Arrow-speaking peer (pyarrow, polars, DuckDB, ...). The schema is taken from the first batch written; later batches are assumed to match it and are not validated. The whole stream is buffered in memory until toBytes.

import { RecordBatch, RecordBatchStreamWriter } from 'fino:data/arrow';

const writer = new RecordBatchStreamWriter({ compression: 'lz4' });
writer
  .write(RecordBatch.from({ id: [1, 2], name: ['a', 'b'] }))
  .write(RecordBatch.from({ id: [3, 4], name: ['c', 'd'] }));
const bytes = writer.toBytes();

Constructors

Methods

class RecordBatchFileWriter {

Writer for the Arrow IPC file format (also known as Feather V2).

Wraps the streaming message sequence in ARROW1 magic bytes and appends a Flatbuffers footer recording the offset and size of every dictionary and record batch block, so readers can seek straight to any batch without scanning. This is the format to persist to disk (conventionally .arrow files); use RecordBatchStreamWriter for sequential transports. Like the stream writer, the schema comes from the first batch and everything is buffered in memory until toBytes.

import { RecordBatch, RecordBatchFileWriter } from 'fino:data/arrow';

const writer = new RecordBatchFileWriter({ compression: 'zstd' });
writer.write(RecordBatch.from({ id: [1, 2, 3], score: [0.5, 0.9, 0.1] }));
const bytes = writer.toBytes(); // write these to a .arrow file

Constructors

Methods