js/data

js/data.ts

fino:data - data engineering and interchange formats.

Umbrella entry point for fino's data toolbox. Rather than exporting symbols directly, it groups each data surface under a namespace so related types travel together and additional surfaces can attach here without colliding. The module exposes the arrow, dataset, and frame namespaces.

The arrow namespace is a pure-TypeScript Apache Arrow implementation: the columnar in-memory format (every logical type), the IPC stream and file formats for interchange with the wider Arrow ecosystem (pyarrow, polars, DuckDB, ...), and the Arrow C Data Interface for zero-copy hand-off to native libraries over fino:ffi. It is a re-export of fino:data/arrow — importing either specifier yields the same module instance, so prefer the namespaced form here when mixing data surfaces and the direct specifier when only Arrow is needed.

The dataset namespace is the shared ingestion path for deterministic indexed data, lazy async streams, source adapters, seeded transforms, and pull-driven batching. Tabular binary adapters yield Arrow record batches; row adapters cross into Arrow explicitly through arrowCollator.

The frame namespace builds bounded lazy query plans directly over Arrow batches: expression-based filtering and projection, aggregation, joins, stable sorting, limits, and selective Parquet scans.

import { arrow } from 'fino:data';

const batch = arrow.RecordBatch.from({ id: [1, 2, 3], name: ['a', 'b', 'c'] });
const bytes = arrow.tableToIPC(batch);
const table = arrow.tableFromIPC(bytes);
console.log(table.numRows); // 3

Arrow columnar format: https://arrow.apache.org/docs/format/Columnar.html

No exported declarations found.