@isotrieve/core — TypeScript

Port of Isotrieve to TypeScript. Fit linear mappings between embedding spaces, gate migrations, recalibrate scores, and serve transformed queries — all from Node.js.

Cross-compatible with the Python package. .isotrieve files work identically in both runtimes.

Key Features

• 5 mapping classes — Ridge, Orthogonal Procrustes, Procrustes Diag, Low-Rank Affine, External
• Quality gate — PASS / WARN / FAIL with calibrated isotonic regression
• Score recalibration — PAVA isotonic regression for margin compression
• Confidence scoring — per-query high/medium/low flags
• Serve mode — map new-model queries into legacy space, zero corpus writes
• Binary format — v2 with CRC32, JSON header, Float64 payload, Python cross-compat
• TypedMatrix — efficient flat Float64Array matrix storage under the hood

Installation

npm install @isotrieve/core

Requires Node.js ≥ 18.

import { RidgeMapping } from '@isotrieve/core';
console.log('Ready');

Quick Start

1. Fit a mapping

import { RidgeMapping } from '@isotrieve/core';

const X = [[...], [...]];  // source vectors (N x d_src)
const Y = [[...], [...]];  // target vectors (N x d_tgt)

const m = new RidgeMapping({ alpha: 'auto', seed: 0 });
m.fit(X, Y);

const report = m.getValidationReport();
console.log(report.holdoutCosineMean, report.top1Retention);

m.save('mapping.isotrieve');

2. Load and transform

const loaded = RidgeMapping.load('mapping.isotrieve');
const Z = loaded.transform(corpus) as Float64Array[];
// Z[i] is the i-th transformed vector (Float64Array, L2-normalized)

3. Gate the migration

import { QualityGate } from '@isotrieve/core';

const gate = new QualityGate();
const report = gate.evaluate(loaded, X_test, Y_test);
console.log(report.verdict); // 'PASS' | 'WARN' | 'FAIL'

Mapping Classes

All mappings implement the same interface:

RidgeMapping

Default. Noise-robust ridge regression with generalized cross-validation for alpha selection. Supports unequal source/target dimensions.

const m = new RidgeMapping({ alpha: 1.0, bias: true, seed: 42 });
m.fit(X_cal, Y_cal);
// d_src != d_tgt — works fine (e.g., 384 -> 1536)

// Forward transform
const Z = m.transform(corpus) as Float64Array[];

// Inverse (back to source space)
const X_back = m.inverseTransform(Z) as Float64Array[];

// Validation
const r = m.getValidationReport();
console.log(r.holdoutCosineMean, r.top1Retention, r.mrr);

OrthogonalProcrustesMapping

Orthogonal rotation + reflection. Requires equal source/target dimensions. Inverse is exact transpose.

import { OrthogonalProcrustesMapping } from '@isotrieve/core';

const m = new OrthogonalProcrustesMapping({ seed: 0 });
m.fit(X, Y); // d_src === d_tgt required

// W^T = exact inverse
const Z = m.transform(V) as Float64Array[];
const V_back = m.inverseTransform(Z) as Float64Array[];

ProcrustesDiagMapping

Axis-aligned scaling + rotation. Diagonal scaling factors capture per-dimension importance. Requires equal dimensions.

import { ProcrustesDiagMapping } from '@isotrieve/core';

const m = new ProcrustesDiagMapping({ seed: 0 });
m.fit(X, Y);

// Diagonal scaling factors (d_tgt,)
console.log(m.diag);

// Inverse exists (diagonal is invertible)
const X_back = m.inverseTransform(m.transform(X)) as Float64Array[];

LowRankAffineMapping

Low-rank constraint for limited calibration data. Truncates SVD to target rank. Supports unequal dimensions.

const m = new LowRankAffineMapping({ rank: 16, alpha: 'auto', bias: true });
m.fit(X_cal, Y_cal);
const Z = m.transform(V) as Float64Array[];

ExternalMapping

Wrap any custom transform function. No fitting required.

import { ExternalMapping } from '@isotrieve/core';

const m = new ExternalMapping({
  forwardFn: (vecs) => vecs.map(v => myForward(v)),
  inverseFn: (vecs) => vecs.map(v => myInverse(v)),
  dSrc: 384,
  dTarget: 1536,
});

const Z = m.transform(V) as Float64Array[];
const X_back = m.inverseTransform(Z) as Float64Array[];

Quality Gate

Data-driven verdict based on a pre-calibrated isotonic regression model (shared with the Python package). Use before committing to migration in production.

import { QualityGate } from '@isotrieve/core';

const gate = new QualityGate();
const report = gate.evaluate(mapping, X_heldout, Y_heldout, {
  holdoutTop1: mapping.getValidationReport()?.top1Retention ?? null,
});

console.log(report.verdict);              // 'PASS' | 'WARN' | 'FAIL'
console.log(report.predictedRetention);    // 0.0 - 1.0
console.log(report.predictionInterval);    // [lower, upper] 80% CI
console.log(report.marginCompression);     // score margin ratio
console.log(report.scoreRecomRecommendation); // recalibration advice

Score Recalibration

Corrects margin compression in post-mapping similarity scores using PAVA isotonic regression.

import { ScoreRecalibrator } from '@isotrieve/core';

const recal = new ScoreRecalibrator();

// Fit on heldout (mapped_score, ceiling_score) pairs
recal.fit(mappedScores, ceilingScores);

// Transform any score array
const adjusted = recal.transform(newScores);

// Inspection
console.log(recal.report?.meanShift);     // how much scores shifted
console.log(recal.report?.marginRatio);   // post/pre margin ratio

The recalibrator can be saved alongside the mapping and loaded at serve time.

Confidence Scoring

Per-query confidence flags based on top-1 vs top-2 margin.

import { ConfidenceScorer, confidenceSummary } from '@isotrieve/core';

const scorer = new ConfidenceScorer();
const results = scorer.scoreQueries(queryIds, similarities, topK);
// results[i].flag = 'high' | 'medium' | 'low'
// results[i].is_reliable = true if 'high'

const summary = confidenceSummary(results);
console.log(summary.pctHigh, summary.pctLow);

Serve Mode (QueryAdapter)

Map new-model queries into legacy vector space on the fly. No corpus re-embedding needed.

import { QueryAdapter } from '@isotrieve/core';

const qa = QueryAdapter.load('mapping.isotrieve');

// Single query
const legacyVec = qa.mapQuery(newQueryVec);

// Batch
const legacyVecs = qa.mapQueries(batchQueryVecs);

// Recalibrate (if recalibrator was fitted)
const adjusted = qa.recalibrateScores(rawScores);

// Dimensions
console.log(qa.d_src, qa.d_tgt);   // 384, 1536
console.log(qa.hasRecalibrator);    // boolean
console.log(qa.hasMappedData);      // boolean

CSLS Hubness Correction

import { cslsScores } from '@isotrieve/core';

const scores = cslsScores(queryVecs, candidateVecs, k=10);
// Cross-domain similarity local scaling

Migration

Offline transform of entire vector store collections.

import { migrateStore } from '@isotrieve/core';

const manifest = migrateStore(sourceStore, targetStore, mapping, {
  batchSize: 1024,               // vectors per batch
  manifestPath: './manifest.json',
  resume: true,                  // skip already-migrated batches
  onBatchComplete: (b) => {
    console.log(`Batch ${b.batchNum}: ${b.rowsProcessed} vectors`);
  },
});

console.log(manifest.rowsTotal, manifest.rowsMigrated);

.isotrieve File Format

Cross-compatible between TypeScript and Python. Binary layout:

[4 bytes] Magic bytes "ISTR"
[4 bytes] Header length (unsigned 32-bit LE)
[4 bytes] CRC32 checksum of header + padding
[N bytes] JSON header (padded to 8-byte alignment)
[...]     Float64 matrix payloads (row-major)

Read/write utilities:

import { readIsotrieveHeader, readIsotrievePayload, writeIsotrieveFile } from '@isotrieve/core';

// Read header only
const header = readIsotrieveHeader('mapping.isotrieve');
console.log(header.mappingType); // 'ridge'
console.log(header.d_src, header.d_tgt);

// Full read
const { header, matrices } = readIsotrievePayload('mapping.isotrieve');

// Write
writeIsotrieveFile('output.isotrieve', header, [W, W_inv]);

Notebooks

Interactive Python notebooks demonstrating end-to-end usage. Concepts apply directly to the TypeScript package.

Which Package to Use?

Use Python for production migrations today. Use this package if you need a TypeScript library for programmatic mapping, gating, or format interop in Node.js.