GeoPBF — Lightweight Binary GIS Format

Protocol Buffers-based geospatial binary format.
Lightweight · Universal converter · ArrayBuffer-native — the wire format for browser GIS

1. Why GeoPBF?

GeoJSON is readable and convenient, but has critical limitations at scale. As a text format it is expensive to transfer, slow to parse, and blocks the browser's main thread. Shapefile is widely adopted but its multi-file structure is ill-suited for the Web. Tiled MVT fragments data by zoom level, increasing round-trip overhead.

GeoPBF is a binary format designed as the communication layer for browser GIS. Protocol Buffers compression efficiency, coordinate delta-encoding, and an ArrayBuffer-native design make transfer, decoding, and inter-Worker handoff uniformly fast.

2. Binary Format Structure

A GeoPBF file consists of two parts: a Header Section and a Body Section (FARRAY).

Field	Tag	Type	Description
GTYPE	8	Varint	0:Point 1:MPoint 2:Line 3:MLine 4:Poly 5:MPoly 6:Collection
LENGTH	9	Packed Varint	Vertex counts for rings and multi-part geometries
COORDS	10	Packed SVarint	Delta-compressed coordinate stream X₀,Y₀,ΔX₁,ΔY₁,... (see below)
INDEX	12	Packed Varint	Indices into the global KEYS dictionary
VALUE	11	Repeated Message	NULL/BOOL/INTEGER/FLOAT/STRING/DATE/COLOR/JSON/BLOB/IMAGE

3. Delta Encoding × Varint

Storing coordinates as floating-point text like GeoJSON means the longitude 139.7412345 alone costs 11 bytes. GeoPBF reduces this in two stages.

The coordinate difference between adjacent vertices is typically just a few hundred metres, so after integer scaling the delta values are small. Varint encodes small integers in fewer bytes, giving GeoPBF a property that is ideal for geographic data: denser vertex clusters compress more aggressively.

4. GIS Conversion Hub

GeoPBF is not merely a storage format — it acts as a conversion hub among major GIS formats, with decoding (import) and encoding (export) paths for all of them. Placing GeoPBF at the center enables any-to-any format conversion in a single step.

The first eight rows go through the auto-detecting entry point geopbf(input). GeoParquet and PMTiles are large enough to be worth loading on demand, so they live behind their own subpath exports (geopbf/geoparquet, geopbf/pmtiles) and are covered in §13. Rasters travel a separate path again: geopbf/cog reads Cloud Optimized GeoTIFF by HTTP Range without converting it at all (§12). Everything here also has a CLI equivalent — npx geopbf enc / dec / pmtiles / parquet / parquet2pbf — running on plain Node with no GDAL.

// Any format, the same API
const pbf = await geopbf(file);          // File (Shapefile .zip, GeoJSON, KMZ ...)
const pbf = await geopbf(arrayBuffer);    // ArrayBuffer (fetch response, etc.)
const pbf = await geopbf(geojsonObject);  // GeoJSON object directly
const pbf = await geopbf(zip, { format: "moj" });   // cadastral .zip — plane-rectangular → WGS84

// Export to any format
const geojson  = pbf.geojson;    // → GeoJSON object
const topojson = pbf.topojson;   // → TopoJSON (topology-preserving)
const shp      = pbf.shape();    // → Shapefile (.zip Blob)
const kmz      = pbf.kmz();      // → KMZ Blob
const gml      = pbf.gml();      // → GML string
const fgb      = await pbf.fgb(); // → FlatGeoBuf ArrayBuffer

// v1.5 — the two large formats, behind their own subpath exports (§13)
import { toPMTiles } from "geopbf/pmtiles";
import { toGeoParquet, fromGeoParquet } from "geopbf/geoparquet";
const { buffer } = await toPMTiles(pbf, { maxZoom: 12 });  // → PMTiles (MVT) bytes
const pq  = await toGeoParquet(pbf);                    // → GeoParquet bytes
const back = await fromGeoParquet(u8);                  // ← GeoParquet, incl. geopandas/DuckDB output

// Save as native .geopbf (gzip-compressed)
await pbf.save('coastline');    // → downloads coastline.geopbf

5. Antimeridian Handling

The antimeridian (±180° longitude) is a well-known trap in geographic data. A polygon that straddles it — for example a geometry spanning the Bering Sea — is stored as a single ring in most formats, with coordinates jumping from +179° to −179°. Renderers, spatial indexes, and clipping operations all break silently on such data.

GeoPBF treats antimeridian correctness as a format invariant: every geometry stored in GeoPBF is guaranteed to be antimeridian-cut. No straddling polygon can exist inside the format.

5.1 Spherical Great-Circle Intersection

The cut is not a naive ±180° coordinate clamp. antimeridianCut() computes the precise latitude at which each crossing segment intersects the antimeridian using the spherical formula — essential near the poles where meridian convergence is significant. After cutting, each sub-polygon is a valid closed ring with vertices inserted at the exact intersection latitude.

// Crossing detection: sign change + span > 180° means antimeridian crossing (not date line jump)
// p[i][0] * p[i+1][0] < 0  &&  abs(p[i][0] - p[i+1][0]) > 180

// Spherical intersection latitude (great-circle formula)
// x = sin(Δlat/2)·sin(avgLng)·cos(halfDeltaLng) - sin(avgLat)·cos(avgLng)·sin(halfDeltaLng)
// z = cos(lat0)·cos(lat1)·sin(Δlng)
// intersectLat = atan2(x, |z|)   ← geometrically exact on the sphere

6. ArrayBuffer & Worker Architecture

GeoPBF's internal representation is consistently an ArrayBuffer. This is a deliberate design choice for deep integration with the browser's parallel processing layer (Web Workers), yielding three important properties.

6.1 Transferable — Zero-Copy Transfer

// ArrayBuffer moves from main thread → Worker with zero copy
// (ownership transfer, not a copy — no extra memory consumed)
worker.postMessage(pbf.arrayBuffer, [pbf.arrayBuffer]);
// After transfer pbf.arrayBuffer is detached (main thread can no longer access it)

// Worker side
onmessage = ({ data: buf }) => {
	const view = new DataView(buf);  // begin decoding immediately
	postMessage(result, [result]);    // return zero-copy
};

6.2 SharedArrayBuffer — Concurrent Multi-Worker Reads

// Expand decoded gint coordinate data into a SharedArrayBuffer
// → multiple render Workers can read concurrently with no copies
const sab = new SharedArrayBuffer(buf.byteLength);
new Uint8Array(sab).set(new Uint8Array(buf));

// N tile workers run LOD filter + draw in parallel
tileWorkers.forEach(w => w.postMessage({ sab, zoom, viewport }));

6.3 CompressionStream — Native gzip

// On save: browser-native gzip (no external library needed)
let blob = new Blob([buf]);
blob = await (new Response(
	blob.stream().pipeThrough(new CompressionStream("gzip"))
)).blob();
postMessage(new File([blob], `${name}.geopbf`, { type: "application/x-geopbf" }));

7. Gint Extension — 64-bit Coordinate Packing

Applying gint encoding to GeoPBF packs each vertex's coordinates into a single 64-bit integer that simultaneously stores both the Morton code and the VW weight.

Packing where (Morton) and which LOD (VW rank) into a single 64-bit integer means both spatial queries and LOD filtering reduce to simple integer arithmetic.

// L1 / L2 check (most-significant bit)
const isAnchor = (code >> 63n) === 1n;

// Extract Morton code (L2: upper 58 bits)
const mortonCode = code & 0x7FFFFFFFFFFFFFF8n;  // bits 3-62

// Extract VW weight rank (L2: lower 6 bits)
const vwRank = Number(code & 0x3Fn);  // bits 0-5 → 0–63

// LOD filter at zoom z (rank threshold corresponding to 1px²)
const minRank = rankFromZoom(z);  // lower threshold at higher zoom (more vertices)
const visible = isAnchor || vwRank >= minRank;

8. Comparison with Other Formats

8.1 Measured File Size

Three public datasets of different shape and scale, each converted and measured end to end on one machine. Every row holds the same features; sizes are decimal MB.

Format	NE 10m admin_0 countries, v5.1.1	KSJ N03 (2026) Japanese municipalities	TIGER 2024 ZCTA5 US ZIP code areas
features	258	125,130	33,791
vertices	548,469	15,669,416	51,258,797
coordinate grid	10⁻⁶° (0.11 m)	10⁻⁶° (0.11 m)	10⁻⁵° (1.1 m)
Shapefile (.shp+.dbf+.shx)	9.7	282.8	825.3
└ +.prj, zipped −9	4.9	174.8	530.5
GML (.xml)	—	580.7	—
└ + gzip −9	—	123.2	—
GeoJSON	13.3	579.6	1,089.6
└ + gzip −9	4.7	138.6	265.8
FlatGeobuf	9.3	266.9	826.8
└ + gzip −9	5.4	156.3	421.9
GeoParquet (WKB, zstd, bbox)	4.5	124.5	333.7
GeoPBF	3.3	48.6	117.8
GeoPBF + gzip −9 (distribution form)	2.7	34.4	95.2

Per vertex, raw → compressed. This is where the formats separate:

Bytes per vertex	NE 10m admin_0	KSJ N03	TIGER ZCTA5
Shapefile	17.7 → 9.0	18.1 → 11.2	16.1 → 10.3
GeoJSON	24.2 → 8.6	37.0 → 8.8	21.3 → 5.2
FlatGeobuf	17.0 → 9.8	17.0 → 10.0	16.1 → 8.2
GeoPBF	6.0 → 4.9	3.1 → 2.2	2.3 → 1.9

Reproduce with npx geopbf enc and npx geopbf parquet; the FlatGeobuf rows use the official flatgeobuf npm writer on the same features, and the zipped-shapefile row is .shp+.dbf+.shx+.prj packed at deflate −9 so that all three datasets are measured the same way — the shipped archives differ in what else they carry (Natural Earth 4.9 MB with an HTML README, TIGER 528.8 MB with ISO metadata XML, KSJ N03 803.2 MB with GML and shapefile and GeoJSON in one archive) and are not directly comparable. The shapefiles hold the original full-precision doubles, but a shapefile spends a fixed 16 bytes per point whatever the precision, so its size rows are unaffected.

8.2 Capability Matrix

Metric	GeoJSON	Shapefile	MVT (tiles)	GeoPBF
Encoding	Text UTF-8	Binary (multi-file)	PBF binary	PBF + delta + Varint
File count	1	.shp + .dbf + .shx + …	zoom × tile count	1
Transfer efficiency	Low (large text)	Medium	Medium (zoom-split)	High (binary + gzip)
Worker transfer	JSON serialize required	Conversion required	ArrayBuffer	ArrayBuffer zero-copy
LOD support	None	None	Per-zoom files	VW weight built-in
Spatial index	None	.shx (linear)	Tile coords	None in the file (Morton 64-bit built in memory on load)
Format conversion	External library needed	External library needed	Limited	12 formats built-in
Topology	None	None	None	Arc sharing · boundary integrity

9. API Reference

// ── Load ───────────────────────────────────────────────
const pbf = await geopbf(input, options);
// input:   File | Blob | ArrayBuffer | GeoJSON object | URL string
// options: { name, gint, nocache, clean }
//   gint:    false → skip topology encode (gint conversion)
//   nocache: true  → bypass IndexedDB cache and re-fetch
//   clean:   true  → run topology integrity clean pass

// ── Data access ────────────────────────────────────────
pbf.length          // feature count
pbf.keys            // attribute name array ['name', 'rank', ...]
pbf.arrayBuffer     // raw ArrayBuffer (for Worker transfer)
pbf.geojson         // convert to GeoJSON FeatureCollection
pbf.topojson        // convert to TopoJSON (topology-preserving)
pbf.getFeature(i)   // decode only the i-th feature

// ── Export ─────────────────────────────────────────────
await pbf.save('name')      // download as .geopbf (gzip)
pbf.shape()               // Shapefile .zip
pbf.kmz()                 // KMZ
pbf.gml()                 // GML string
pbf.gpx()                 // GPX string
await pbf.fgb()           // FlatGeoBuf ArrayBuffer

// ── O(1) metadata update ───────────────────────────────
pbf.header({ name: 'New Name', description: '...', license: 'MIT' });

// ── Render ─────────────────────────────────────────────
pbf.draw(canvasCtx, { fill: '#5a8050', stroke: '#a8d47e' });

10. Map Library Integrations (v1.1)

Since v1.1.0 the npm package ships ready-made adapters for the major web map libraries. None of them import the host library — each is a zero-dependency subpath export built on one shared loader: whole-file gzip is detected by magic bytes, decoding avoids new Function (CSP-safe noeval mode), and properties are mapped to be JSON-safe (Date → ISO string, bbox → plain array, binary values dropped) so they survive each library's worker handoff.

Import	Shape	For
geopbf/maplibre	addProtocol handler	MapLibre GL JS (v4+)
geopbf/leaflet	L.GeoJSON subclass	Leaflet (1.x)
geopbf/openlayers	VectorSource loader factory	OpenLayers (6+)
geopbf/loaders	loaders.gl Loader object	deck.gl · kepler.gl
geopbf/load	plain loadGeopbf(url)	Cesium · D3 · anything that eats GeoJSON

// ── MapLibre — protocol source ─────────────────────────
import { geopbfProtocol } from "geopbf/maplibre";
maplibregl.addProtocol("geopbf", geopbfProtocol);
map.addSource("rail", { type: "geojson", data: "geopbf://https://…/N02-25_RailroadSection" });

// ── Leaflet — layer plugin ─────────────────────────────
import { extendLeaflet } from "geopbf/leaflet";
extendLeaflet(L);
L.geoPBF("geopbf://https://…").on("load", e => console.log(e.meta)).addTo(map);

// ── OpenLayers — source loader ─────────────────────────
import { makeGeopbfLoader } from "geopbf/openlayers";
const source = new VectorSource({ loader: makeGeopbfLoader(url, new GeoJSON()) });

// ── deck.gl — loaders.gl ───────────────────────────────
import { GeoPBFLoader } from "geopbf/loaders";
new GeoJsonLayer({ data: url, loaders: [GeoPBFLoader] });

// ── Everything else — plain loader ─────────────────────
import { loadGeopbf } from "geopbf/load";
const r = await loadGeopbf(url);   // { geojson, name, license, attribution, minZoom, maxZoom, … }
viewer.dataSources.add(await Cesium.GeoJsonDataSource.load(r.geojson));

Attribution travels inside the file (§2), and the adapters honor it: Leaflet and OpenLayers wire the header's attribution into the map credit automatically (an explicit option wins), and loadGeopbf hands back all header metadata for manual wiring. One contract to know: with MapLibre, write the inner URL absolute (geopbf://https://…, pmtiles-style) — MapLibre normalizes source URLs through new URL(), which corrupts relative forms.

11. Untrusted HTML & Sanitizing (v1.2)

GeoPBF is an open format: anyone can produce a file and hand you a URL. Display properties — @tip, @pop, or any property your viewer renders as HTML — must therefore be treated as untrusted input. A crafted file could otherwise run script in your page (XSS) the moment a tooltip or popup opens. The rule is the classic one: sanitize at the output boundary, right before the string reaches innerHTML — never assume a file was cleaned upstream, because files reach viewers without passing through any editor.

// Sanitize at the output boundary — right before innerHTML
import { sanitizeHTML } from "geopbf/sanitize";
el.innerHTML = sanitizeHTML(feature.properties["@pop"]);

sanitizeHTML is a dependency-free allowlist filter: formatting tags, lists, tables, images and links pass through; script, iframe, svg and the like are dropped with their content; unknown tags are unwrapped, keeping their children; event handlers and style never survive. href accepts http(s)/mailto, img src accepts http(s)/data:image, and every link gets target="_blank" rel="noopener noreferrer". It touches the DOM only when called, so importing it in a worker is safe. Note the division of labor with §10: sanitizeProperties makes values JSON-safe (types), sanitizeHTML makes them display-safe (markup). Use both.

12. COG — Rasters, the Same Way (v1.4)

A COG (Cloud Optimized GeoTIFF) is a static file read by HTTP Range requests — the same no-server philosophy as GeoPBF, applied to rasters. v1.4's geopbf/cog is a hand-written reader with zero new dependencies: the header is fetched in one 16 KB range request, tile requests are sorted and adjacent ranges coalesced into a single request, decode and reprojection run in a worker pool, and JPEG/WebP tiles go straight to the browser's native decoder. cog.metrics() reports the numbers at any time (range requests, coalesce ratio, decode time).

import { openCog } from "geopbf/cog";
const cog = await openCog("https://…/TCI.tif");   // header in one range request
const bm  = await cog.renderXYZ(14, x, y);          // ImageBitmap, warped to Web Mercator

The supported subset follows what public COG feeds actually serve: tiled and stripped layouts, BigTIFF, compression none/deflate/LZW/JPEG/WebP, predictor 2, uint8 RGB(A)/palette/single-band uint16/int16/float32 (auto percentile stretch, GDAL_NODATA → transparent), CRS EPSG:4326/3857/UTM (hand-written Krüger n-series, nm accuracy). Anything outside fails with an explicit error pointing to gdal_translate -of COG. One line each for MapLibre (cog:// protocol via geopbf/maplibre-cog) and Leaflet (geopbf/leaflet-cog). The CLI prints structure and measured numbers: npx geopbf cog info <url> --bench.

13. PMTiles & GeoParquet — the Way Out (v1.5)

Through v1.4 the hub read formats. v1.5 opens the other direction: a GeoPBF leaves as a PMTiles archive of Mapbox Vector Tiles or as a GeoParquet file, and a GeoParquet comes back in. Still no new dependencies — in fact pako was removed. Every codec now goes through the platform (CompressionStream in browsers and workers, node:zlib in Node), leaving pbf as the only runtime dependency — and 1.6 dropped that one too, by bringing the protobuf wire reader/writer in-house. The dependency list is now empty.

# CLI — plain Node, no native build
npx geopbf pmtiles countries.geopbf countries.pmtiles --maxzoom 10
npx geopbf parquet countries.geopbf countries.parquet
npx geopbf parquet2pbf countries.parquet back.geopbf     # and back again

import { toPMTiles } from "geopbf/pmtiles";
import { toGeoParquet } from "geopbf/geoparquet";

const pbf = await geopbf(file, { gint: true });
const { buffer, stats } = await toPMTiles(pbf, { maxZoom: 12 });  // stats.engine → "gpu" | "cpu"
const pq = await toGeoParquet(pbf);                      // pq.buffer, pq.geo

Tiles are cut from Gint, not from the raw features. That is the same derived buffer ortho-earth draws on the GPU: a shared border exists once, as a single arc, and every vertex carries its Visvalingam–Whyatt rank. Per-zoom simplification is therefore a rank >= threshold filter, which means two neighbouring polygons are necessarily reduced to the identical vertex list — no slivers, no gaps, at any zoom. The thresholds are calibrated per arc against a Douglas–Peucker decomposition, so vertex counts land within roughly 10 % of tippecanoe's while which vertices survive is still decided by rank, keeping both sides of a border in step.

On Natural Earth 10m countries (258 polygons, 480k vertices, z0–10, 4-core Node) that is about 6.5 s from GeoJSON for 573,896 tiles / 151.0 MB, against tippecanoe v2.82 at 58 s for 573,885 tiles / 147 MB — the same tile set within a handful, with interior tiles byte-for-byte the same size apart from the layer name and the feature id. GDAL's PMTiles driver (3.12) took 780 s for the same job. On US Census ZCTA5 (33,092 ZIP areas, 52M vertices, z0–12): 47 s for 238,322 tiles / 251 MB against tippecanoe's 251 s for 238,319 tiles / 246 MB, indistinguishable in MapLibre at z3/z7/z11.

Parquet rows come out spatially ordered. Feature bbox centres are packed Sort-Tile-Recursive style (order: "str", the default), so each row group's bbox statistics cover a disjoint patch and any reader that pushes an area filter down — DuckDB, pyarrow datasets, GeoPandas — fetches only the row groups that actually intersect it. On 1,000,000 points in 50 row groups the row-group overlap ratio runs 24.5 unsorted → 0.87 Morton → 0.28 Hilbert → 0.00 STR, and a city-sized query touches 1–2 row groups instead of all 50. This follows Kanahiro Iguchi's Spatial sort for well-packed GeoParquet (CNG Japan 2026).

Where the GPU is, honestly. Projection, the per-zoom rank filter, the integer→double conversion and the bbox reduction run on WebGPU when it is available, with a CPU path performing the same integer arithmetic when it is not — the output is byte-identical either way, verified kernel by kernel in headless Chromium. But those stages are only 3–6 % of the job on every dataset measured so far; ring assembly, clipping, MVT encoding and gzip dominate. The GPU keeps the browser's main thread free. It is not the reason the converter is fast — that comes from Gint and from the assembly stage.

And back again. fromGeoParquet() turns a GeoParquet into a GeoPBF using a dependency-free Parquet reader — Thrift compact footer, DataPage v1/v2, PLAIN and dictionary encodings, codecs none / snappy / gzip / zstd — plus a WKB parser. A file geopbf wrote comes back bit-identical, and files written by GeoPandas, pyarrow and DuckDB read back to the same features. The CRS must be lon/lat (CRS84 / EPSG:4326) unless ignoreCrs is passed. PMTiles has no such inverse: its tiles are simplified and quantized, so only an approximate reassembly would be possible, and this package does not pretend otherwise.

---