Inputs larger than memory

manby keeps its features in memory, which is much of why it is fast, and its peak memory is still below tippecanoe’s on every benchmark. For inputs larger than memory, --memory-budget bounds it. The default is half the physical memory, as tippecanoe sizes its sorting.

manby --memory-budget=1G -zg --drop-densest-as-needed -o points.pmtiles points40m.geojsonl

Past the budget, the features and the tile data live in an unlinked temporary file in the -t directory, mapped into memory and laid out in the order the tiling reads them, and manby keeps its resident memory near the budget. Under a budget, the tiles being made take most of what is resident, so the biggest tiles run together only as far as three quarters of the budget allows.

Inputs several times larger than a 1 GB budget, once each, against tippecanoe 2.82.0.

Seconds, with a 1 GB memory budget for manbymanbytippecanoe

40 million points, macOS

79 s, 8.3x faster

652 s

40 million points, Linux

121 s, 8.5x faster

1,025 s

The tracts 20 times over, macOS

137 s, 4.1x faster

564 s

The tracts 20 times over, Linux

269 s, 3.2x faster

854 s

The figures as a table

Benchmark	tippecanoe	manby	Speedup
40 million points, macOS	652 s	79 s	8.3x
40 million points, Linux	1,025 s	121 s	8.5x
The tracts 20 times over, macOS	564 s	137 s	4.1x
The tracts 20 times over, Linux	854 s	269 s	3.2x

Peak memory in MB, with a 1 GB memory budget for manbymanbytippecanoe

40 million points, macOS

1,139 MB

10,206 MB

40 million points, Linux

1,188 MB

6,379 MB

The tracts 20 times over, macOS

1,022 MB

4,810 MB

The tracts 20 times over, Linux

1,206 MB

3,730 MB

The figures as a table

Benchmark	tippecanoe	manby
40 million points, macOS	10,206 MB	1,139 MB
40 million points, Linux	6,379 MB	1,188 MB
The tracts 20 times over, macOS	4,810 MB	1,022 MB
The tracts 20 times over, Linux	3,730 MB	1,206 MB

The tiles are tippecanoe’s for both. The benchmarks give the commands and describe the machines.