Installation and Building¶

Table of Contents¶

Installing
Python CLI
Building
Running the extension

Installing¶

MIINT is available in the DuckDB Community Extensions repository:

INSTALL miint FROM community;
LOAD miint;

You can verify the installed version with:

SELECT miint_version();

Installing from a local build¶

If you are building MIINT from source (see Building), launch DuckDB with the allow_unsigned_extensions option:

CLI:

duckdb -unsigned

Python:

con = duckdb.connect(':memory:', config={'allow_unsigned_extensions' : 'true'})

NodeJS:

db = new duckdb.Database(':memory:', {"allow_unsigned_extensions": "true"});

Then load the extension binary directly:

LOAD '/path/to/miint.duckdb_extension';

Python CLI¶

A lightweight Python CLI (miint) wraps the DuckDB extension for common bioinformatics workflows: format conversion, sequence alignment, and feature table generation. All computation happens in SQL — Python handles argument parsing and query construction.

Installing from a local checkout¶

cd python
pip install -e .

This requires the duckdb Python package (installed automatically as a dependency). The CLI is not published to PyPI at this time.

Usage¶

# Use a local extension build
miint --extension-path ./build/release/extension/miint/miint.duckdb_extension \
    convert sequence -1 reads.fastq.gz -o reads.parquet

# Without --extension-path, the CLI installs miint from community extensions
miint convert alignment -i alignments.bam -o alignments.parquet

Available commands¶

miint convert sequence    Convert FASTQ/FASTA to parquet
miint convert alignment   Convert SAM/BAM to parquet
miint convert biom        Convert BIOM to parquet
miint convert parquet     Convert parquet to FASTQ/FASTA/SAM/BAM/BIOM

miint transform genome-coverage   Compute genome coverage from alignments
miint transform woltka-ogu        Generate OGU feature table (with optional filters)

miint align minimap2              Align sequences with minimap2
miint align minimap2-sharded      Sharded minimap2 alignment with RYpe classification

Run miint --help or miint <command> --help for detailed usage.

Building¶

System prerequisites¶

These are the system-level tools the build invokes directly. CI installs the same set via duckdb/extension-ci-tools/.github/workflows/_extension_distribution.yml (with extra_toolchains: 'rust;omp'); see .github/workflows/MainDistributionPipeline.yml for the upstream reference.

macOS (Homebrew):

brew install ninja pkg-config autoconf automake libtool autoconf-archive libomp

- ninja — build driver (bash build.sh runs GEN=ninja make). - pkg-config — required by vcpkg's openssl port (transitively pulled in by curl / rocksdb). - autoconf, automake, libtool, autoconf-archive — vsearch ships autogen.sh; without these the configure step fails with autoreconf: command not found. - libomp — required when MIINT_ENABLE_UNIFRAC=ON (the default). Apple Clang doesn't ship libomp, and the CMake check looks for Homebrew's libomp.a at /opt/homebrew/opt/libomp (Apple Silicon) or /usr/local/opt/libomp (Intel). Pass -DMIINT_ENABLE_UNIFRAC=OFF to build without UniFrac and skip this.

Linux (Debian/Ubuntu):

sudo apt-get install -y ninja-build pkg-config autoconf automake libtool autoconf-archive

OpenMP comes from libgomp shipped with gcc, so no separate install is needed for UniFrac.

Windows: see CI for the reference setup. The windows_amd64 MSVC and windows_amd64_rtools builds are currently excluded from CI (see exclude_archs in MainDistributionPipeline.yml); windows_amd64_mingw is the supported path.

Managing dependencies¶

Rust toolchain (required): The RYpe sequence classification library, the embedded sylph profiler, the sc random-forest library and SourceTracker3 (sourcetracker) are written in Rust. Install Rust via rustup:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

VCPKG (required): DuckDB extensions use VCPKG for dependency management. Enabling VCPKG is very simple: follow the installation instructions or just run the following:

git clone https://github.com/Microsoft/vcpkg.git
./vcpkg/bootstrap-vcpkg.sh
export VCPKG_TOOLCHAIN_PATH=`pwd`/vcpkg/scripts/buildsystems/vcpkg.cmake

Submodules (required): Several embedded libraries (MAFFT, vsearch, sylph, st3, scikit-bio-binaries, unifrac-binaries, etc.) are git submodules. After cloning:

git submodule update --init --recursive

Build system¶

We use Ninja for quick builds. The easiest install is to use prebuilt release binaries, or brew install ninja / apt-get install ninja-build as listed above.

Optional feature flags¶

Pass these to CMake via EXT_FLAGS (e.g. EXT_FLAGS="-DMIINT_ENABLE_UNIFRAC=OFF" bash build.sh) to opt out of optional subsystems:

Flag	Default	What it controls
`MIINT_ENABLE_CURL`	`ON`	libcurl-based streaming uploads (auto-disabled on macOS + WASM)
`MIINT_ENABLE_HDF5`	`ON`	HDF5/BIOM reader and writer
`MIINT_ENABLE_MAFFT`	`ON`	MAFFT multiple sequence alignment
`MIINT_ENABLE_SORTMERNA`	`ON`	SortMeRNA rRNA alignment (requires RocksDB)
`MIINT_ENABLE_SYLPH`	`ON`	sylph FracMinHash relative-abundance profiling (Rust)
`MIINT_ENABLE_ST3`	`ON`	SourceTracker3 microbial source attribution, `sourcetracker()` (Rust)
`MIINT_ENABLE_GPL_BOUNDARY`	`ON`	gpl-boundary subsystem (process pipes + Arrow IPC over POSIX shm)
`MIINT_ENABLE_UNIFRAC`	`ON`	UniFrac (PCoA / PERMANOVA / Faith PD) — requires libomp on macOS
`MIINT_ENABLE_KREPP`	`ON`	krepp phylogenetic placement and index building (`place_krepp`, `krepp_index_create`) — requires the parallel-hashmap and boost-math headers from vcpkg

WASM (Emscripten) builds automatically disable HDF5, SortMeRNA, sylph, gpl-boundary, krepp, MAFFT, and libcurl. MAFFT is off because its fixed-size local arrays need 2–4 MB stack frames and duckdb-wasm runs with a 1 MB stack. UniFrac, vsearch, rype and SourceTracker3 remain enabled. The Rust crates use the C++ side's exception model, so on wasm_eh, the only WASM platform miint ships, they are built with WebAssembly exceptions (see the Rust glue); SourceTracker3 runs its sampler on the calling thread there. UniFrac builds against a dedicated single-threaded WASM target (libssu_wasm.a / libskbb_wasm.a, Eigen-backed, OpenMP stubbed out), and vsearch runs its work on the calling thread (src/include/vsearch_serial.hpp) because its threaded entry points cannot start threads there; results are identical to native.

Build steps¶

Now to build the extension, run:

bash build.sh

This is equivalent to:

export VCPKG_TOOLCHAIN_PATH=`pwd`/vcpkg/scripts/buildsystems/vcpkg.cmake
GEN=ninja make

The main binaries that will be built are:

./build/release/duckdb
./build/release/test/unittest
./build/release/extension/miint/miint.duckdb_extension
./build/release/extension/miint/tests
./build/release/extension/miint/tests_duckdb

duckdb is the binary for the DuckDB shell with the extension code automatically loaded.
unittest is the DuckDB test runner with the extension linked in (for SQL logic tests).
miint.duckdb_extension is the loadable binary as it would be distributed.
tests is the Catch2 C++ unit test runner.
tests_duckdb runs the C++ tests that need a running DuckDB.

Running the extension¶

To run the extension code, simply start the shell with ./build/release/duckdb.