Reproducibility

Goal: regenerate a signature from the same input, and check that two sketches describe the same k-mer set without diffing the files themselves.

Sketching is deterministic: the same input, sketched with the same parameters, produces the same hashes and the same checksum. Nothing in a sketch is random — the hash seed is a compile-time constant, and every signature carries a canonical checksum that is stable across a save and reload.

You will learn:

• Why snipe sketching is deterministic — canonical murmur3 with seed 42, a fixed --scale, and no hidden state.
• Where the canonical checksum lives in a .snipesig, and how snipe info surfaces it.
• Why matching k1 and scale is required before two signatures can be compared at all.
• How to verify reproducibility from the CLI and from Python.

1. Understand why sketching is deterministic.

   Two things make a snipe sketch reproducible, and both are fixed rather than chosen at runtime:

   • The hash is a pure function of the k-mer. Every k-mer is reduced to one 64-bit number by canonical murmur3 with seed 42 — snipe hashes the lexicographically smaller of the k-mer and its reverse complement, so the DNA strand you happened to read from never changes the answer. The seed is a compile-time constant (MURMUR3_HASH_SEED = 42); no CLI flag or API argument can move it. Two runs, two machines, two versions of snipe all agree on the hash of a given k-mer. See Hashing & canonical k-mers.
   • The sample is a fixed fraction, chosen by a fixed rule. FracMinHash keeps a k-mer only when its hash falls below u64::MAX / scale. Because u64::MAX is constant, the keep-or-drop decision is a deterministic function of --scale alone — the same k-mer makes the same decision in every file it appears in. See FracMinHash & scale.

   Same input + same k1/k2 + same scale → same hashes → same signature.

2. Sketch with explicit parameters.

   Reproducibility is easiest to reason about when the parameters are written down rather than left implicit. The defaults are k1 = 51, k2 = 53, and scale = 10000; the tiny phiX174 example below uses --scale 1 so every k-mer is kept (the right choice only for a tiny input).

   snipe sketch --sample phix.fasta -o phix.snipesig --scale 1

   The Python API takes the same knobs:

   import snipe
   
   sig = snipe.sketch(["phix.fasta"], k1=51, k2=53, scale=1)
   sig.save("phix.snipesig")

   Re-run either form on the same input and you get the same signature — same hashes, same abundances, same checksum. The .snipesig format stores the full signature, so a sketch written today reloads with the same checksum tomorrow.

3. Read the canonical checksum.

   Every signature carries a single canonical identity: k1_checksum, an md5 computed over the signature’s sorted, distinct K1 hash set. It is abundance-independent — two signatures with the same K1 hashes but different counts share a checksum — so it answers “are these the same k-mer set?” reliably, and it is stable across a save and reload. It lives in the .snipesig Parquet footer alongside the header, and snipe info surfaces it directly:

   snipe info phix.snipesig

   snipe info phix.snipesig

   
   === Signature Information ===
   Version: 2.0.0
   Created: <at sketch time>
   K1 length: 51
   K2 length: 53
   K2 extension length: 2 bases
   Scale: 1
   Hash threshold: 18446744073709551615
   Hash space coverage: 100.000000%
   Theoretical sampling rate: 1/1 = 1.000000
   
   === Input Files ===
     1: phix.fasta
   
   === K-mer Statistics ===
   Unique K1 k-mers: 5336
   Total K1 abundance: 5336
   Average K1 abundance: 1.00
   Unique K2 k-mers: 5334
   Total K2 abundance: 5334
   Average K2 abundance: 1.00
   Unique k-mer edgemers: 5334
   Total edgemer abundance: 5334
   Average pair abundance: 1.00
   K1 checksum (md5): 85e65cc582d48cbf2fa1b2b7c9b552c7
   K2 checksum (md5): 91adce917f75e09fb1d6790e1619d446
   Failed K2 pairs: 2 (total abundance: 2)
   K2 extension success rate: 99.96%
   K1s processed for K2 extension: 5336
   
   === Processing Statistics ===
   Processing time: 0.02s
   Sequences processed: 1
   Total bases processed: 5386
   Average sequence length: 5386.0 bp
   K1 positions examined: 5336
   K1 hashes generated: 5336
   K1 hashes below threshold: 5336
   K1 hashes above threshold: 0
   Hash efficiency: 100.00%
   K1 with valid extensions: 5334
   K1 without extension space: 2
   

   The K1 checksum (md5) line — here 85e65cc582d48cbf2fa1b2b7c9b552c7 — is the signature’s identity. Sketch the same input the same way and you get the same checksum, every time. In Python the same value is one property away:

   sig = snipe.load("phix.snipesig")
   print(sig.md5)

   Note

   sig.md5, the info output, and the value in the Parquet footer are all the same canonical checksum — computed once from the sorted unique K1 hashes. That single number is the signature’s identity across the whole toolchain; see The .snipesig format.

4. Verify two sketches match.

   To prove reproducibility, compare checksums, not files. Two signatures with the same K1 checksum (md5) describe the same K1 hash set. From the CLI, run snipe info on each and read off the checksum line. From Python, compare the property directly — Signature equality is defined on this canonical checksum:

   a = snipe.sketch(["phix.fasta"], scale=1)
   b = snipe.sketch(["phix.fasta"], scale=1)
   assert a.md5 == b.md5   # same input, same params -> identical identity

   Comparing the checksum is stronger than comparing bytes for the question that matters — it ignores incidental metadata (like the creation timestamp) and answers precisely “are these the same k-mer set?”

5. Match k1 and scale before comparing.

   Reproducibility across different samples only holds when they were sketched by the same rule. Two signatures built at different scale values sampled different fractions of the hash space, and two built at different k1 describe different-length features — in neither case are the retained hashes drawn from a comparable sub-space. snipe therefore requires matching k1 and scale before it will combine or compare two signatures; a mismatch raises an incompatible-signatures error rather than returning a wrong answer.

   # These are comparable only because k1 and scale match on both sides.
   snipe qc --reference phix_ref.snipesig --sample phix.snipesig -o qc.tsv

   Caution

   You can only ever downscale an existing signature (raise its scale), never recover hashes a higher scale already dropped. So pick one scale for a whole study up front and sketch everything with it — that is the single most important habit for keeping a project’s signatures mutually comparable.

Recap

snipe sketches are deterministic because the hash is a pure function of the k-mer (canonical murmur3, seed 42, fixed at compile time) and the sample is a fixed fraction set by --scale alone. Same input, same k1/k2/scale → same hashes → same signature. Every .snipesig carries one canonical k1_checksum in its Parquet footer; snipe info and sig.md5 both surface it, and it is the reliable way to check that two sketches describe the same k-mer set. Comparing signatures requires matching k1 and scale, so choose a scale once and keep it across a study.

Next

• Hashing & canonical k-mers → — how a k-mer becomes its deterministic number.
• FracMinHash & scale → — why matching scale is non-negotiable.
• The .snipesig format → — where the checksum and header live.
• snipe info reference → — the full inspector surface.
• Python API reference → — sketch, load, and Signature.md5.