Skip to content
- Two lists, one ranking: meaning (EmbeddingGemma 2: paraphrases, other languages) and exact words (FTS5/BM25:
function names, error messages), fused with Reciprocal Rank Fusion. Each result shows where it came from, e.g.
(meaning #1 · keywords #3). Keyword search skips stop words and matches Polish inflections by stem.
Letters match without their accents and strokes in any language: lodz finds Łódź, soren finds Søren,
strasse finds Straße.
- One folder only:
in: and a folder name keeps results to files under folders whose name starts with it, in the
panel, Raycast, Alfred and the CLI: in:down umowa najmu, in:Studia, in:"Moje dokumenty" faktura, or a path,
in:~/Projects/lantern readme. The folder word is never spell-checked or sent to the model; apps, settings and
answers are left out, since they are not in a folder.
- Indexed types: code (Python, JS/TS, Go, Rust, Java, Kotlin, Swift, C/C++, C#, Ruby, PHP and more, split into
functions and classes), text (Markdown, txt, JSON, YAML, HTML…; text files over 2 MB and binaries are skipped),
PDFs, audio (mp3, m4a, wav, flac, ogg, opus, aac, aiff…), video (mp4, mov, mkv, webm, avi…: sound and frames) and
photos (jpg, png, heic, gif, webp, tiff, bmp).
- PDFs are read page by page with Apple PDFKit; each result shows its page (
report.pdf (page 12)). Pages without
a text layer (scans) are read with OCR. Password-protected PDFs are skipped; very large ones are capped (1000 pages,
100 scanned pages).
- Text in images is read by macOS itself (the Vision framework behind Live Text: on device, Polish and English,
no model to download, ~110 MB while running) in a cascade: screenshots and scans by name go straight to reading;
every other image first passes a fast check (~10 ms) for at least two real words, so ordinary photos cost almost
nothing; low-confidence lines, symbol garbage and camera watermarks are dropped.
- The text becomes searchable by exact words (
0x80070002, a VIN, an invoice number, 245,30 zł) and by meaning;
the photo appears in the Photos section with the matching text under it (“text in image”). Numbers and codes with
separators (187,15, FV/2025/09/0932) must match as a whole, not as loose parts.
- The photo models alone already find images with text by meaning (“faktura za prąd”, “błąd aktualizacji
windows”); OCR adds exact strings and small print. Measured: DECISIONS.md, “OCR research” and Phase 11.
- Off switch: Settings → File Types → Text in photos and screenshots, or
lantern config index_image_text false.
- Paste a line of code or just a name:
def add_two(x, y):, add_two, addTwo, class Client, Client::connect,
BinMapper.transform, public void testReplaceIndex(). The result is the file, the exact line and the
signature, e.g. script.py:200 def add_two(x, y).
- Names match regardless of style (
add_two = addTwo = AddTwo): exactly first, then by prefix (add_two →
add_two_numbers) and by fragment (UserToken → parseUserToken). The number of arguments, the kind (def,
class…) and the parent class help pick the right definition. Sentences (“tree pruning”) never list definitions.
- Languages: Python, JavaScript, TypeScript, Go, Rust, Java, Kotlin, Swift, C, C++, Objective-C, C#, Scala, Ruby,
PHP, Lua, R, Dart, Elixir, Haskell, SQL, Bash (each covered by a test). Only definitions are searched, not calls.
- Find a photo by what is in it (“dog”, “bicycle”, “child in the water”, “sunset”), in English or Polish, or by
its file name.
- Two models rate every photo: the EmbeddingGemma 2 vision tower (better at sentences and Polish) and SigLIP 2 base
(better at single words). Their scores are normalised and summed; a photo is listed only when it clearly stands
out from the rest.
- Videos: frames at scene changes (macOS’s AVFoundation; ffmpeg for MKV/WebM), at least one every 30 s, near-duplicates dropped before embedding. A
video is found by what is seen in it even when nobody speaks (“sea”, “beach”), at the moment it is seen; text on
its frames (slides, screen recordings, captions) is read like text in photos and found at its moment too.
- Skipped: images under 96 px (icons), over 100 MB, or broken. HEIC is decoded with the system’s
sips.
- Speed (M4): ~3 photos/s including model loading, i.e. ~1 hour for 10,000 photos in the background.
- macOS (AVFoundation, through Lantern’s
lantern-media helper) streams the audio (16 kHz mono) window by window, so
the length of a recording does not affect memory. MKV and WebM, which macOS cannot read, go through ffmpeg if it is
installed; measured equal for search (docs/DECISIONS.md, “Phase 14”).
- Two tracks in one ranking: audio embeddings of 30 s windows with 5 s overlap (the meaning of what is said,
also across languages) and Whisper large-v3-turbo transcripts in ~20 s chunks (single words, text previews).
lantern index <folder> --no-transcribe skips Whisper (faster, but single words are not found).
