TL;DR — Existing LVLM fingerprints are easy to detect and erase because they are semantically abnormal. SIF keeps the ownership signal in-distribution, achieving state-of-the-art stealthiness and robustness with no parameter changes.
The public accessibility of Large Vision–Language Models (LVLMs) raises concerns about unauthorized reuse. Existing ownership-verification methods rely on semantically abnormal queries or out-of-distribution responses, which adversaries can easily detect and remove. We first expose this via the Semantic Divergence Attack (SDA), which filters fingerprint checks by measuring semantic divergence from a reference model. To address it, we propose SIF (Semantically In-Distribution Fingerprints), a non-intrusive framework. Semantic-Aligned Fingerprint Distillation (SAFD) distills text-generation watermark signals into the visual modality for semantically coherent yet fingerprinted responses, and Robust-Fingerprint Optimization (RFO) simulates worst-case representation perturbations for resilience to fine-tuning and quantization. Experiments on LLaVA-1.5 and Qwen2.5-VL show that SIF achieves superior stealthiness and robustness.
Prior fingerprints are semantically abnormal by design, so a stealer can run a small reference LVLM beside the stolen model: abnormal prompts (high perplexity) or outputs that diverge from the reference are flagged and replaced with the reference response — scrubbing the fingerprint.
Distills a decoding-based text watermark into the input image. A small perturbation (16/255) is optimized so the model generates green-list watermark tokens while keeping the response close to a natural teacher output — so the reply is natural yet carries a detectable signal.
Stealers quantize or fine-tune models, which shifts internal representations and breaks fragile fingerprints. RFO simulates worst-case representation perturbations during optimization to make the trigger more robust, so the fingerprint survives such model modifications.
In practice, a stolen model is often modified before deployment. Across quantization (4-/8-bit) and full fine-tuning on six downstream datasets, SIF consistently retains the highest fingerprint matching rate on both LLaVA-1.5-7B and Qwen2.5-VL-7B, while prior methods are largely erased.
@InProceedings{Zhao_2026_CVPR,
author = {Zhao, Yifei and Lou, Qian and Zheng, Mengxin},
title = {SIF: Semantically In-Distribution Fingerprints for Large Vision-Language Models},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2026},
pages = {17399-17408}
}