One-sentence Summary

Researchers from MIPT et al. present GigaEvo and OpenEvolve, an evolutionary attack framework that targets perceptual hash algorithms (PHAs) with black-box, query-efficient, low-distortion adversarial images, achieving composite score reductions of up to 41.2% on NeuralHash across four deployed PHAs and exposing vulnerabilities in content-moderation pipelines without requiring gradient information.

Key Contributions

• A black-box evolutionary framework that combines GigaEvo and OpenEvolve performs targeted second-image attacks on perceptual hash algorithms without requiring internal knowledge of the hash functions, naturally handling their non-differentiable and discretized outputs.
• Experiments on four widely deployed perceptual hash algorithms (pHash, PDQ, PhotoDNA, NeuralHash) across 30 ImageNet image pairs show that the evolutionary approach attains comparable or better attack success rates than existing black-box baselines while using substantially fewer queries and producing adversarial images with lower L2 distortion.
• The best evolved programs reduce a composite score (jointly capturing attack success, query count, and distortion) relative to the best optimized seed by 41.2% for NeuralHash, 38.3% for PDQ, 34.0% for pHash, and 8.1% for PhotoDNA, revealing previously unreported vulnerabilities in content-moderation pipelines.

Introduction

Perceptual hash algorithms (PHAs) map images to compact binary strings for fast near-duplicate detection, copyright enforcement, and safety scanning, where they must remain stable under benign edits (resizing, compression) while distinguishing distinct content. Prior work established that PHAs resist common non-adversarial transformations but are vulnerable to white-box gradient-based attacks, and that even large hash spaces can collapse for certain semantic classes (e.g., faces). A key obstacle to systematic robustness guarantees is that PHAs combine continuous feature extraction with discrete quantization, causing small feature changes to produce large Hamming gaps when thresholds are crossed, yet no per-instance theory had been developed. The authors address this gap by proposing a black-box evolutionary attack framework, built on GigaEvo and OpenEvolve, that searches for adversarial perturbations to make a source image’s hash closely match a designated target hash. Evaluating against pHash, PDQ, PhotoDNA, and NeuralHash, the approach achieves higher attack success rates and lower L2 distortion than existing black-box baselines, demonstrating that evolutionary search naturally handles the non-differentiable, discrete output of PHAs without requiring gradient information or internal algorithm knowledge.

Dataset

Here is a concise dataset description structured around the requested points.

• Composition and sources The dataset consists of 30 image pairs sampled from ImageNet.

• Subset details There is a single, fixed benchmark subset of 30 image pairs. No additional filtering rules are described beyond the initial sampling from ImageNet. Each image is resized to 256 × 256 pixels using bilinear interpolation and remains in RGB format. Pixel values are used directly when computing the L2 distance metric.

• Usage in the paper The authors use the same 30 pairs for hyperparameter optimization, for program evolution (an automated search process), and for the final reported comparisons. Because the same small set serves both search and evaluation, the benchmark functions as a controlled testbed rather than a deployment-scale generalization test. No separate training or validation split exists; the dataset is not used for supervised model training but as a fixed evaluation environment for algorithm search.

• Processing details No cropping strategy or additional metadata construction is mentioned. The only preprocessing is bilinear resizing to 256×256 and keeping the RGB representation. L2 distances are computed directly on the resized pixel values.

Method

The authors formulate the targeted second-image attack as a black-box optimization problem. Given a source image $x_s$xs​, a target image $x_t$xt​, and a target hash $h_t = \mathcal{H}(x_t)$ht​=H(xt​), the attacker must produce an adversarial image $x'$x′ close to $x_s$xs​ that satisfies

$$d(\mathcal{H}(x'), h_t) \leq \tau_0,$$d(H(x′),ht​)≤τ0​,

where $d$d is Hamming distance for binary hashes and $L_1$L1​ distance for PhotoDNA descriptors. The benchmark only uses the target image to compute $h_t$ht​; attacks receive the source image, the target hash through an oracle, the matching threshold, and query feedback. They do not have access to gradients, internal hash features, or the target image itself. The matching threshold $\tau_0$τ0​ is fixed per hash function and is not optimized by the attack: 12 for pHash, 92 for PDQ, 3855 for PhotoDNA, and 17 for NeuralHash. A run is considered successful only when the externally recomputed distance falls within the corresponding threshold.

The optimization process is guided by a hash similarity loss evaluated over a set of image pairs. The attack success rate (ASR) is defined as

$$\mathrm{ASR} = \frac{1}{N} \sum_{i=1}^{N} \mathbf{1}\!\left[ \min_j d(\mathcal{H}(x_{i,j}'), h_{t,i}) \leq \tau_0 \right],$$ASR=N1​i=1∑N​1[jmin​d(H(xi,j′​),ht,i​)≤τ0​],

where $j$j indexes candidates observed during the attack. The oracle maintains the best hash-distance candidate throughout a run, and the attack returns an $x_{\mathrm{best}}$xbest​ image for external evaluation. This design avoids penalizing attacks whose final iterate is worse than an earlier observed candidate.

Visual distortion is measured by pixel-space $L_2$L2​ distance and LPIPS, while query cost counts the number of hash-oracle calls. The primary ranking criterion is an implementation score

$$S = \frac{\overline{L_2} + 0.02\,\overline{Q}}{\max(\mathrm{ASR}, 0.05)},$$S=max(ASR,0.05)L2​​+0.02Q​​,

with a technical zero-success penalty added: $S \leftarrow S + 10000 \cdot \mathbf{1}[\mathrm{ASR} = 0]$S←S+10000⋅1[ASR=0]. Lower scores are better. The means are computed over all evaluated pairs, including failures. The denominator penalizes low-ASR attacks, while the numerator favors lower $L_2$L2​ distortion and fewer queries. The coefficient $0.02 = 1/50$0.02=1/50 equates 50 extra oracle calls to one unit of $L_2$L2​ cost. LPIPS is reported separately and is not part of $S$S in the current benchmark tables.

All attacks are executed through a common functional interface:

$$\text{run\_attack}(x_s, O, \theta, B, r) \rightarrow R,$$run_attack(xs​,O,θ,B,r)→R,

where $O$O is the hash oracle, $\theta$θ are attack parameters, $B$B is the query budget, $r$r is a random seed, and $R$R contains the best image found by the attack. This simple contract is used for both hand-written baselines and LLM-generated Python programs.

The oracle is the sole source of feedback. It stores the target hash, clips candidate images to the valid pixel range, counts queries, enforces the budget, and returns only the current distance to the target hash. This constitutes a graded-distance black-box oracle, not a hard-label match/no-match API. The initial distance from $x_s$xs​ to $h_t$ht​ is computed when the oracle is created but is not counted as an attack query. During a run, the oracle maintains a best-so-far candidate by hash distance.

The evaluator acts as the final source of truth. After an attack returns, it recomputes $\mathcal{H}(x_{\mathrm{best}})$H(xbest​), success, final hash distance, $L_2$L2​ distortion, LPIPS (when requested), query count, and score. It does not trust success flags or metrics reported by the attack itself. This is especially important for program evolution: generated code can search for better candidates, but it cannot improve the reported result by altering bookkeeping or output formatting. The same evaluator is used for original baselines, optimized configurations, and evolved programs.

Configuration optimization is performed independently for each hash using the Optuna framework with a tree-structured Parzen estimator sampler. The first phase searches for configurations that reach the threshold reliably; the second phase ranks configurations by the score in Eq. 3. This two-phase design avoids selecting visually clean attacks that rarely collide, while still preferring lower distortion and fewer queries once success is achieved. The resulting optimized configurations form the seed context for the LLM-evolution stage.

For program evolution, GigaEvolve and OpenEvolve receive the optimized attack implementations, empirical notes, and the evaluator contract. Candidate programs are generated by LLM mutation and executed on the benchmark image pairs. A valid candidate must implement the interface in Eq. 5; invalid programs, runtime failures, and malformed results are assigned penalty scores by the evaluator. OpenEvolve maximizes a framework-native score, so the adapter exposes $-S$−S while preserving all original metrics for analysis. GigaEvolve uses the same evaluator and stores candidate lineage during search. The evolved changes are algorithmic rather than cosmetic: the search basis, update schedule, restarts, clipping strategy, plateau handling, and post-success refinement can all be modified by the generated code, but the scoring function and benchmark pairs remain external to the candidate program.

The seed algorithm portfolio contains five attacks. SimBA tests signed basis directions greedily. NES estimates a gradient from random perturbations. ZO-SignSGD is a local zeroth-order sign-gradient baseline. The Prokos seed is an adaptation of the targeted perceptual-hash attack family introduced by Prokos et al. ATKScopes uses multiresolution perturbations optimized with Adam and is the strongest scale-aware seed in the portfolio. Each seed attack is evaluated in an original configuration and in a per-hash optimized configuration.

Experiment

The study evaluates a three-stage attack pipeline: first, hyperparameter optimization of hand-written seed attacks to maximize attack success rate; second, joint optimization of visual quality and query efficiency using a composite score; and third, LLM-driven evolutionary search that mutates full Python attack programs on a fixed 30-image-pair benchmark under hash-specific query budgets. The experiments compare evolved programs against optimized seeds and several soft-label baselines, finding that program evolution consistently improves the composite score over seed tuning alone, though improvements are not always uniform across all individual metrics. The evolved attacks exhibit hash-specific structural adaptations, and the results highlight that robust evaluation of perceptual hash systems requires auditing against algorithm-level attacks rather than fixed default configurations.

Perceptual hash algorithms vary in susceptibility to program-evolved attacks. Classical DCT-based hashes such as pHash and PDQ are easier targets when attacks operate in low-frequency search spaces, while PhotoDNA and NeuralHash require larger perturbations and achieve benchmark collisions with high perceptual distortion; NeuralHash remains the hardest target with limited attack success and a much higher query cost. pHash and PDQ are easier to attack once the search operates in a suitable low-frequency space. For PhotoDNA and NeuralHash, LPIPS remains high, so the achieved collisions are benchmark results rather than visually imperceptible examples. Evolved programs reduce the final score for every hash, showing that algorithmic search-space changes improve over mere parameter tuning. PhotoDNA requires larger perturbations, but evolved candidates still lower the score relative to optimized seeds. NeuralHash is the hardest target: its best evolved attack success rate is 0.83 and it demands substantially more queries than classical hashes. OpenEvolve gives the best score for pHash and PDQ, while the GigaEvo-based pipeline gives the best score for PhotoDNA and NeuralHash, indicating complementary strengths of evolution backends.

The study compares four perceptual hashing algorithms under targeted second-image attacks, revealing that hash geometry strongly influences attack difficulty. Classical hashes using global DCT features (pHash, PDQ) are easier targets, while PhotoDNA and NeuralHash require larger distortions and yield lower success rates, with NeuralHash being the most robust. The fixed matching thresholds per hash define the collision criteria, and attacks on the neural-based hashes often produce benchmark collisions rather than visually imperceptible results. pHash and PDQ rely on global DCT structure and are easier to attack, whereas PhotoDNA uses mid-scale gradient features and NeuralHash uses a neural embedding, making them more resistant to collision attacks. NeuralHash is the hardest target: the best evolved attack achieves only 0.83 success rate and demands substantially more queries than the classical hashes. The fixed matching thresholds (12 for pHash, 92 for PDQ, 3855 for PhotoDNA, 17 for NeuralHash) are not optimized by the attack and set the distance boundary for a successful collision. Attacks on PhotoDNA and NeuralHash tend to produce high perceptual distortion, so they are interpreted as benchmark collisions rather than visually imperceptible adversarial examples.

The external evaluator tracked attack success using a composite score that combines detection rate, distortion, and query cost. Evolved programs improved this composite score for every hash, but gains did not always translate to simultaneous improvements in perceptual similarity, pixel distance, or query count. NeuralHash was the hardest target, requiring more queries and yielding only benchmark collisions with high LPIPS, not visually imperceptible examples. Evolved programs reduced the composite score for all hashes, demonstrating that algorithmic changes offer benefits beyond parameter tuning alone. The composite score S is a ranking criterion; individual metrics like L2, LPIPS, and query count must still be reported because improvements are not monotonic across them. For NeuralHash, LPIPS remains high, so attacks should be interpreted as benchmark collisions rather than visually imperceptible examples. pHash and PDQ are easier targets once attacks exploit a low-frequency search space, while PhotoDNA and NeuralHash require larger perturbations.

Program evolution reduces the attack score for all perceptual hashes beyond hyperparameter tuning alone, with OpenEvolve yielding the best scores for pHash and PDQ and the GigaEvolve pipeline leading for PhotoDNA and NeuralHash. For PhotoDNA and NeuralHash, LPIPS remains high, so these are not visually imperceptible attacks but benchmark collisions. The evolved programs make algorithmic changes rather than just parameter variations, and they often trade off distortion, query count, and success rate, making all metrics necessary to report. The two evolution backends are complementary: OpenEvolve performs best on pHash and PDQ, while the GigaEvolve-based pipeline excels on PhotoDNA and NeuralHash. Evolved programs are not mere parameter variants; they incorporate structural changes and a refinement phase that reduces distortion after reaching the collision threshold. For PhotoDNA and NeuralHash, LPIPS remains high, so these attacks should be viewed as benchmark collisions rather than visually imperceptible examples. Evolved candidates do not improve all metrics simultaneously; they may spend more queries to get a cleaner image or preserve attack success rate while lowering distortion. The benchmark serves as an audit protocol, showing that systems that appear robust under default attacks may need evaluation against strengthened, program-evolved attacks.

The protocol progresses from hand-written seed evaluation through per-hash configuration optimization to program evolution, showing that evolving attack programs reduces the composite score for every hash beyond what hyperparameter tuning alone achieves. The improvements are not uniform across metrics, and the two evolution backends are complementary: OpenEvolve performs best on pHash and PDQ, while GigaEvolve gives the best scores for PhotoDNA and NeuralHash. PhotoDNA and NeuralHash attacks remain benchmark collisions with high LPIPS, and NeuralHash is the hardest target, with the best evolved attack reaching only 0.83 attack success rate. Configuration optimization substantially improves pHash, PDQ, and NeuralHash, but evolved programs further reduce the final score for every hash, indicating that algorithmic changes matter beyond scalar hyperparameters. Evolved candidates do not improve all metrics simultaneously; a candidate may use more queries to achieve lower distortion, so reporting all individual metrics is necessary. PhotoDNA and NeuralHash attacks retain high LPIPS and should be interpreted as benchmark collisions rather than visually imperceptible examples. The two evolution backends are complementary: OpenEvolve yields the best score for pHash and PDQ, while the GigaEvo-based pipeline gives the best score for PhotoDNA and NeuralHash. pHash and PDQ become easier targets when attacks operate in a low-frequency search space, while NeuralHash requires substantially more queries and its best evolved ASR is only 0.83. The protocol acts as an audit: a hash system that appears robust under default attack settings may be vulnerable after per-hash hyperparameter tuning and program evolution.

The study evaluates four perceptual hashing algorithms using a multi-stage attack protocol that evolves search programs beyond hyperparameter tuning, comparing hand-written seeds, configuration optimization, and program evolution. Classical DCT-based hashes (pHash, PDQ) are easier targets when attacks exploit low-frequency search spaces, while PhotoDNA and NeuralHash require larger perturbations and yield only benchmark collisions with high perceptual distortion. Program evolution reduces the composite score for every hash, with complementary backends (OpenEvolve for pHash and PDQ, GigaEvolve for PhotoDNA and NeuralHash) and NeuralHash proving the most robust target, demanding more queries and achieving only 0.83 attack success. The protocol acts as an audit, showing that systems that appear robust under default attacks can become vulnerable after evolved program changes.