One-sentence Summary

Researchers from the University of Texas at Arlington and Shanghai Jiao Tong University introduce an iterative minimal-pair-based algorithm that automatically discovers a canine phonetic alphabet, identifies distinct cognitive vocal units, and derives activity-correlated vocal patterns, thereby offering a novel framework for analyzing non-human vocalization systems with potential applications across other animal species and advancing the study of cross-species communication.

Key Contributions

• Introduces an iterative algorithm inspired by human phonetic discovery that uses minimal pairs to identify distinct canine cognitive vocal units and construct a phonetic alphabet from raw dog vocalizations.
• Derives canine vocal patterns exhibiting structured correlations with specific environments and activities, indicating potentially meaningful communicative patterns.
• Experimental results show 42% of vocal units verified as distinct cognitive units via human evaluation, 11% of patterns with high context-specific likelihood, and discovered patterns following Zipf's law, with the framework designed for extension to other animal species.

Introduction

Understanding animal vocal communication is a complex challenge, and a key part of that effort is identifying the basic phonetic units that might exist in animal calls. These units, while not forming a full language on their own, are the building blocks for exploring any grammatical or semantic structure hidden in vocalizations. Recent machine learning advances have opened the door to more granular, data-driven studies in this "animal language" research space, but prior attempts have hit notable roadblocks. Some approaches forced animal sounds into the human International Phonetic Alphabet, which ignores the fundamental acoustic differences between species. Others trained cross-species transcription systems on hundreds of different animals, producing a coarse-grained model that misses species-specific vocal traits. Meanwhile, self-supervised models like HuBERT applied to dog audio often mistook background noise for true vocalizations, and specialized methods for sperm whale clicks were too narrowly tailored to that species' unique sound production mechanism.

To address these gaps, the authors propose an iterative approach that simultaneously filters qualified canine cognitive vocal units (similar to human phonemes) and canine vocal patterns (similar to words) from noisy, inaccurate symbolic transcripts of a large dog vocalization corpus. Their method leverages the minimal pairs principle from human linguistics to filter out non-vocalization symbols and merge those that are acoustically and semantically similar. The authors' contributions include creating the largest dog vocalization dataset to date with 152 hours of pure barks from six breeds, and proposing an iterative discovery algorithm that produces 105 distinct canine cognitive vocal units and 69 vocal patterns from 6,285 hours of YouTube video. Their human-verified evaluations show that 42% of the discovered vocal units are distinct cognitive units, 11% of the vocal patterns are highly likely to appear in specific contexts, and the discovered pattern distribution closely follows Zipf's law.

Dataset

Dataset Composition and Sources

• The authors build their dataset exclusively from YouTube dog videos, totaling 6,235 hours of raw footage.
• Since these videos commonly contain background music, human speech, and other non-canine noises, the raw material is heavily contaminated and requires extensive cleaning before use.

Preprocessing and Filtering Pipeline

• Noise removal: The authors apply AudioSep to strip extraneous sounds from the video data.
• Dog vocalization detection: They use a fine-tuned DCASE2023 Challenge Task 4 baseline model for sound event detection (SED), replacing the default encoder with a pretrained BEATs encoder to improve detection performance.
• SED post-training: To boost accuracy, the authors manually label over 9,000 seconds of pure dog vocal data and use it to post-train the SED model, achieving an F1 score of 0.8556 on the test set.
• Extraction result: After this pipeline, the authors obtain 306,233 canine vocal segments, totaling approximately 152 hours of pure dog vocalizations from the original 6,235 hours.

Vocal Pattern Segmentation

• The authors use the Auditok library to segment continuous canine vocalizations by detecting energy-based silences, which identifies natural pause boundaries and yields candidate vocal patterns.
• Dynamic thresholding: For each utterance, the segmentation threshold is computed as the product of a tunable coefficient ρ and the root mean square (RMS) energy of the full audio segment. The authors empirically explore a range of ρ values and manually select the one giving the most consistent segmentation on a development set.
• These energy-based segments act as proxies for vocal patterns, since no predefined lexicon exists for animal vocalizations.

Transcription and Refinement Outcomes

• The transcription pipeline initially produces 250 candidate vocal units.
• A human inspection of 10 random samples per unit labels 52 of the 250 as noise and 198 as legitimate vocalizations.
• After iterative refinement, the inventory is reduced to 105 cognitive vocal units (only 22 still labeled as noise) and 69 cognitive vocal patterns.
• The refinement process eliminates 26 noisy units, and an additional 4 are removed during vocal pattern segmentation, demonstrating the effectiveness of the iterative denoising.

How the Data Is Used

• The full dataset is fed through the preprocessing, segmentation, and refinement stages to construct the final canine phonetic alphabet.
• The 105 cognitive vocal units and 69 cognitive vocal patterns are the core outputs used for acoustic and semantic evaluation.
• The vocal patterns are examined for contextual significance to support minimal pair analysis, though they are not the primary output of the pipeline.

Method

The authors employ a data-driven approach to uncover the canine phonetic alphabet through a multi-stage pipeline. The overall framework consists of three primary steps: preprocessing, transcription, and vocal unit refinement.

Refer to the framework diagram

In the preprocessing stage, the system extracts clean canine vocal segments from raw audio tracks, such as those from online videos, which often contain background noise. The authors utilize AudioSep to remove extraneous noise and a fine-tuned sound event detection model based on the DCASE2023 Challenge Task 4 baseline. To enhance detection performance, the default encoder is replaced with a pretrained BEATs model, and the system undergoes post-training on manually labeled canine vocal data. A canine vocal segment is defined as a continuous vocalization bounded by pauses exceeding 0.5 seconds.

The transcription stage converts these segments into a sequence of symbols representing canine vocal units. The authors train a self-supervised audio representation model, HuBERT, on the canine segments to obtain frame embeddings for each 20ms audio frame. These frames are clustered into K clusters, with K determined by the elbow method. The cluster center serves as the feature embedding for a unique phone, which is assigned a unique symbol to form an initial cognitive vocal unit.

The final stage refines these units into a final set of cognitive vocal units and vocal patterns using a minimal pairs methodology.

Refer to the refinement algorithm

The core of this algorithm involves identifying minimal pairs among candidate vocal patterns. A minimal pair is defined as two vocal patterns that differ by exactly one cognitive vocal unit and are empirically distinguishable. The process generates initial candidates for vocal units and patterns, then identifies valid minimal pairs. Since semantic differences cannot be definitively determined, the authors incorporate a vocal unit merging step. If a pair of cognitive vocal units consistently appears as the contrasting unit in minimal pairs within similar vocal environments, they are considered acoustically and functionally equivalent and are merged. The algorithm retains only vocal units that serve as contrasting units in at least one valid minimal pair and filters patterns to include only those composed of valid units. This iterative process continues until convergence.

To generate the initial vocal pattern candidates, the authors segment continuous canine vocalizations using the Auditok library, which identifies energy-based silences.

As shown in the figure below:

This yields candidate patterns corresponding to regions of elevated acoustic energy. To handle intensity variations across clips, a dynamic thresholding strategy is employed. The segmentation threshold is calculated as the product of a tunable coefficient $\rho$ρ and the root mean square (RMS) energy of the full audio segment. The resulting candidate vocal patterns are separated by low-energy intervals determined by this adaptive threshold.

During the refinement of minimal pairs, the authors evaluate contextual similarity to address potential oversegmentation or clustering artifacts. They extract context distributions based on preceding and following vocal units, co-occurring patterns, and local N-gram structures. The Jensen–Shannon (JS) divergence is computed between context distributions for each pair. Pairs with low JS divergence, suggesting highly similar surrounding contexts, are assumed to be over-clustered variants of the same unit and are merged. The system also discards rare vocal patterns occurring fewer than a minimum threshold $K$K times in the corpus. This iterative refinement yields a stable set of cognitive vocal units and vocal patterns, constituting the canine phonetic alphabet.

Experiment

The evaluation used 6,235 hours of dog videos, yielding over 300,000 vocal segments refined into 105 cognitive vocal units and 69 vocal patterns. Acoustic tests, including noise removal checks and ABX listening tests, confirmed that iterative refinement eliminated most noise and produced acoustically distinct units with about 80% inter-annotator agreement. Downstream classification tasks showed that cognitive vocal unit embeddings outperformed raw acoustic features for bark type and context prediction, indicating stronger semantic grounding. Contextual analysis revealed that most high-frequency vocal patterns correlated strongly with specific environments or activities, supporting their potential referential roles.

The table defines key terms in a canine vocalization analysis framework, comparing them to linguistics concepts: vocal segments are like sentences, vocal units like phones, cognitive vocal units like phonemes, and vocal patterns like words. The approach uses data-driven preprocessing, transcription, and refinement to identify these units, with evidence that more abstract features improve classification performance and that certain vocal patterns show contextual associations. Vocal segments are continuous dog vocalizations bounded by pauses over 0.5 seconds, analogous to human sentences. Cognitive vocal units represent perceptually and contextually similar vocal units, analogous to phonemes. Classification performance improves as features become more abstract, from MFCCs to cognitive vocal units. Most of the 69 high-frequency vocal patterns show strong positive or negative correlation with one to two semantic categories.

The dataset comprises over 6,000 hours of dog videos, from which roughly 152 hours of pure vocalization segments were extracted. Video and vocalization hours vary notably across breeds, with Shiba Inu contributing the most video but not the highest vocalization ratio, while Labrador and Husky show relatively large vocalization shares. Shiba Inu provides the largest video volume, yet its vocalization hours are modest relative to video length, indicating lower vocal density. Labrador and Husky both yield substantial vocalization hours relative to their video durations, suggesting higher vocal activity. Overall, the preprocessing pipeline condenses the source video into a small fraction of pure vocalization time, supporting downstream analysis.

The table lists selected cognitive vocal units and vocal patterns derived from a large dataset of dog vocalizations. It supports the finding that discrete symbolic vocal representations improve semantic classification performance, and that certain vocal patterns show consistent correlations with specific environmental or behavioral contexts. The pipeline produced 105 cognitive vocal units and 69 cognitive vocal patterns from over 300,000 canine vocal segments. Classification performance improves as features become more abstract and symbolic, moving from MFCCs to cognitive vocal units. Most of the 69 high-frequency vocal patterns show strong positive or negative correlation with one to two semantic categories, suggesting potential referential consistency. One example vocal pattern is negatively associated with indoor and passive behaviors while positively correlated with active, outdoor activities.

Two independent rater groups achieved higher ABX accuracy on the final cognitive vocal units than on the initial HuBERT-derived units, while maintaining roughly 80% inter-group agreement. This indicates that the refined units are more acoustically distinct and consistently perceived by human listeners. Both rater groups improved accuracy when evaluating the final cognitive vocal units compared to the initial set. Inter-group agreement stayed near 80% across both sample sets, confirming test reliability. The final cognitive units were perceived more consistently, supporting the refinement algorithm's effectiveness.

The table compares Macro F1 scores for dog bark type classification using different feature types and classifiers. Performance generally improves as features become more abstract and symbolic, with cognitive vocal unit embeddings achieving the highest scores across most classifiers. This supports the idea that discrete vocal representations provide stronger semantic grounding for downstream tasks. Cognitive vocal unit embeddings consistently outperform MFCCs across all classifiers, with the largest gains in random forest and XGBoost. Frame embeddings and vocal unit embeddings show mixed results, but cognitive vocal units yield the best overall performance. The trend indicates that more abstract and symbolic features lead to better classification, reinforcing the semantic value of cognitive vocal units.

The experiments validate a hierarchical framework for analyzing canine vocalizations by showing that abstract and symbolic features, such as cognitive vocal units, consistently improve semantic classification over raw acoustic features. Human perception tests confirm that the refined units are more acoustically distinct and reliably recognized, while pattern analysis reveals that most high-frequency vocal patterns correlate strongly with specific behavioral contexts. Overall, the results support the use of discrete, data-driven vocal representations to enable meaningful downstream interpretation of dog vocalizations.