One-sentence Summary

Carnegie Mellon University and University of Southern California propose DPD-BiReconstructor, a semisupervised neural architecture for proto-language reconstruction that incorporates the comparative-method insight that reconstructed forms must be deterministically transformable back into their daughter words, leveraging small labeled cognate sets and large unlabeled data to outperform strong semisupervised baselines.

Key Contributions

• The paper introduces semisupervised historical reconstruction as a novel task for protolanguage reconstruction, training a model on a small set of labeled cognate sets with protoforms and a larger set of unlabeled cognate sets without protoforms.

• DPD-BiReconstructor is a neural architecture for comparative reconstruction that combines daughter-to-proto reconstruction with reflex prediction. This design operationalizes the insight that reconstructed protoforms should be deterministically transformable back into daughter-language reflexes.

• Experiments show that DPD-BiReconstructor leverages unlabeled cognate sets to outperform strong semisupervised baselines, especially when protoform labels are scarce. Combining DPD with existing semisupervised strategies also yields significantly better performance than baseline strategies in almost all situations, and analysis suggests reflex prediction supervision may aid supervised reconstruction.

Introduction

Historical linguists use the comparative method to reconstruct unattested ancestor languages, or protolanguages, by comparing cognate sets and applying regular sound-change patterns. This matters because it reveals human migrations, cultural contacts, and language evolution. The method is complex because it requires balancing phonetic similarity with deterministic sound-change rules across many languages. Prior computational systems mostly rely on fully supervised sequence-to-sequence models, which need gold protoforms for every cognate set and are therefore limited to later-stage reconstruction. The authors address this by introducing semisupervised protoform reconstruction, where only a small number of labeled cognate sets are needed alongside many unlabeled sets. The authors leverage DPD-BiReconstructor, a multi-task model that reconstructs protoforms and then predicts reflexes from intermediate representations. Reflex prediction losses are propagated back into the reconstruction network, enabling the model to learn from unlabeled cognate sets and outperform baseline strategies when protoform labels are scarce.

Dataset

• Sources and composition: The authors use two fully labeled cognate datasets for historical language reconstruction. WikiHan is used for Middle Chinese reconstruction, and Romphon, a phonetic version of Meloni et al. (2021)’s Romance dataset, is used for Latin reconstruction.

• Subset scale and label control: For each dataset, the authors construct semisupervised training subsets by hiding a random subset of protoform labels from the fully labeled train set. The percentage of retained labels defines the labeling setting. At 10% labels, the subsets contain approximately 516 labeled cognate sets for WikiHan and 870 labeled cognate sets for Romphon. Across 5%, 10%, 20%, and 30% labeling settings, labeled cognate sets range from 181 to 1,084 for WikiHan and from 304 to 1,821 for Romphon.

• Processing and filtering rules: To reduce label-selection noise, the authors use a monotonic nested subset selection rule. For increasing retained-label percentages, the smaller labeled set must be a subset of the larger labeled set. They also generate four random semisupervised training sets per dataset for repeated evaluation, with each strategy tested multiple times within each generated set.

• Use in the model: These subsets are used to evaluate reconstruction strategies under fixed and varying amounts of labeled target data. A fixed 10% labeled semisupervised training set is also used for hyperparameter tuning. The provided text does not describe cropping, additional metadata construction, or source filtering beyond label hiding and nested subset selection.

Method

The proposed method frames protoform reconstruction as a bidirectional, self-supervised reconstruction problem. At its core, the model contains a daughter-to-protoform sub-network, denoted as D2P, which reconstructs a hypothesized protoform from the observed daughter-language reflexes, and a protoform-to-daughter sub-network, denoted as P2D, which predicts the reflexes that should be derivable from a protoform. Both sub-networks share phoneme embeddings, allowing reconstruction and reflex prediction to operate in a common phonetic representation space. The key training idea is that, when labeled protoforms are unavailable, P2D can provide weak supervision to D2P by evaluating whether the reconstructed protoform can still generate the observed reflexes through the predicted sound-change mapping. This design directly reflects the methodological constraint that a valid reconstruction should be compatible with the reflexes derived from it.

Refer to the framework diagram below:

In the overall pipeline, D2P receives an embedded sequence of concatenated daughter forms and outputs a protoform prediction. For labeled examples, D2P is trained with a standard cross-entropy loss, written as $L_{\mathrm{D2P}}$LD2P​, against the gold protoform. Independently, P2D can also be trained on labeled data by taking the gold protoform as input and predicting the target reflexes, using a supervised loss denoted as $L_{\mathrm{P2D}}^{\mathrm{gold}}$LP2Dgold​. This supervised pathway teaches the model the directional sound-change relationship from protoforms to reflexes.

A central architectural requirement is that D2P must be able to learn from the feedback provided by P2D even when the protoform is not observed. Directly passing discrete token predictions from D2P into P2D is not differentiable, so the method instead treats the final-layer decoder state of D2P as a continuous latent representation of the reconstructed protoform. A trainable dense component, called the bridge network, maps this latent state into the encoder input of P2D. This allows gradients from the reflex-prediction branch to flow back into the reconstruction branch.

To keep the bridge-network representation aligned with true phoneme-level protoform embeddings, the method adds a cosine-similarity loss, written as $L_{\mathrm{BRIDGE}}$LBRIDGE​, between the bridge-network output and the actual phoneme embeddings of D2P’s predicted protoforms. In effect, the bridge network is trained to behave as an alternative embedding layer for reconstructed protoforms. This gives P2D a stable and interpretable representation of the reconstructed input.

For both labeled and unlabeled data, P2D also evaluates the reflexes inferred from the predicted protoform. It applies a cross-entropy loss on predicted-protoform reflex outputs, denoted as $L_{\mathrm{P2D}}^{\mathrm{CE,pred}}$LP2DCE,pred​. In addition, a contrastive penalizing loss, called the CRINGE loss and written as $L_{\mathrm{P2D}}^{\mathrm{CR,pred}}$LP2DCR,pred​, discourages P2D from producing plausible reflexes when the input protoform is incorrect. This loss is especially important for negative supervision: if a protoform is wrong, but P2D still predicts a correct-looking reflex sequence, the loss penalizes that behavior. The result is that D2P receives more informative gradients when its proposed reconstructions are misleading.

The final training objective is a weighted combination of the reconstruction loss, the supervised reflex-prediction loss, the predicted-protoform reflex loss, the CRINGE loss, and the bridge-network cosine loss. Conceptually, the objective has the form:

where the weighting coefficients are tunable hyperparameters.

The DPD architecture itself is modular and can be instantiated with encoder-decoder variants. In this implementation, GRU-based and Transformer-based backbone models are used, but the bidirectional reconstruction strategy is defined at the training and interface level rather than being tied to one specific network type.

To exploit unlabeled cognate sets, the method also incorporates additional semi-supervised strategies. One baseline keeps only labeled data and performs standard supervised training. A bootstrapping procedure treats the model’s most confident protoform predictions on unlabeled data as pseudo-labels, filtering them by confidence and limiting their number per epoch. A $\Pi$Π-model regularization path creates two stochastic augmentations of the same cognate set and minimizes the mean-squared difference between the two model outputs. Because phoneme-level noise would damage the regularity assumption underlying sound change, augmentation is instead implemented at the cognate-set level by permuting daughter-language order and randomly dropping daughter languages. These components can be combined, so bootstrapping can be layered on top of other strategies, and the DPD bidirectional model can also be merged with consistency regularization for a more comprehensive semi-supervised training pipeline.

Experiment

The experiments evaluate semi-supervised protoform reconstruction on phonetic Romance and Sinitic datasets by hiding random subsets of protoform labels and comparing reconstruction strategies across architectures, label proportions, and statistical significance tests. Across most settings, DPD-based strategies, especially combined with Pi-model and bootstrapping, outperform supervised and weak baselines, show high sensitivity to label subset and architecture, and improve nonlinearly with label size, approaching supervised-level performance at higher label proportions. Additional analyses show that these gains depend on effective use of unlabeled data, align with linguistically meaningful phoneme representations, and remain useful even in supervised training.

The illustration uses Tangkhulic forms to show that daughter-to-protolanguage models can overregularize unlabeled reconstructions by choosing a uniform form from daughter words. In the examples with labeled references, both D2P and DPD match the protoform, but in the unlabeled laugh case, D2P predicts the same form used elsewhere while DPD predicts a different vowel. This supports the broader motivation for daughter-to-proto-to-daughter training as a way to avoid mechanically applying a single reconstruction pattern. When the reference protoform is labeled, the table shows D2P and DPD agreeing with the reconstructed form. In an unlabeled case, D2P produces a uniform reconstruction, while DPD predicts a different vowel, exposing a limitation of daughter-to-protolanguage modeling.

On the 10 percent labeled WikiHan and Rom-phon semisupervised reconstruction tasks, DPD-based methods generally outperform standard supervised and weak semisupervised baselines, with the best method depending on dataset and architecture. On WikiHan, Transformer configurations using DPD-based methods give the strongest results, while on Rom-phon Transformer and GRU favor different DPD-based variants. Performance also varies noticeably across dataset groups, indicating that labeled subset choice and architecture interaction affect outcomes. DPD-based strategies usually improve over weak baselines. Transformer performs better with DPD-based methods on WikiHan. GRU favors DPD-BST on Rom-phon while Transformer favors DPD-PiM-BST on Rom-phon. Transformer outperforms GRU on WikiHan but not on Rom-phon under most strategies. Performance shows high variation across dataset groups.

Reflex prediction accuracy rises as more labels are used, yet low-label results are closer to fully supervised performance than reconstruction results in the reported settings. The best strategy and architecture vary by dataset, with DPD-based models appearing frequently among the top performers. The reported analysis also notes that strong P2D accuracy on gold protoforms is not required for the P2D sub-network to help D2P training. Semisupervised reflex prediction can remain close to supervised performance even with limited labels. DPD variants are repeatedly strong performers across the reported datasets and labeling settings. The best strategy and architecture depend on the dataset and amount of labeled data.

Experiments validate daughter-to-proto-to-daughter training through qualitative Tangkhulic examples, showing that daughter-to-protolanguage models can overregularize unlabeled reconstructions while DPD variants avoid this pitfall. On semi-supervised WikiHan and Rom-phon reconstruction, DPD-based methods generally outperform standard and weak baselines, though the best architecture and strategy vary by dataset, labeled subset, and architecture interaction. Reflex prediction accuracy also improves with more labels and remains relatively close to fully supervised performance at low label counts, with DPD variants frequently performing well and strong protoform prediction not necessarily required to aid daughter-to-protolanguage training.