PRISM-Net: Patient-specific reference-guided inter-breast symmetry matching for three-class breast DCE-MRI classification

Existing deep learning research in breast MRI has predominantly focused on lesion-level or binary diagnostic tasks, commonly using localized lesion regions or unilateral breast inputs.

Deep learning in breast MRI has predominantly focused on lesion-level characterization. CNNs, recurrent networks, projection-based models, and multi-task architectures have been used to distinguish benign from malignant lesions from manually segmented, radiologist-selected, or automatically detected regions. Representative approaches include multiparametric CNN classification, four-dimensional CNN–LSTM aggregation, deep-feature maximum-intensity projections, and joint lesion segmentation and classification [38, 4, 18, 19, 37]. DAE-CNN and its PBPK-informed extension further incorporate contrast-agent disentanglement and physiologically constrained data augmentation to improve lesion representation under limited training data [11, 10]. Despite differences in architecture and input representation, these methods generally assume that a target lesion has already been identified.

Existing breast MRI diagnostic models also predominantly adopt binary task formulations. Lesion-level studies commonly distinguish benign from malignant findings, whereas breast-level or volume-level models usually predict malignant versus non-malignant outcomes [29, 17]. Such formulations either exclude breasts without detectable lesions or combine no-lesion and benign cases within a single negative category. The ODELIA dataset and benchmark provide a clinically relevant task setting by assigning no-lesion, benign, and malignant labels to each breast in multicentre DCE-MRI examinations [28, 30]. Its Medical Slice Transformer baseline aggregates slice-level features from pre-contrast T1-weighted, first post-contrast subtraction, and T2-weighted volumes, establishing breast-side three-class diagnosis as a direct precedent [28].Because the two breasts are treated as separate samples, the model does not explicitly learn cross-breast interactions.

Taken together, current breast MRI deep-learning methods remain largely shaped by lesion-level inputs and binary label spaces. Model development would benefit from prediction units and label definitions that better reflect clinical diagnostic tasks. Breast-side three-class classification addresses this requirement by jointly distinguishing no-lesion, benign, and malignant states without presuming that a target lesion has already been identified.

Bilateral comparison is a routine interpretive strategy in breast imaging, allowing local morphology and enhancement to be evaluated against a patient-specific internal reference. Because the contralateral breast may itself contain abnormalities and bilateral anatomy is affected by natural asymmetry, nonrigid deformation, positioning variation, and heterogeneous background enhancement, bilateral symmetry is an approximate prior and direct voxel-wise correspondence is unreliable [2, 16, 21, 24, 42]. Early breast MRI studies quantified bilateral symmetry using registration-free handcrafted three-dimensional descriptors or predefined whole-breast kinetic differences, but remained limited to global representations [2, 41].

In subsequent breast MRI deep learning studies, bilateral information has not been entirely absent, but it was often incorporated implicitly. Zhang et al. proposed a two-stage framework using Mask R-CNN for suspicious lesion detection followed by ResNet50 for malignancy classification in breast MRI [43]. In the detection stage, the model used pre-contrast images and subtraction images of the left and right breasts as inputs, allowing breast symmetry to be considered within the whole-breast context. However, the method did not explicitly establish correspondence between the two breasts, nor did it construct asymmetry features, a contralateral reference branch, bilateral attention, or a dedicated bilateral fusion module. The subsequent classification stage mainly focused on local candidate regions detected by Mask R-CNN and used DCE parametric maps for lesion characterization. Therefore, this type of work suggested that breast MRI deep learning had recognized the importance of bilateral information, but such information was mainly incorporated as global contextual input. Within BreastMRI-FCDD, which performed breast-level binary cancer detection on DCE-MRI subtraction maximum intensity projections, Oviedo et al. [31] further investigated FCDD-Symmetric, in which the contralateral breast representation defined the normal-class center for anomaly scoring. However, its projection-level, center-based formulation did not explicitly establish spatially adaptive correspondence between bilateral three-dimensional feature volumes or use spatially matched contralateral features for background signal baseline normalization.

In mammography, digital breast tomosynthesis, and other breast imaging modalities, bilateral information has been more explicitly embedded into deep learning models. Earlier studies introduced distortion-insensitive, logic-guided bilateral comparison and paired asymmetry modeling for cancer localization and classification, while BilAD subsequently extended bilateral asymmetry learning to DBT [34, 25, 12]. DisAsymNet attempted to disentangle asymmetric abnormalities from symmetric breast structures in bilateral mammograms, reconstructing abnormality-removed symmetric images to improve interpretability [40]. A recent bilateral information-guided Vision Transformer framework further proposed a registration-free strategy for mammography, directly concatenating bilateral images and introducing soft spatial prompts to model cross-breast structural differences without explicit image registration [42]. BiGAM-Net also used asymmetry-aware fusion and inspectable gating mechanisms to quantify model reliance on intrinsic lesion features and contralateral asymmetry evidence in paired breast imaging [14]. These studies collectively suggest that bilateral information can serve as a structural medical prior to improve diagnostic performance and interpretability.

These studies support the value of explicit bilateral reasoning, while their two-dimensional or quasi-two-dimensional formulations provide limited coverage of the deformable volumetric anatomy and heterogeneous enhancement of breast DCE-MRI. Registration-free, spatially adaptive cross-breast matching coupled with patient-specific background normalization therefore remains the central methodological gap.

PRISM-Net is motivated by the clinical use of bilateral comparison in breast DCE-MRI. Rather than interpreting a target-side finding solely from its absolute appearance, radiologists compare it with contralateral tissue to determine whether it represents a meaningful deviation from patient-specific anatomical and physiological background patterns. Because the contralateral breast is neither anatomically identical to the target breast nor guaranteed to be lesion-free, we treat it as an approximate patient-specific reference rather than a normal template.

Operationalizing this reference-based interpretation raises three modeling challenges. First, corresponding bilateral regions are not necessarily located at identical mirrored coordinates because of nonrigid breast deformation, patient positioning, and anatomical asymmetry, making direct left-right subtraction unreliable. Second, bilateral mismatch may arise from moderate/marked BPE, heterogeneous/dense FGT, or other natural asymmetry rather than local pathology. Third, bilateral asymmetry alone is insufficient for three-class diagnosis, because distinguishing no-lesion, benign, and malignant findings also requires intrinsic target-side appearance and cross-slice volumetric context.

Accordingly, PRISM-Net learns two complementary representations for each breast: a target-appearance representation that preserves intrinsic diagnostic features, and a reference-conditioned deviation representation that characterizes how local target features differ from adaptively matched contralateral context.

Formally, let $\mathbf{X}^{L}$ and $\mathbf{X}^{R}$ denote the paired left and right breast DCE-MRI volumes from one examination, with $\mathbf{X}^{L},\mathbf{X}^{R}\in\mathbb{R}^{D\times H\times W}$, where $D$ is the number of axial slices and $H$ and $W$ are the in-plane spatial dimensions. Let $\mathcal{S}=\{L,R\}$ denote the set of breast sides. For each side $a\in\mathcal{S}$, the diagnostic label is $y^{a}\in\{0,1,2\}$, corresponding to no-lesion, benign, and malignant findings, respectively. Given the paired bilateral volumes, PRISM-Net learns a function $f_{\theta}$ that jointly predicts the side-specific class probability distributions $(\bm{\pi}^{L},\bm{\pi}^{R})=f_{\theta}(\mathbf{X}^{L},\mathbf{X}^{R})$, where $\bm{\pi}^{a}\in[0,1]^{3}$ and $\sum_{c=0}^{2}\pi_{c}^{a}=1$ for each $a\in\mathcal{S}$.

To make the approximate contralateral reference computationally usable, the bilateral volumes must first be mapped into a comparable semantic space. PRISM-Net horizontally flips the right breast volume, denoted by $\bar{\mathbf{X}}^{R}=\operatorname{Flip}(\mathbf{X}^{R})$, to place both breasts in a common orientation. This operation provides only a coarse spatial prior and does not assume voxel-wise correspondence. The left volume $\mathbf{X}^{L}$ and the flipped right volume $\bar{\mathbf{X}}^{R}$ are then encoded slice-wise using a shared DINOv3 ViT-S/16 backbone [36]. Weight sharing ensures that bilateral features are represented in the same semantic space, thereby enabling subsequent feature-space matching.

For the $d$-th slice and breast side $a\in\mathcal{S}$, the shared encoder produces a global slice token and a matrix of local patch tokens:

where $\mathbf{s}_{d}^{a}$ preserves the global target-side appearance, $\mathbf{P}_{d}^{a}$ contains the corresponding local patch representations, $C$ is the feature dimension, and $N$ is the number of image patches. The patch-level representations are subsequently used to construct adaptive contralateral references, whereas the global slice tokens are retained for appearance-preserving classification.

With bilateral features represented in a common semantic space, the next step is to construct a patient-specific reference for each target patch. Because identical mirrored coordinates do not guarantee anatomical correspondence, direct patch-wise subtraction may confound pathological differences with variations in breast shape, positioning, and deformation. PRISM-Net therefore allows each target patch to retrieve an adaptive contralateral reference through soft feature-space matching within the paired axial slice, without requiring voxel-wise registration.

Before matching, unreliable regions are excluded using binary masks. Let $\mathbf{M}^{L},\mathbf{M}^{R}\in\{0,1\}^{D\times H\times W}$ denote the exclusion masks corresponding to the original bilateral volumes, where $1$ indicates a region excluded from matching. The right-side mask is flipped together with the right breast volume, $\bar{\mathbf{M}}^{R}=\operatorname{Flip}(\mathbf{M}^{R})$, and the bilateral exclusion mask in the common orientation is defined as $\mathbf{M}^{U}=\mathbf{M}^{L}\vee\bar{\mathbf{M}}^{R}$. For slice $d$, the valid patch set is

where $\mathcal{O}_{i}$ denotes the spatial support of patch $i$. This filtering ensures that bilateral matching is performed only between patches that are reliable on both sides.

For one matching direction, let $a\in\mathcal{S}$ denote the target side and let $b\in\mathcal{S}$, $b\neq a$, denote the contralateral reference side. Based on the valid patch-token matrices, the matching weights and adaptive contralateral references are computed as

where $d_{k}$ is the projected query–key dimension and $\mathbf{W}_{Q}$, $\mathbf{W}_{K}$, and $\mathbf{W}_{V}$ are learnable projection matrices. The softmax operation is applied over the valid contralateral patches, so each row of $\mathbf{A}_{d}^{a\leftarrow b}$ describes the correspondence distribution between one target patch and the candidate reference patches. Consequently, each row $\mathbf{r}_{d,i}^{a}$ of $\mathbf{R}_{d}^{a}$ represents a soft, content-adaptive contralateral reference for the corresponding target token $\mathbf{p}_{d,i}^{a}$, rather than the feature at a fixed mirrored location. The same matching operation is applied in both directions with shared parameters. These adaptively matched reference pairs provide the basis for bilateral relation modeling. However, their discrepancies are not uniformly diagnostic.

A large target-to-reference difference does not necessarily signify pathology, as it may arise from natural asymmetry in breast morphology, FGT distribution, asymmetric BPE, hormonal or lactational influences, or inflammatory and edematous changes. Therefore, after obtaining the adaptive contralateral reference, PRISM-Net preserves both the target and reference representations and explicitly encodes their bilateral relationship.

For each valid patch $i\in\mathcal{V}_{d}$, the bilateral relation representation is constructed as

where $\mathbf{p}_{d,i}^{a}$ preserves target-side appearance, $\mathbf{r}_{d,i}^{a}$ provides patient-specific contralateral context, their difference encodes the directional target-to-reference deviation, and their element-wise product captures feature-wise bilateral interaction. The projection layer $\phi(\cdot)$ maps the concatenated representation to a $C$-dimensional relation feature $\mathbf{g}_{d,i}^{a}$.

Although the relation representation captures bilateral differences, diffuse mismatch may reflect background variation rather than focal pathology. PRISM-Net therefore measures how strongly each patch-level discrepancy stands out from its local spatial neighborhood. Let $\mathcal{N}_{d}(i)=\mathcal{N}(i)\cap\mathcal{V}_{d}$ denote the valid patches within the $3\times 3$ neighborhood of patch $i$. The local-contrast asymmetry score and reweighted relation feature are computed as

This local-contrast formulation emphasizes discrepancies that are salient relative to their surroundings while attenuating spatially diffuse bilateral mismatch.

Because diagnostically relevant deviations may be sparse or distributed across multiple regions, direct global averaging could dilute their contribution. We therefore stack the reweighted relation features as $\widehat{\mathbf{G}}_{d}^{a}\in\mathbb{R}^{|\mathcal{V}_{d}|\times C}$ and summarize them using a shared set of learnable asymmetry queries $\mathbf{Q}_{A}\in\mathbb{R}^{K\times C}$:

The complete relation-modeling module is applied bidirectionally with shared parameters. The resulting tokens remain side-specific because the target and contralateral reference roles are reversed in the two matching directions. The resulting side-specific asymmetry tokens summarize focal reference-conditioned deviations, but they do not replace the intrinsic appearance of the target breast. We therefore integrate them with the global target-side slice representations for final volumetric classification.

The preceding module produces side-specific tokens that summarize focal bilateral deviations. For diagnosis, however, this information should complement rather than replace intrinsic target-side appearance, because symmetric findings are not necessarily normal and asymmetric findings are not necessarily pathological. Moreover, the contralateral breast may itself contain abnormalities. PRISM-Net therefore retains the target-side slice token as the primary appearance representation and introduces the bilateral-deviation tokens through a residual cross-attention pathway.

For side $a\in\mathcal{S}$ and slice $d$, the reference-conditioned slice representation is computed as

where $\lambda$ is a fixed residual scaling hyperparameter that controls the contribution of the bilateral-deviation information. The residual scaling factor $\lambda$ was fixed at $0.8$ throughout training and evaluation. This residual formulation preserves the intrinsic target-side appearance while allowing the slice representation to be selectively conditioned on patient-specific contralateral context.

Because diagnostically relevant evidence may extend across multiple slices, the reference-conditioned slice representations are subsequently aggregated to capture volumetric context. To preserve slice order, a positional embedding $\mathbf{e}_{d}\in\mathbb{R}^{C}$ is added to each reference-conditioned slice token. A learnable fusion token $\mathbf{u}\in\mathbb{R}^{C}$ is then appended to the resulting sequence, and a slice-level Transformer produces the side-specific volumetric representation:

where $(\cdot)_{\mathbf{u}}$ denotes the output representation associated with the fusion token. The slice positional embeddings enable the Transformer to distinguish slice locations and model ordered inter-slice context. The same slice-level Transformer is applied to both breast sides with shared parameters.

Finally, a shared MLP classifier $h(\cdot)$ maps the volumetric representation to side-specific logits $\bm{\ell}^{a}=h(\mathbf{z}^{a})\in\mathbb{R}^{3}$, and the corresponding class probability distribution is $\bm{\pi}^{a}=\operatorname{softmax}(\bm{\ell}^{a})$. The classifier parameters are shared between the left and right breasts. PRISM-Net is trained using the summed side-level cross-entropy loss:

Thus, the left and right breast labels provide separate side-level supervision within a single bilateral forward pass.

Model development used the public ODELIA breast DCE-MRI dataset. Evaluation was performed using held-out ODELIA data and an independent institutional dataset from the General Hospital of the People’s Liberation Army. The selection workflow for the private institutional set is shown in Fig. 3.

Public ODELIA dataset. The public ODELIA dataset comprised $741$ examinations from $741$ women acquired at six European medical centers between December $2006$ and May $2024$ [30]. Each breast was independently labeled as no-lesion, benign, or malignant. $291$ examinations exhibited no lesions, $146$ examinations were diagnosed with benign but no malignant lesions, and $304$ examinations had malignant lesions. In total, the dataset includes $978$ breasts without lesions, $195$ with benign lesions, and $309$ with a malignant lesion. The original training (n = $408$) and validation (n = $103$) partitions were retained, while the test partition was split into an internal set (n = $130$) and a center-held-out set (n = $100$).

Private institutional dataset. An independent institutional cohort was retrospectively assembled to evaluate generalization and performance under complex background conditions. The primary candidate cohort was drawn from the $2025$ institutional archive. As examinations with moderate or marked BPE were underrepresented in this cohort, additional candidates meeting this BPE criterion were retrieved from the $2024$ and $2026$ archives. Examinations were reviewed by two breast radiologists and one clinician using DCE-MRI findings, BI-RADS assessments, radiology reports, and pathological results. Examinations were excluded for substantial treatment-related or postoperative changes, breast implants, or missing key DCE-MRI phases. After patient-level deduplication, $171$ examinations remained ($135$ from $2025$ and $36$ BPE-enriched examinations from $2024$ and $2026$), yielding $342$ evaluable breast sides. Overlapping complexity subsets included $55$ examinations with moderate/marked BPE and $92$ with heterogeneous/dense FGT; $49$ met both criteria, as shown in Fig. 3. $79$ examinations exhibited no lesions, $32$ women were diagnosed with benign but no malignant lesions, and $60$ women had malignant lesions.

Breast-side reference standard The left and right breasts were treated as separate prediction targets and labeled as no-lesion, benign, or malignant. Labels were assigned using a pathology-prioritized hierarchy. Malignant or benign labels were assigned according to ipsilateral pathological findings. A no-lesion label was assigned when no ipsilateral enhancing lesion or pathological diagnosis was identified. For breasts with multiple findings, the most severe diagnosis determined the final label, with malignancy taking precedence over benignity and no-lesion. In total, the dataset includes $240$ breasts without lesions, $38$ with benign lesions, and $64$ with a malignant lesion. Cohort characteristics and acquisition protocols are summarized in Tables 1 and Supplementary Table S1.

PRISM-Net achieved the best performance on both the ODELIA in-distribution and RSH out-of-distribution sets (Table 2). Its Macro AUC, Micro AUC, and QWK were $84.11\pm 2.33$, $90.64\pm 1.61$, and $60.94\pm 5.64$ in-distribution, and $68.51\pm 4.54$, $80.74\pm 2.68$, and $43.45\pm 7.10$ out-of-distribution. Relative to the strongest baseline, ODELIA, the corresponding gains were $7.19$, $5.08$, and $14.19$ points in-distribution and $8.20$, $3.30$, and $13.99$ points out-of-distribution. ROC curves and confusion matrices are shown in Figs. 4 and 5.

We compared PRISM-Net with DisAsymNet and ODELIA, which were the two strongest baseline methods on the ODELIA public development dataset. The independent 2025 institutional cohort was not used for training, validation, model selection, or tuning. PRISM-Net achieved the highest Macro AUC, Micro AUC, and QWK: $66.85\pm 3.47$, $86.38\pm 1.84$, and $38.85\pm 6.43$, respectively (Table 3). Its QWK exceeded ODELIA and DisAsymNet by $40.86$ and $26.41$ points, respectively. The corresponding ROC curves and confusion matrices are shown in panels (a) and (d) of Supplementary Fig. S1.

The comparison was also restricted to DisAsymNet and ODELIA, the two best-performing baselines in the public-dataset evaluation. PRISM-Net also ranked first on both complexity cohorts (Table 3). On the BPE-complexity set, Macro AUC, Micro AUC, and QWK were $66.23\pm 4.90$, $82.66\pm 2.91$, and $31.48\pm 9.36$; on the FGT-complexity set, they were $67.71\pm 3.38$, $80.11\pm 2.37$, and $40.88\pm 6.83$. Corresponding ROC curves and confusion matrices are shown in panels (b,e) and (c,f) of Supplementary Fig. S1.

The full PRISM-Net was best in both settings (Table 4). Removing the bilateral asymmetry branch reduced QWK by $14.56$ points in-distribution and $13.40$ points out-of-distribution; removing focal score reweighting reduced QWK by $5.55$ and $14.38$ points, respectively. Macro and Micro AUC also decreased for both variants, supporting the contribution of each component.

Second, we compared different feature extraction backbones while keeping the remaining bilateral learning framework unchanged. With the bilateral framework and evaluation protocol fixed, DINOv3 outperformed ResNet-18, ResNet-50, ViT, and DenseNet-121 on all six metrics (Table 5). DenseNet-121 was the second-best backbone. Relative to DenseNet-121, DINOv3 improved Macro AUC, Micro AUC, and QWK by $2.91$, $1.48$, and $6.74$ points in-distribution and by $2.17$, $0.77$, and $3.29$ points out-of-distribution.

Representative maps (Fig. 6) show that focal reweighting suppresses diffuse or background-related bilateral differences and concentrates responses on localized asymmetry. The no-lesion case showed diffuse activation, the benign case a mild localized response, and the malignant case a compact high-response region corresponding to the enhancing lesion. These maps provide feature-level explanations rather than lesion segmentations.

We further analyzed representative failure cases to characterize the limitations and decision boundaries of PRISM-Net. As shown in Fig. 7, the errors mainly occurred in three challenging scenarios: benign lesions with malignant-like enhancement patterns, bilateral benign lesions that weakened the discriminative value of contralateral symmetry, and small low-suspicion lesions embedded within dense fibroglandular tissue. These cases suggest that the model may still be affected by overlapping enhancement appearances between benign and malignant lesions, confusing bilateral enhancement patterns, and limited lesion conspicuity in complex backgrounds.