Where a New Concept Must Enter:
Entry Point Gates Cross-Task Usability in Unified Multimodal Models

BAGEL-7B-MoT (Deng et al., 2025), a Mixture-of-Transformers (Liang et al., 2025) with 28 decoder layers and hidden size 3584. Each layer carries a shared text/understanding expert (self_attn.{q,k,v,o}_proj, mlp.*) and a private generation expert (*_proj_moe_gen, mlp_moe_gen.*). Understanding runs the shared expert only; generation runs both.

We create 56 pseudo-named entities, each one Objaverse (Deitke et al., 2023) asset rendered in Blender from 60 training and 20 held-out viewpoints over disjoint azimuth ranges, with randomized lighting and backgrounds, paired with a pronounceable pseudo-word screened so the frozen model produces no consistent visual or lexical behavior for it (Figure 1). Concepts form seven disjoint groups of eight. Every number is measured within a group against a 24-wide retrieval bank of its own eight plus the next two groups, cyclically, so all groups sit at identical task difficulty.

G-inject trains caption $\rightarrow$ image with the rectified flow objective (Liu et al., 2023); U-inject trains image $\rightarrow$ text with language-model cross-entropy. Adapters are LoRA (Hu et al., 2022), rank 32, with placement a controlled variable (§4.2). An adapter trained in one direction is evaluated with both branches active, so a failure to export is a failure of the model and not of the harness.

Across 36 distinct configurations spanning injection side, adapter placement, constant-budget capacity split, substrate and data size, SAR correlates with export at Spearman $\rho=+0.68$, family-clustered bootstrap $[+0.37,+0.88]$. The relationship holds inside all four configuration families and no single family carries it. We deliberately quote no $p$-value, as the points are not independent systems. Four are duplicate configurations; the rest come from four sweeps sharing concepts, model, data, and code path, and every point is one model on one concept group, so a $p$-value would not answer anything about unified models. What the clustering does cost is precision: a fit that has never seen a family predicts its members’ export to within $0.12$–$0.20$ in accuracy units, which supports ranking configurations and not point prediction.

The direct test of the predictor is to optimize it. Semantic Anchoring turns the retrieval criterion into a training objective: with the addresses $a_{k}$ frozen, the name states $h_{k}$ are trained under the $K$-way InfoNCE loss

which pulls each name’s state onto its own concept’s address and pushes it from the other $K-1$. The layer at which $h_{k}$ is read is the objective’s one free choice, and it is the variable under test. Applied at the readout, after the final norm, of an understanding-side injection on one group, the objective drives that run’s SAR from $0.583$ to a perfect $1.000$ and moves its export from $0.562$ to $0.492$, a change of $-0.07$: perfect alignment, zero transfer (Figure 3). Read the other way, the observational fit says $+0.10$ of SAR accompanies $+0.26$ of export, and the intervention buys $+0.42$ of SAR and none of it. SAR also reads $1.000$ on a model that learned nothing else. What the readout intervention lacks is not alignment but position.

If position is what the readout intervention lacked, moving the entry point should recover transfer with everything else held fixed. The rest of the section therefore sweeps the depth at which Eq. 2 is applied, in two media that differ in what they may change. In the weight medium, LoRA adapters are trained against the objective read at layer $\ell$, for 480 steps. In the activation medium no weight changes at all: the objective’s alignment term is maximized in closed form on the residual stream itself. Freeze every weight and add one $d$-vector per concept to the residual stream at a single site, at the token positions where that concept’s name appears; positions are recovered at layer 0 by matching the exact sub-word embedding sequence, so training and evaluation locate them identically. The alignment term of that objective admits a minimal analytic edit: keep the name state’s own mean and deviation norm, and rotate only the deviation direction onto the image address. The analytic edit exists because a weight-based comparison alone would confound the site with the optimization, a deeper site might simply be harder to train, and an edit with no gradient step removes that confound: it puts the alignment probe within $0.0123$ of ceiling on average at every depth by construction, and never further than $0.2905$ against a chance loss of $2.0794$, so the sweep measures capability rather than optimization, at a cost of $8\times 3584=28.7$K cached values per group and no gradient steps. The naive alternative, setting the name’s state to the image address, also imports the modality offset and the magnitude, and produces an edit large enough to destabilize the forward pass. Appendix C derives the solution and gives the two places where it is not exact.

Entering mid-stack beats entering at the readout by $+0.705$ [+0.61, +0.79] for weights and $+0.545$ [+0.40, +0.70] for activations, $7/7$ groups in both cases. The activation window rises to a peak of $0.584$ at layer 7 and from layer 14 onward is indistinguishable from the base model at the resolution of this experiment: every site falls within $\pm 0.008$ of the final-norm baseline, with at most $5/7$ groups agreeing in sign (Table 4). The weight window peaks later, at layers 10–14 with $0.800$ and $0.808$, and then declines steadily to $0.115$ at layer 27. So changing what a site computes roughly doubles the usable depth relative to adding a fixed offset to what it outputs. It does not remove the depth dependence, and the activation arm’s descent is steep rather than discontinuous: the transition falls between the probed layers 10 and 14, and the one group where layer 12 was also measured reads $0.164$ there, between $0.469$ at layer 10 and $0.016$ at layer 14.

At layer 0 the activation arm gives $0.323$ [0.22, 0.42] against $0.584$ at layer 7, and at the input embeddings it sits at the base model’s level ($0.023$, one group). An edit therefore needs enough computation before it as well as after: at those sites the state being edited is still essentially the token itself, and overwriting its discriminative direction with an image-derived one places it in a region of state space the downstream network does not read. Layer 0 is nonetheless well above the norm ($+0.283$, $7/7$), so the shallow end is a graded rise and not a second cliff. The weight arm is less affected, reaching $0.602$ already at layer 3, because it changes a function rather than overwriting a state.

Identity is a 24-way retrieval hit rate and so inherits DINOv2’s notion of sameness. The two identity columns of Table 4 differ in nothing but the encoder, and scoring the same generations under CLIP’s vision tower (Radford et al., 2021) gives the same curve: Pearson $r=0.997$ across the twelve sites, the same peak site, a mean absolute difference of $0.034$, and the same sign and $k/7$ verdict at every site. CLIP reads about $0.06$ lower inside the window and $0.015$ higher at the base level, which narrows the measured window slightly and moves nothing about where it is.

Across this sweep 4-way matching stays at $0.84$–$0.90$ from layer 0 to layer 10 while identity traverses its entire range beneath it, and Table 2 settles the interpretation: every activation patch is at chance on producing the name, so the patch installs drawability and not both modalities. Figure 4 shows a second hazard. Adding a flow-matching gradient flat-lines the curve at $0.95$–$0.99$ from layer 3 to layer 26, because a generative objective produces identity wherever the anchor sits. A task loss can therefore mask depth dependence entirely, a caution for any alignment-depth ablation that also trains the task.

Three controls rule out the obvious alternatives directly (Table 17, Appendix I). The derangement placebo gives each concept another concept’s address at identical magnitude, $250.7$ against $250.4$ at layer 3, and identical direction statistics, with only the pairing wrong; identity goes to $0.000$, below the base model’s $0.036$, because the edited names now retrieve the wrong concept’s images rather than no concept at all. The effect is not a magnitude artifact, since $\lVert\delta\rVert/\lVert\text{state}\rVert$ stays in $0.37$–$0.44$ from layer 3 to layer 27 while the residual norm grows sixteenfold. And a quarter-magnitude edit degrades gracefully at a working site and stays unresponsive at a site that shows no effect, so deep sites are not merely over-perturbing.

Depth therefore gates usability on this model, in both media and under both encoders. Whether it gates usability on any model is the next question, and the answer turns out to require a second condition (§6.2).

The window is not an artifact of one model. Janus-Pro shares none of BAGEL’s backbone, scale, encoder coupling or generation mechanism, and it reproduces the shape. The curve rises out of the embeddings and peaks at relative depth $0.12$ with $0.583$, against its own LoRA-trained ceiling of $0.625$. By half depth it is extinguished ($0.135$ at rel. $0.54$ against a base level of $0.052$; Table 14). An intervention with no gradient steps therefore recovers $93\%$ of what training recovers, against $80\%$ on BAGEL. What replicates is that a window exists, opens shallow and closes by half depth; where it peaks does not, since BAGEL is maximized at rel. $0.29$ and Janus at rel. $0.12$.

The two null sweeps peak at $0.167$ against a base level of $0.104$ and at $0.250$ against $0.250$. On its own neither is yet a finding about the model, because a patch that fires and does nothing is indistinguishable from a faulty port. Three checks separate the two readings. The patch is applied: every port counts its own applications, and both models re-run the whole stack at every denoising step, so the edit fires 32 times per image on Lumina-DiMOO and 260 times on Omni-Diffusion. It is not too weak to be detected: it already reaches $0.62\times$ and $0.51\times$ the state norm against $1.04\times$ on Janus, and raising it further only degrades the alignment it was meant to install, to centered cosine $0.772$ from $0.993$ on Lumina. The decisive check runs the same code path with the visual address replaced by a real word’s state at that layer, rotating brindolt onto “elephant”. It is calibrated on Janus, where the two targets are interchangeable ($0.625$ word against $0.583$ visual). On Lumina-DiMOO and Omni-Diffusion it moves the model to $6.3\times$ and $2.0\times$ its base level, while the visual target leaves both at or below base (Table 15). Both nulls are therefore about the target rather than the method: the editing machinery works on all four models, and what these two cannot deliver is a visual address.

The four models split two against two, and the split follows what the understanding pathway encodes rather than anything the word “unified” names. §6.2 tests that reading against the alternatives.

“Unified” is a qualitative label, so we made the axis quantitative: for one real training batch per direction we record the set of parameters each direction uses and report the parameter-count-weighted Jaccard overlap. The estimator has to intersect a forward-hook trace with a gradient trace, since either alone is badly wrong. A hook trace counts modules that are computed and discarded, and a gradient trace counts empty-slice experts, which scores BAGEL at $0.961$ when the true figure is $0.500$ (Appendix F). Measured this way, the model that shares the least transfers the most: BAGEL shares $0.500$ of its backbone and reaches TransferRate $0.539$ on $G\!\to\!U$ and $0.924$ on $U\!\to\!G$, while Janus-Pro shares $0.873$ and reaches $0.291$ and $0.473$. These four rates come from the cross-architecture protocol of 12 concepts against a 12-way bank, not from the 56-concept runs of Table 1, so they are comparable to each other and not to that table. More shared weight is plainly not sufficient for more transfer. We stop there rather than fitting a trend: two architectures that also differ in scale cannot separate sharing from anything else, and the split that does hold across all four models is the one below.

The grouping in Table 5 is not by backbone family, since BAGEL and Omni-Diffusion share a Qwen2.5-7B lineage and land on opposite sides, nor by scale, since Janus succeeds at $1.5$B while Lumina-DiMOO fails at $8$B. The two models where a visual address is usable read their understanding features from a semantic vision encoder; the two where it is not derive them from a reconstruction objective over a VQ codebook. Lumina-DiMOO is the most diagnostic case: images and text occupy one vocabulary and one embedding table, the edit demonstrably lands at centered cosine $0.993$, its derangement placebo sits at $0.094$, and the address is still unusable, because a VQ index encodes which codebook entry reproduces a patch and not which object is depicted. Entry point is therefore necessary and not sufficient. The two directions must also represent concepts in a common semantic format at that depth, which unified weights do not supply and a semantic understanding encoder does.