GraphRAG-Router: Learning Cost-Efficient Routing over GraphRAGs and LLMs with Reinforcement Learning

Dongzhe Fan Affiliation: Department of Computer Science, New York University (Shanghai), Chuanhao Ji Affiliation: Department of Computer Science, New York University (Shanghai), Zimu Wang Affiliation: Department of Computer Science, University of Liverpool {df2362, cj2851, qiaoyu.tan}@nyu.edu, {zimu.wang, tong.chen}@liverpool.ac.uk Tong Chen Affiliation: Department of Computer Science, University of Liverpool {df2362, cj2851, qiaoyu.tan}@nyu.edu, {zimu.wang, tong.chen}@liverpool.ac.uk Qiaoyu Tan Affiliation: Department of Computer Science, New York University (Shanghai),

Abstract

Graph-based retrieval-augmented generation (GraphRAG) has recently emerged as a powerful paradigm for knowledge-intensive question answering, especially for tasks that require structured evidence organization and multi-hop reasoning. However, existing GraphRAG systems are typically built in a one-size-fits-all manner, relying on a fixed retrieval framework and a single, often large and costly, generator LLM for all queries. This static design limits their ability to adapt to the complexity of varying questions and often incurs unnecessary computational cost. To fill in the gap, we propose GraphRAG-Router, a cost-efficient framework that adopts a hierarchical routing strategy to coordinate heterogeneous GraphRAGs and generator LLMs. Specifically, GraphRAG-Router is first warmed up through supervised fine-tuning and then optimized with a two-stage reinforcement learning procedure, whose second stage introduces a curriculum cost-aware reward to encourage difficulty-aware and economical generator allocation. Extensive experiments on six general-domain and multi-hop QA benchmarks show that GraphRAG-Router consistently outperforms state-of-the-art baselines, reducing the overuse of large LLMs by nearly 30% while maintaining strong generalization capability.

1 Introduction

Graph-based retrieval-augmented generation (GraphRAG) has emerged as a powerful paradigm for knowledge-intensive question answering (QA) 8; 27; 1; 5, especially for multi-hop questions that require structured evidence organization and compositional reasoning. By organizing knowledge into entities, relations, and higher-level graph structure, GraphRAG enables more explicit evidence aggregation than standard text-based retrieval-augmented generation, making it particularly well suited for complex reasoning over dispersed evidence 7; 25.

Refer to caption — (a) Dataset preference of GraphRAGs.

Despite these advances, current GraphRAG systems are still largely built in a one-size-fits-all manner: they typically rely on a single retrieval framework and a single generator LLM for all queries. Such a static design is inherently suboptimal, because questions vary substantially in both the form of evidence they require and the level of reasoning capacity needed to answer them. On the retrieval side, different QA benchmarks exhibit markedly different preferences over GraphRAG frameworks (Figure 1(a)), suggesting that different questions demand evidence at different levels of granularity. For instance, entity-centric lookup questions are better served by local subgraph retrieval, whereas questions requiring thematic synthesis benefit more from community-level summarization. On the generation side, existing GraphRAG systems often rely on a large and costly LLM to ensure strong overall performance 1; 6 (e.g., GPT-4o 23), yet Figure 1(b) shows that many questions can already be solved correctly by much smaller models. Uniformly invoking a large model therefore wastes computation, while relying only on a smaller model may fail on more challenging queries. These observations indicate that both retrieval infrastructure and generator allocation should be adaptive, so that each query can be matched with the most appropriate evidence source and model capacity. This raises a central question: How can we adaptively coordinate multiple GraphRAGs and generator LLMs for each query, while maintaining strong QA performance without incurring unnecessary generator cost?

Solving this problem is non-trivial due to two major challenges. (1) Combinatorial action space in joint GraphRAG–LLM selection. While prior studies have explored model-wise routing among candidate LLMs 36; 17, our setting requires the routing agent to jointly select both the GraphRAG and the generator LLM at each reasoning step. This joint decision substantially expands the decision space compared to model-only routing, and the exponential growth of the action space makes effective optimization particularly challenging. (2) Cost-efficient routing. An effective routing strategy must balance answer quality against the computational overhead of generator LLM invocation. Since queries differ widely in their reasoning complexity, the optimal model scale, defined as the minimum capacity required to produce a correct response, is inherently context-dependent. Therefore, the routing policy must calibrate its allocations precisely, as over-provisioning leads to unnecessary computational cost, while under-provisioning risks degrading answer accuracy.

To this end, we introduce GraphRAG-Router, a cost-efficient RL framework that enables multi-round joint routing and aggregation across heterogeneous GraphRAG frameworks and generator LLMs. Instead of making a single monolithic dispatch decision over an LLM–GraphRAG pair, we formulate the coordination process as a hierarchical routing path that allows the model to select the GraphRAG framework and generator LLM step by step. Specifically, we first warm up the routing policy via supervised fine-tuning (SFT) using curated routing trajectories. We then optimize the policy with a two-stage RL procedure that incorporates three complementary reward signals: a format reward encouraging well-structured outputs, an outcome reward based on final task correctness, and a curriculum cost-aware reward that promotes difficulty-aware model utilization by penalizing unnecessarily expensive routing decisions. Together, these components make GraphRAG-Router a cost-efficient and generalizable solution for multi-round coordination across heterogeneous GraphRAG frameworks and LLMs, achieving state-of-the-art performance on diverse knowledge-intensive QA benchmarks.

The key contributions of this work can be summarized as follows:

$\star$

We introduce GraphRAG-Router, an RL–based framework for multi-round routing and aggregation across heterogeneous GraphRAGs and generator LLMs, enabling question-adaptive routing across retrieval and generation modules.
$\star$

We propose a hierarchical routing strategy and optimize GraphRAG-Router with a two-stage RL procedure with a curriculum cost-aware reward, allowing the model to route queries to appropriate GraphRAG–LLM pairs while better balancing performance and LLM cost.
$\star$

Experimental results on six QA benchmarks demonstrate that GraphRAG-Router consistently outperforms strong baselines, achieving superior cost-efficiency, robust generalization, and state-of-the-art overall performance.

2 Related Work

GraphRAG.

GraphRAG extends the conventional RAG paradigm by structuring retrieved knowledge as entities and relations, thereby providing stronger support for multi-hop reasoning and knowledge-intensive QA. Early GraphRAG methods 1; 27 typically construct knowledge graphs with entities as nodes and relations as edges. Recent works 4; 37; 5; 6 further enhance the representation power of GraphRAG by incorporating richer graph structures (e.g., HyperGraph), thereby improving evidence organization and multi-hop reasoning over complex knowledge, typically with the support of large-scale LLMs. Meanwhile, another line of work explores GraphRAG systems 19; 33; 35 that rely on smaller models, whose reasoning capabilities are enhanced through RL. In this study, we focus on the setting where GraphRAG frameworks are paired with large and costly LLMs, which often incurs substantial computational cost and still relies on a fixed retrieval infrastructure.

LLM Routing.

Current LLM routing frameworks aim to select appropriate generator models from a candidate pool according to query complexity. Existing studies have explored both predictive model routing 17; 2 and multi-round routing strategies over candidate LLMs, including RL-based approaches such as Router-R1 36 that interleave internal reasoning with dynamic model selection and response aggregation. However, these methods focus solely on routing among generator LLMs. In contrast, our setting requires joint routing over both GraphRAG frameworks and generator LLMs, which introduces a substantially larger and more complex action space.

3 Problem Definition

We formulate our task as follows: Given a query $q$ , a candidate pool of GraphRAGs $\mathcal{G}=\{g_{1},g_{2},\dots,g_{n}\}$ , and a candidate pool of generator LLMs $\mathcal{L}=\{l_{1},l_{2},\dots,l_{m}\}$ , the goal is to learn a routing policy $\pi_{\theta}$ that selects GraphRAGs and generator LLMs over multiple rounds to produce the final answer $a$ under the instruction $\mathcal{T}$ . At each round $t$ , the router first performs internal reasoning $\mathcal{R}_{t}$ to select a GraphRAG $g_{t}\in\mathcal{G}$ and a generator LLM $l_{t}\in\mathcal{L}$ based on the current reasoning state $s_{t}$ . The selected GraphRAG–LLM pair then returns information $I_{t}=l_{m}\bigr(q,g_{t}(q)\bigl)$ . The information is incorporated into the state for subsequent states, progressively constructing the final answer:

		$\displaystyle P(\mathcal{R},a\mid q,\mathcal{T},I)=$
		$\displaystyle\underbrace{\prod_{t=1}^{T_{r}}P(\mathcal{R}_{t}\mid\mathcal{R}_{<t},q,\mathcal{T},I_{<t})}_{\text{Reasoning Process}}\cdot\underbrace{\prod_{t=1}^{T_{a}}P(a_{t}\mid a_{<t},\mathcal{R},q,\mathcal{T})}_{\text{Answer Generation}}$

Where $T_{r}$ and $T_{a}$ are the number of tokens in the reasoning sequence and answer, respectively. $I_{<t}$ denotes all the returned information up to token $t$ .

4 Methods

In this section, we detail GraphRAG-Router into four components: Section 4.1 describes the hierarchical routing strategy over candidate GraphRAGs and generator LLMs. Section 4.2 introduces the SFT paradigm, which equips the model with well-structured reasoning and routing formats. Sections 4.3 and 4.4 describe two complementary RL-based training stages: Stage 1 enables the model to acquire core routing behavior, while Stage 2 further regulates the invocation of generator LLMs to achieve cost-efficient routing.

4.1 Hierarchical Routing Strategy

To make GraphRAG–LLM coordination tractable, we formulate GraphRAG-Router as a hierarchical routing strategy that progressively selects retrieval infrastructures and generators through reasoning. Upon receiving a question, the model first performs internal analysis within and to assess the query’s evidence requirement, and then selects a GraphRAG within and based on the predefined GraphRAG pool and its descriptions, such as evidence granularity and retrieval specialization. The model then performs further reasoning based on the selected GraphRAG, the query, and the expected type and granularity of the retrieved evidence, and selects the most suitable generation LLM within and accordingly. Subsequently, it queries the specialist generation LLM with retrieved evidence via Query:LLM;GraphRAG . The resulting information is returned within and tags. The model may iteratively route across multiple GraphRAG–LLM pairs to gather complementary insights, which are then integrated to produce the final answer within and . This hierarchical routing strategy reduces the per-step action space from $\mathcal{O}(|\mathcal{G}|\cdot|\mathcal{L}|)$ joint choices to $\mathcal{O}(|\mathcal{G}|+|\mathcal{L}|)$ staged choices, thereby facilitating more effective optimization.

4.2 Cold Start Supervised Fine-Tuning

To equip the model with a proper routing skeleton, long-horizon reasoning ability, and initial routing capability, we first conduct supervised fine-tuning (SFT) on curated routing trajectories. To this end, we construct two types of supervised data, namely general routing traces and self-reflection routing traces, and then train the model on these trajectories to learn well-structured reasoning and routing behaviors.

General Routing Trace Generation.

We generate high-quality single-turn trajectories that correctly solve the given question while adhering to the desired reasoning, routing, and tool-use manner. Specifically, we begin by collecting GraphRAG–LLM pairs $(g_{t},l_{t})$ that can correctly solve the question via direct inference, which serve as the basis for constructing general routing trajectories. We then leverage a strong reasoning model (e.g., GPT-5.2 29) to generate complete trajectories, including the rationale for each routing decision, such as GraphRAG and generation LLM selection. More details are provided in Appendix H.2

Self-Reflection Routing Trace Generation.

Since a single routing decision may fail to select the optimal GraphRAG–LLM pair, especially for challenging questions, we further introduce multi-round self-reflective routing traces. To equip the model with self-reflection abilities, we construct self-reflective trajectories on top of single-turn traces. For each question, we first collect multiple GraphRAG–LLM pairs via direct inference. We then use a strong reasoning model to synthesize multi-turn trajectories in which the model reflects on why the previous route failed to solve the question, identifies whether the failure stems from inadequate retrieval or insufficient reasoning ability, and accordingly decides whether to switch to a different GraphRAG or a stronger LLM. In this way, the model learns to diagnose failure, adjust routing decisions, and progressively reach the final answer. More details are provided in Appendix H.2

Training Paradigm.

By constructing both general and self-reflection routing trajectories, we collect a high-quality hierarchical routing dataset, denote as $\mathcal{D}_{\mathrm{SFT}}$ . We then apply SFT to train the model to conduct expect routing strategy:

\mathcal{L}_{\mathrm{SFT}}=-\mathbb{E}_{(x,y)\sim\mathcal{D}_{\mathrm{SFT}}}\left[\sum_{t=1}^{|y|}\log P_{\theta}(y_{t}\mid x,y_{<t})\right]

where $y$ is the synthetic trajectory.

4.3 Routing Policy Alignment

Following the cold-start phase, which helps the model acquire an initial structured routing pattern and basic tool-use capability, we leverage RL-based training to further enhance the model’s tool-use ability and optimize routing strategies. We extend the general RL optimization objective with external routing pools:

	$\displaystyle\max_{\pi_{\theta}}$	$\displaystyle\mathbb{E}_{x\sim\mathcal{D},\,y\sim\pi_{\theta}(\cdot\mid x;\mathcal{G},\mathcal{L})}\Bigl[r_{\phi}(x,y)$
		$\displaystyle-\beta\,\mathbb{D}_{\mathrm{KL}}\!\left[\pi_{\theta}(y\mid x;\mathcal{G},\mathcal{L})\,\middle\\|\,\pi_{\mathrm{ref}}(y\mid x;\mathcal{G},\mathcal{L})\right]\Bigr]$

where $\pi_{\theta}$ and $\pi_{\mathrm{ref}}$ represent the policy and reference model, respectively, both of which are initialized from the SFT model, $r_{\phi}$ is the reward function and $\mathbb{D}_{\mathrm{KL}}$ is the KL-divergence. To optimize the hierarchical routing policy, we adopt a rule-based reward function including fine-grained format rewards and final outcome rewards:

r_{\phi(x,y)}=r_{\mathrm{format}(y)}+r_{\mathrm{outcome}}

Fine-grained Format Reward.

Since our framework produces structured hierarchical routing strategy, format correctness is crucial for ensuring valid reasoning traces and executable routing actions. We therefore design a fine-grained format reward that assigns progressively larger penalties to structural errors of different severity, rather than using a binary format signal:

r_{\mathrm{format}}(y)=-\min\left(1,\sum_{k=1}^{K}\lambda_{k}\,\mathbb{I}_{k}(y)\right)

where $\mathbb{I}_{k}(y)$ indicates whether the $k$ -th format rule is violated. The detailed rules are provided in Appendix B.4. This formulation provides more informative supervision for learning stable multi-round hierarchical routing behavior.

Final Outcome Reward.

In GraphRAG-Router, we leverage Exact Match (EM) to assess the correctness of the final answer generated by the routing agent with respect to the ground truth, and use it as the sole outcome reward to guide optimization:

r_{\mathrm{outcome}}=\mathrm{EM}(a,a_{gt})

4.4 Difficulty-Aware Generator Allocation

Although the Routing Policy Alignment stage equips the model with foundational routing ability, the learned policy is often feasible but not optimal, since it remains biased toward large-scale generator LLMs. In practice, a substantial fraction of questions can be handled by smaller models, and thus the unnecessary invocation of large-scale models leads to avoidable computational cost. To bridge this gap, we introduce Difficulty-Aware Generator Allocation, which adaptively penalizes the overuse of large-scale generator models according to question difficulty, encouraging cost-efficient routing without compromising answer correctness. Specifically, we first categorize questions by difficulty level, and then design a curriculum cost-aware reward that penalizes unnecessarily expensive model usage accordingly.

Question Difficulty Categorization.

Inspired by 24, we extend the definition of question difficulty to the setting of multi-scale LLM routing. Specifically, we define it as the minimum generator model scale required to answer a question correctly and reliably. Specifically, for each question $q$ , we perform direct inference using every generator LLM in the routing pool for $N$ independent trials, and compute the success rate of each model $l_{m}$ as

\mathrm{SR}(l_{m},q)=\frac{c(l_{m},q)}{N}

where $c(l_{m},q)$ denotes the number of successful trials of model $l_{m}$ on question $q$ . We then regard a model $\mathcal{L}_{m}$ as being able to reliably solve question $q$ if

\mathrm{SR}(l_{m},q)\geq\tau

where $\tau$ is a predefined success-rate threshold. Further, we define the difficulty of question $q$ by the minimum model scale that satisfies this condition:

	$\displaystyle m(q)=\min\Bigl\{s\in\{\text{small},\text{medium},\text{large}\}$
	$\displaystyle\mid\ \mathrm{SR}(q,s)\geq\tau\Bigr\}$

Accordingly, the difficulty level of question $q$ is defined as

\mathrm{D}(q)=\begin{cases}\text{Easy},&m(q)=\text{small}\\ \text{Medium},&m(q)=\text{medium}\\ \text{Hard},&m(q)=\text{large}\end{cases}

Curriculum Cost-aware Reward.

To facilitate cost-efficient routing, we first assign each generator LLM a predefined cost $C(l_{m})$ according to its model scale, so that model usage becomes explicitly measurable. Combined with the above difficulty categorization, this allows us to define, for each question $q$ , its minimum required cost $C_{\min}(q)$ as the cost of the least expensive generator LLM that can reliably answer it correctly:

C_{\min}(q)=\min\left\{C(l_{m})\ \middle|\ \mathrm{SR}(l_{m},q)\geq\tau\right\}

Based on the minimum required cost $C_{\min}(q)$ , we can quantify whether the selected generator LLM is unnecessarily expensive for question $q$ . However, such overuse should not be penalized uniformly, since harder questions require greater flexibility to explore stronger models. We therefore adopt a curriculum design that adjusts the penalty strength according to question difficulty:

r_{\mathrm{cost}}(y)=\beta\,w_{\mathrm{D}(q)}\,\max\bigl(0,\,C(l_{m})-C_{\min}(q)\bigr)

where $C(l_{m})$ denotes the cost of the generator LLM selected for question $q$ , $\beta$ is a scaling coefficient, and $w_{\mathrm{Difficulty}(q)}$ is the penalty weight associated with the difficulty level of $q$ , with larger weights assigned to easier questions and smaller weights assigned to harder ones.

Reward Shaping.

To sum up, the overall reward in the Difficulty-Aware Cost Optimization stage is formulated as:

r_{\phi(x,y)}=r_{\mathrm{format}(y)}+r_{\mathrm{outcome}}-\mathbbm{1}\{\mathrm{Correct}\}\,r_{\mathrm{cost}(y)}

where $\mathbbm{1}\{\mathrm{\mathrm{Correct}}\}$ indicates whether the trajectory yields the correct answer. This design encourages the model to improve cost efficiency under successful task completion, naturally converging to the minimal sufficient routing trajectory.

System	Methods	General QA			Multi-Hop QA
		NQ^†	PopQA^⋆	TriviaQA^⋆	HotpotQA^†	2Wiki^⋆	Musique^⋆	Avg.
Basic LLM	Direct Infer^♠	0.107	0.117	0.308	0.269	0.295	0.118	0.202
	CoT^♠	0.286	0.327	0.552	0.313	0.293	0.165	0.323
RAG-Based	Vanilla RAG	0.294	0.242	0.557	0.326	0.348	0.186	0.326
	HippoRAG2	0.353	0.266	0.601	0.395	0.374	0.263	0.375
	LinearRAG	0.302	0.269	0.608	0.385	0.435	0.238	0.373
	HyperGraphRAG	0.268	0.272	0.585	0.253	0.336	0.209	0.321
	RAPTOR	0.271	0.258	0.620	0.307	0.318	0.231	0.334
	GraphRAG	0.301	0.291	0.604	0.292	0.327	0.108	0.321
Training-free Agent	Search-o1	0.348	0.302	0.618	0.328	0.230	0.149	0.329
	GraphSearch	0.368	0.324	0.623	0.317	0.427	0.133	0.365
RL-based Agent	Search-R1	0.403	0.256	0.568	0.247	0.269	0.100	0.307
	Graph-R1	0.267	0.274	0.464	0.298	0.407	0.289	0.333
	Router-R1	0.386	0.351	0.663	0.368	0.456	0.140	0.394
Our Method	GraphRAG-Router	0.426	0.368	0.672	0.461	0.523	0.443	0.482

Table 1: Overall Exact Match (EM) on six QA datasets.

\spadesuit

denotes the best result among different scale of LLMs. ^† denotes the in-domain dataset, and ^⋆ denotes the cross-domain dataset. Bold and underline indicate the best and best baseline results, respectively.

5 Experiments

5.1 Experimental Setup

In this section, we conduct extensive experiments to answer the following key research questions (RQs): RQ1: How does GraphRAG-Router perform compared to state-of-the-art baseline models on QA tasks? RQ2: Does GraphRAG-Router yield a better performance-cost trade-off? RQ3: Can GraphRAG-Router generalize to unseen GraphRAGs and generator LLMs? RQ4: How do different components of the GraphRAG-Router contribute to performance and cost efficiency?

Datasets. We evaluate both GraphRAG-Router and baselines on six QA benchmarks: (1) General QA: Natural Questions (NQ) 14, PopQA 21, and TriviaQA 13; (2) Multi-Hop QA: HotpotQA 34, 2WikiMultiHopQA (2Wiki) 9, and Musique 30. More details are in Appendix A.

Baselines. We compare GraphRAG-Router with four distinct set of up-to-date, strong baseline models: (1) Basic LLMs: Direct Inference, Chain-of-Thought (CoT) Prompting 31; (2) RAG-based Methods: Vanilla RAG 15, GraphRAG 1, RAPTOR 27, HippoRAG2 6, HyperGraphRAG 20, and LinearRAG 37; (3) Training-free Agentic Search Systems: Search-o1 16 and GraphSearch 33; (4) RL-based Search Agent: Search-R1 11, Graph-R1 19, and Router-R1 36.

Implementation Details. We pre-train both GraphRAG-Router and baselines on the NQ and HotpotQA datasets. We construct a joint train set of 5K samples from HotpotQA and NQ. For the test, we sample 1000 data points from the original test set or development set. For a fair comparison, we use Qwen2.5-3B-Instruct 26 as the backbone LLM for all training-based methods. For the SFT stage of our method, we leverage GPT-5.2 29 to generate 450 general traces and 50 self-reflect traces, respectively. For both RL training stages, we use GRPO 28 as the default algorithm. For GrapRAGs, we select five representative frameworks: GraphRAG, RAPTOR, HippoRAG2, HyperGraphRAG, and LinearRAG. For generator LLMs, we select 5 cut-edge LLMs that cover three scale ranges: (1) Small: Qwen2.5-7B-Instruct 26, LLaMA3.1-8B-Instruct 3, Ministral-8B-2512 18. (2)Medium: Mixtral-8 $\times$ 22B-Instruct 10 and (3) Large: LLaMA3.3-70B-Instruct 3. Evaluations are conducted using two metrics, including exact match (EM) and F1-score. More details are in Appendix B.

5.2 Overall Performance (RQ1)

In this section, we evaluate the overall performance of GraphRAG-Router under both in-domain and cross-domain settings. The results are shown in Table 1. Based on the results, we observe that:

Observation 1: GraphRAG-Router consistently outperforms all baselines on QA tasks. As shown in Table 1, GraphRAG-Router outperforms competing methods, including basic LLM, RAG-based inference, training-free agent, and RL-based agent, across all six QA benchmarks. While the RL-based agent, such as Router-R1 36 improves upon other baselines by interleaved mutlti-turn search and reasoning, GraphRAG-Router achieves even stronger results, especially on multi-hop QA datasets. Notably, GraphRAG-Router exceeds the performance of leading RL-based baselines by an average margin of $+4.71\%$ on general QA and $+38.28\%$ on multi-hop QA, respectively. This highlights the effectiveness of our proposed approach.

Observation 2: GraphRAG-Router generalizes well to unseen datasets. Despite being pre-trained on NQ and HotpotQA, it maintains strong performance on cross-domain QA datasets, with an average improvement of $+18.55\%$ , compared to leading baselines. This indicates that, even pretrained on limited data, GraphRAG-Router still demonstrates strong generalization capabilities with a transferable routing policy.

5.3 Can GraphRAG-Router Generalize to Unseen GraphRAGs and LLMs (RQ2)

To evaluate GraphRAG-Router’s generalization to unseen LLMs and GraphRAG systems, we expand both the routing LLM pool and the GraphRAG pool. Specifically, we introduce two additional state-of-the-art LLMs, Qwen3-8B 32 and gpt-oss-120b 22, as well as one additional GraphRAG method, LightRAG 4. We then incorporate the descriptions of these newly added LLMs and GraphRAG systems into the routing template. Without any further training, we directly perform inference using the pre-trained GraphRAG-Router. The results are presented in Table 2.

Model	Dataset
Model	NQ	HotpotQA	2Wiki
Router-R1	0.386	0.368	0.456
Router-R1^∗	0.390	0.377	0.458
GraphRAG-Router	0.426	0.461	0.523
GraphRAG-Router^∗	0.439	0.458	0.550

Table 2: Results of the generalization capability of GraphRAG-Router.

Observation 3: GraphRAG-Router exhibits strong generalization to previously unseen LLMs and GraphRAG systems, and can be seamlessly extended to new candidates without additional training. The results, as shown in Table 2, indicate that GraphRAG-Router achieves comparable or even slightly improved performance across multiple QA datasets. Notably, it establishes new best EM scores on several benchmarks, including NQ and 2Wiki. This suggests that GraphRAG-Router does not merely overfit to the candidate set seen during training but instead learns a robust and transferable routing strategy with strong scalability, enabling it to accommodate newly introduced LLMs and GraphRAG systems effectively.

5.4 Cost-efficiency Analysis (RQ3)

In this section, we analyze the performance-cost trade-off of GraphRAG-Router. Specifically, we present the routing statistics for Stages 1 and 2 across small-, medium-, and large-scale LLMs. The results are shown in Figure 3(a) and 3(b).

Observation 4: GraphRAG-Router achieves a more favorable performance-cost trade-off by reducing reliance on large-scale LLMs. In contrast to Stage 1 and Router-R1, which predominantly route queries to large-scale models, GraphRAG-Router reduces large-model usage by nearly $30\%$ , reallocating a considerable fraction of queries to medium- and small-scale models. This result highlights the effectiveness of the proposed curriculum cost-aware reward in enabling GraphRAG-Router to adaptively select models that are better aligned with query difficulty while remaining cost-efficient. Moreover, compared with the Stage 1-only setting, GraphRAG-Router still achieves consistent improvement, underscoring its ability to optimize performance while maintaining cost efficiency.

5.5 Ablation Study (RQ4)

To answer RQ4, we conduct a comprehensive analysis of each component of GraphRAG-Router.

5.5.1 Impact of Hierarchical Routing Process

To further evaluate the effectiveness of the hierarchical routing process, we implement two additional variants of GraphRAG-Router with alternative routing strategies, namely one-time routing and LLM-first routing. For one-time routing, the model directly selects a GraphRAG–LLM pair in a single reasoning process. For LLM-first routing, we reverse the routing order and let the model choose the LLM before selecting the GraphRAG. The results are presented in Table 3, from which we draw the following conclusion:

Observation 5: The hierarchical routing strategy is consistently more effective than flat routing alternatives. Specifically, compared with one-time routing, hierarchical routing consistently achieves stronger results, showing the advantage of decomposing the routing decision into multiple stages. Moreover, among the hierarchical variants, the GraphRAG-first design outperforms the LLM-first design across all datasets, indicating that deciding the retrieval framework before the generator better aligns with the dependency between evidence acquisition and answer generation.

Strategy	Dataset
Strategy	NQ	HotpotQA	2Wiki
One-time	0.398	0.433	0.479
LLM-First	0.415	0.446	0.518
GraphRAG-First	0.426	0.461	0.523

Table 3: EM of different routing strategies.

5.5.2 Impact of Cold Start SFT

To assess the effectiveness of SFT, we isolated this stage by training the model from the base model. We then compared their EM and the average number of valid tool calls. From the results in Table 4, we draw the following conclusion:

Method	NQ	HotpotQA	Valid call
w/o SFT	0.185	0.207	0.35
w/ SFT	0.426	0.461	1.29

Table 4: Ablation study of SFT. Valid call indicates the average valid tool call for both datasets.

Observation 6: SFT serves to regularize the output format, thereby providing a better foundation for subsequent optimization. Removing SFT leads to a significant drop in both EM and the average number of valid tool calls. This indicates that SFT plays a critical role in standardizing the model’s routing behavior.

5.5.3 Ablation on RL Training Strategies

To study the impact of our proposed two-stage RL training: Routing Policy Alignment and Difficulty-Aware Cost Optimization, we conduct a stage-wise ablation study. In detail, we ablate each stage and compare the EM of these variants. Observation 7: The proposed two-stage RL training significantly enhance the performance of GraphRAG-Router. Compare to other variants, GraphRAG-Router with two stage RL training achieves superior performance, with an average improvement of $+2.31\%$ . The overall results confirm that both designed training strategies contribute positively to the performance of GraphRAG-Router.

SFT	Stage1	Stage2	Dataset
SFT	Stage1	Stage2	NQ	HotpotQA
✓	✗	✗	0.237	0.245
✓	✓	✗	0.419	0.448
✓	✓	✓	0.426	0.461

Table 5: Ablation study on training strategies of GraphRAG-Router.

6 Conclusion

In this paper, we introduce GraphRAG-Router, a cost-efficient framework that coordinates GraphRAGs and generator LLMs through agentic routing. GraphRAG-Router leverages a hierarchical routing strategy and a two-stage RL training paradigm to balance performance and inference cost. Across six complex QA benchmarks, it outperforms all state-of-the-art baselines, achieving an average improvement of $22.3\%$ while reducing the overuse of large LLMs by nearly $30\%$ .

Limitation

While GraphRAG-Router demonstrates strong performance on six QA benchmarks, we acknowledge a few limitations that present opportunities for future work. First, the current study has a limited task scope and primarily considers Wikipedia-based QA settings. it remains an open question whether the proposed GraphRAG-Router can generalize equally well to more complex retrieval-heavy tasks, such as multi-hop reasoning over long enterprise/scientific documents. Second, our current evaluation is conducted mainly in an offline retrieval setting, where the underlying corpora, graph indices, and candidate GraphRAG systems are pre-built and fixed. It remains unclear whether GraphRAG-Router can maintain the same effectiveness in more dynamic online environments with continuously updated knowledge sources and evolving retrieval indices.

References

Edge et al. (2025) D. Edge, H. Trinh, N. Cheng, J. Bradley, A. Chao, A. Mody, S. Truitt, D. Metropolitansky, R. O. Ness, and J. Larson From local to global: a graph rag approach to query-focused summarization. External Links: 2404.16130, Link Cited by: 5th item, §1, §1, §2, §5.1.
Feng et al. (2025) T. Feng, Y. Shen, and J. You GraphRouter: a graph-based router for llm selections. External Links: 2410.03834, Link Cited by: §2.
Grattafiori et al. (2024) A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, and e. a. Ahmad Al-Dahle The llama 3 herd of models. External Links: 2407.21783, Link Cited by: §5.1.
Guo et al. (2025) Z. Guo, L. Xia, Y. Yu, T. Ao, and C. Huang LightRAG: simple and fast retrieval-augmented generation. External Links: 2410.05779, Link Cited by: 1st item, §2, §5.3.
Gutiérrez et al. (2025a) B. J. Gutiérrez, Y. Shu, Y. Gu, M. Yasunaga, and Y. Su HippoRAG: neurobiologically inspired long-term memory for large language models. External Links: 2405.14831, Link Cited by: §1, §2.
Gutiérrez et al. (2025b) B. J. Gutiérrez, Y. Shu, W. Qi, S. Zhou, and Y. Su From rag to memory: non-parametric continual learning for large language models. External Links: 2502.14802, Link Cited by: 2nd item, §1, §2, §5.1.
Han et al. (2024) H. Han, Y. Wang, H. Shomer, K. Guo, J. Ding, Y. Lei, M. Halappanavar, R. A. Rossi, S. Mukherjee, X. Tang, et al. Retrieval-augmented generation with graphs (graphrag). arXiv preprint arXiv:2501.00309. Cited by: §1.
He et al. (2024) X. He, Y. Tian, Y. Sun, N. V. Chawla, T. Laurent, Y. LeCun, X. Bresson, and B. Hooi G-retriever: retrieval-augmented generation for textual graph understanding and question answering. External Links: 2402.07630, Link Cited by: §1.
Ho et al. (2020) X. Ho, A. D. Nguyen, S. Sugawara, and A. Aizawa Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps. External Links: 2011.01060, Link Cited by: 6th item, §5.1.
Jiang et al. (2024) A. Q. Jiang, A. Sablayrolles, A. Roux, A. Mensch, B. Savary, C. Bamford, D. S. Chaplot, D. de las Casas, E. B. Hanna, F. Bressand, G. Lengyel, G. Bour, and e. a. Guillaume Lample Mixtral of experts. External Links: 2401.04088, Link Cited by: §5.1.
Jin et al. (2025a) B. Jin, H. Zeng, Z. Yue, J. Yoon, S. Arik, D. Wang, H. Zamani, and J. Han Search-r1: training llms to reason and leverage search engines with reinforcement learning. External Links: 2503.09516, Link Cited by: 1st item, §5.1.
Jin et al. (2025b) J. Jin, Y. Zhu, Z. Dou, G. Dong, X. Yang, C. Zhang, T. Zhao, Z. Yang, and J. Wen FlashRAG: a modular toolkit for efficient retrieval-augmented generation research. In Companion Proceedings of the ACM on Web Conference 2025, WWW ’25, New York, NY, USA, pp. 737–740. External Links: ISBN 9798400713316, Link, Document Cited by: Appendix A.
Joshi et al. (2017) M. Joshi, E. Choi, D. S. Weld, and L. Zettlemoyer TriviaQA: a large scale distantly supervised challenge dataset for reading comprehension. External Links: 1705.03551, Link Cited by: 3rd item, §5.1.
Kwiatkowski et al. (2019) T. Kwiatkowski, J. Palomaki, O. Redfield, M. Collins, A. Parikh, C. Alberti, D. Epstein, I. Polosukhin, J. Devlin, K. Lee, K. Toutanova, L. Jones, M. Kelcey, M. Chang, A. M. Dai, J. Uszkoreit, Q. Le, and S. Petrov Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics 7, pp. 452–466. External Links: Link, Document Cited by: 1st item, §5.1.
Lewis et al. (2021) P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, S. Riedel, and D. Kiela Retrieval-augmented generation for knowledge-intensive nlp tasks. External Links: 2005.11401, Link Cited by: §5.1.
Li et al. (2025) X. Li, G. Dong, J. Jin, Y. Zhang, Y. Zhou, Y. Zhu, P. Zhang, and Z. Dou Search-o1: agentic search-enhanced large reasoning models. External Links: 2501.05366, Link Cited by: 1st item, §5.1.
Li (2025) Y. Li Rethinking predictive modeling for llm routing: when simple knn beats complex learned routers. External Links: 2505.12601, Link Cited by: §1, §2.
Liu et al. (2026) A. H. Liu, K. Khandelwal, S. Subramanian, V. Jouault, A. Rastogi, A. Sadé, A. Jeffares, A. Jiang, and e. a. Alexandre Cahill Ministral 3. External Links: 2601.08584, Link Cited by: §5.1.
Luo et al. (2025a) H. Luo, H. E, G. Chen, Q. Lin, Y. Guo, F. Xu, Z. Kuang, M. Song, X. Wu, Y. Zhu, and L. A. Tuan Graph-r1: towards agentic graphrag framework via end-to-end reinforcement learning. External Links: 2507.21892, Link Cited by: 2nd item, §2, §5.1.
Luo et al. (2025b) H. Luo, H. E, G. Chen, Y. Zheng, X. Wu, Y. Guo, Q. Lin, Y. Feng, Z. Kuang, M. Song, Y. Zhu, and L. A. Tuan HyperGraphRAG: retrieval-augmented generation via hypergraph-structured knowledge representation. External Links: 2503.21322, Link Cited by: 1st item, §5.1.
Mallen et al. (2023) A. Mallen, A. Asai, V. Zhong, R. Das, D. Khashabi, and H. Hajishirzi When not to trust language models: investigating effectiveness of parametric and non-parametric memories. External Links: 2212.10511, Link Cited by: 2nd item, §5.1.
OpenAI et al. (2025) OpenAI, :, S. Agarwal, L. Ahmad, J. Ai, S. Altman, A. Applebaum, E. Arbus, R. K. Arora, Y. Bai, B. Baker, and e. a. Haiming Bao Gpt-oss-120b & gpt-oss-20b model card. External Links: 2508.10925, Link Cited by: §5.3.
OpenAI et al. (2024) OpenAI, J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, and e. a. Florencia Leoni Aleman GPT-4 technical report. External Links: 2303.08774, Link Cited by: §1.
Parashar et al. (2025) S. Parashar, S. Gui, X. Li, H. Ling, S. Vemuri, B. Olson, E. Li, Y. Zhang, J. Caverlee, D. Kalathil, and S. Ji Curriculum reinforcement learning from easy to hard tasks improves llm reasoning. External Links: 2506.06632, Link Cited by: §4.4.
Peng et al. (2025) B. Peng, Y. Zhu, Y. Liu, X. Bo, H. Shi, C. Hong, Y. Zhang, and S. Tang Graph retrieval-augmented generation: a survey. ACM Transactions on Information Systems 44 (2), pp. 1–52. Cited by: §1.
Qwen et al. (2025) Qwen, :, A. Yang, B. Yang, B. Zhang, and e. a. Binyuan Hui Qwen2.5 technical report. External Links: 2412.15115, Link Cited by: §5.1.
Sarthi et al. (2024) P. Sarthi, S. Abdullah, A. Tuli, S. Khanna, A. Goldie, and C. D. Manning RAPTOR: recursive abstractive processing for tree-organized retrieval. External Links: 2401.18059, Link Cited by: 4th item, §1, §2, §5.1.
Shao et al. (2024) Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. K. Li, Y. Wu, and D. Guo DeepSeekMath: pushing the limits of mathematical reasoning in open language models. External Links: 2402.03300, Link Cited by: §5.1.
Singh et al. (2025) A. Singh, A. Fry, A. Perelman, A. Tart, A. Ganesh, A. El-Kishky, and e. a. Aidan McLaughlin OpenAI gpt-5 system card. External Links: 2601.03267, Link Cited by: §B.1, §4.2, §5.1.
Trivedi et al. (2022) H. Trivedi, N. Balasubramanian, T. Khot, and A. Sabharwal MuSiQue: multihop questions via single-hop question composition. External Links: 2108.00573, Link Cited by: 5th item, §5.1.
Wei et al. (2023) J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou Chain-of-thought prompting elicits reasoning in large language models. External Links: 2201.11903, Link Cited by: §5.1.
Yang et al. (2025a) A. Yang, A. Li, B. Yang, B. Zhang, and e. a. Binyuan Hui Qwen3 technical report. External Links: 2505.09388, Link Cited by: §5.3.
Yang et al. (2025b) C. Yang, X. Wu, X. Lin, C. Xu, X. Jiang, Y. Sun, J. Li, H. Xiong, and J. Guo GraphSearch: an agentic deep searching workflow for graph retrieval-augmented generation. External Links: 2509.22009, Link Cited by: 2nd item, §2, §5.1.
Yang et al. (2018) Z. Yang, P. Qi, S. Zhang, Y. Bengio, W. W. Cohen, R. Salakhutdinov, and C. D. Manning HotpotQA: a dataset for diverse, explainable multi-hop question answering. External Links: 1809.09600, Link Cited by: 4th item, §5.1.
Yu et al. (2026) C. Yu, K. Zhao, Y. Li, H. Chang, M. Feng, X. Jiang, Y. Sun, J. Li, Y. Zhang, J. Li, and Z. Zhang GraphRAG-r1: graph retrieval-augmented generation with process-constrained reinforcement learning. External Links: 2507.23581, Link Cited by: §2.
Zhang et al. (2025) H. Zhang, T. Feng, and J. You Router-r1: teaching llms multi-round routing and aggregation via reinforcement learning. External Links: 2506.09033, Link Cited by: 3rd item, §1, §2, §5.1, §5.2.
Zhuang et al. (2025) L. Zhuang, S. Chen, Y. Xiao, H. Zhou, Y. Zhang, H. Chen, Q. Zhang, and X. Huang LinearRAG: linear graph retrieval augmented generation on large-scale corpora. External Links: 2510.10114, Link Cited by: 3rd item, §2, §5.1.

Appendix A Dataset Details

We conduct evaluations on six widely used RAG benchmarks from the FlashRAG toolkit 12, covering both single-hop and multi-hop question answering tasks:

•

Natural Questions (NQ) 14. Real user questions from Google Search paired with Wikipedia passages/answers; commonly used for open-domain, mostly single-hop QA.
•

PopQA 21. Popular-knowledge question set designed for retrieval-based QA, emphasizing factual queries where the answer must be grounded in retrieved evidence.
•

TriviaQA 13. Trivia-style questions with evidence documents (often web/Wikipedia); used for open-domain factual QA and long-context evidence matching.
•

HotpotQA 34. Multi-hop QA requiring reasoning over multiple supporting Wikipedia passages; includes labeled supporting facts.
•

Musique 30 Multi-hop QA benchmark built to test compositional reasoning across several pieces of evidence, often with more challenging, structured multi-step requirements.
•

2WikiMultiHopQA (2Wiki) 9 Multi-hop QA constructed from Wikipedia that typically requires linking two (or more) pages to reach the answer, focusing on cross-article reasoning.

Dataset	Task	Knowledge Source	#Train	#Dev	#Test
NQ	General QA	Wiki	79,168	8,757	3,610
PopQA	General QA	Wiki	-	-	14,267
TriviaQA	General QA	Wiki & Web	78,785	8,837	11,313
HotpotQA	Multi-hop QA	Wiki	90,447	7,405	-
Musique	Multi-hop QA	Wiki	19,938	2,417	-
2WikiMultiHopQA	Multi-hop QA	Wiki	15,000	12,576	-

Table 6: Dataset Statistics

Appendix B Implementation Details

B.1 Baselines

Training-free Search Agents:

These approaches do not train an explicit control policy; rather, they use structured prompts and heuristic rules to steer multi-step retrieval and reasoning at inference time. Specifically, we evaluate;

•

Search-o1 16: Augments large reasoning models with an agentic RAG workflow and a Reason-in-Documents module that refines retrieved evidence before integration, enabling dynamic, noise-reduced knowledge retrieval to improve reliability on complex reasoning tasks and open-domain QA. Our implementation is based on https://github.com/RUC-NLPIR/Search-o1
•

GraphSearch 33: An agentic deep-search workflow for GraphRAG that performs multi-turn, modular retrieval with dual-channel querying over both text chunks (semantic) and structural graphs (relational), consistently improving multi-hop RAG accuracy and generation quality over traditional GraphRAG retrieval. Our implementation is based on https://github.com/DataArcTech/GraphSearch

RL-based Search Agent:

These approaches use an RL policy to optimize the agent’s search and reasoning behavior, typically adopting GRPO as the reinforcement learning algorithm. Specifically, we choose:

•

Search-R1 11: A RL-based retrieval-augmented reasoning framework that trains LLMs to autonomously generate multi-turn search queries during step-by-step reasoning, using retrieved-token masking and an outcome-based reward to improve QA performance over standard RAG baselines. Our implementation is based on https://github.com/PeterGriffinJin/Search-R1
•

Graph-R1 19: An agentic GraphRAG framework trained end-to-end with reinforcement learning that builds lightweight knowledge hypergraphs and performs multi-turn retrieval as an agent–environment interaction. Our implementation is based on https://github.com/LHRLAB/Graph-R1
•

Router-R1 36: An RL-based multi-LLM routing framework that formulates model selection and response aggregation as a sequential decision-making process. By interleaving internal reasoning with dynamic routing actions, Router-R1 can invoke multiple LLMs adaptively and optimize the trade-off between task performance and inference cost. Our implementation is based on https://github.com/ulab-uiuc/Router-R1

For GraphRAGs in main experiments, we adopt 5 representative GraphRAGs:

•

Hypergraph-based: HypergraphRAG 20: A hypergraph-based RAG framework that represents real-world n-ary facts using hyperedges and integrates hypergraph construction, retrieval, and generation. Our implementation is based on https://github.com/LHRLAB/Graph-R1
•

Entity Graph based: HippoRAG2 6:A memory-inspired RAG framework that extends HippoRAG’s Personalized PageRank retrieval with deeper passage integration and stronger online LLM usage, improving factual, sense-making, and associative memory. Our implementation is based on https://github.com/OSU-NLP-Group/HippoRAG
•

Tri-Graph based: LinearRAG 37: An efficient GraphRAG framework that avoids noisy, costly relation extraction by building a lightweight relation-free hierarchical “Tri-Graph” (via entity extraction + semantic linking) and retrieving evidence with a two-stage process—local entity activation followed by global importance aggregation—yielding stronger and more reliable passage retrieval on multi-hop QA benchmarks. Our implementation is based on https://github.com/DEEP-PolyU/LinearRAG
•

Tree based: RAPTOR 27: A retrieval-augmented approach that builds a hierarchical tree of recursive embeddings, clusters, and bottom-up summaries, enabling inference-time retrieval across long documents at multiple abstraction levels and delivering strong gains. Our implementation is based on https://github.com/parthsarthi03/raptor
•

Tree based: GraphRAG 1:A graph-based QA framework for private corpora that tackles global, corpus-level questions by (1) building an entity knowledge graph and precomputing community summaries, then (2) answering queries via summary-to-partial-response generation followed by a final aggregation, improving comprehensiveness and diversity over standard RAG at million-token scale. our implementation is based on https://microsoft.github.io/graphrag/

For the generalization experiment, we add an additional GraphRAG:

•

LightRAG 4: A graph-enhanced retrieval-augmented generation framework that integrates graph structures into indexing and retrieval to better capture entity relationships and complex contextual dependencies. By combining graph-based retrieval with vector representations and a dual-level retrieval mechanism, it improves both retrieval accuracy and efficiency. our implementation is based on https://github.com/hkuds/lightrag

For all the GraphRAGs, we utilize the context of each question as the document and organize the corpus by the official settings. For retrieval, we set the top-k as top-5. For the description of each GraphRAG, we adpot GPT-5.2 29 to summary the key method from the original paper. The detailed description is provided in H.

Learning Rate	Batch Size	Epochs	Weight Decay	Optimizer	Lr Scheduler	BF16
2e-5	4	2	0.01	Adam	Cosine	True

Table 7: Hyperparameter setting for SFT

Hyperparameter	Value	Hyperparameter	Value
Learning Rate	1e-6	Mini-batch Size	32
Train Batch Size	64	Micro-batch Size	8
Rollout Group size	5	Max Training Steps	80
KL coefficient	0.001	Warm Up Ratio	0
Max turns	4	Max Sequence Length	4096
Max Response Length	1024	Max Length for LLM Response	600
Tensor Parallel Size	1	GPU Utilization Ratio	0.45
Rollout Temperature (Train)	1.0	Rollout Temperature	1.0

Table 8: Hyperparameter settings for stage 1

Hyperparameter	Value	Hyperparameter	Value
Learning Rate	1e-6	Mini-batch Size	32
Train Batch Size	64	Micro-batch Size	8
Rollout Group size	5	Max Training Steps	40
KL coefficient	0.001	Warm Up Ratio	0
Max turns	4	Max Sequence Length	4096
Max Response Length	1024	Max Length for LLM Response	600
Tensor Parallel Size	1	GPU Utilization Ratio	0.45
Rollout Temperature (Train)	1.2	Rollout Temperature (Eval)	1.0

Table 9: Hyperparameter settings for stage 2

B.2 Generator LLMs

All the generator LLMs are accessed via OpenRouter APIs¹¹ 1 https://openrouter.ai/. We employ the model card as the description of each LLM.

B.3 GraphRAG-Router

We use verl²² 2 https://github.com/verl-project/verl as our reinforcement learning training framework. The cost of each LLM scale is set to 1 (small), 2 (medium) and 4 (large). The penalty weight of each difficulty level is: 1 (easy), 0.6 (medium), 0.2 (hard). The threshold of $SR(q)$ is set to 0.8. The scaling parameter $\beta$ is set to 0.05. The detailed hyper-parameters of each stage is provided in Table 7, 8 and 9.

System	Methods	General QA			Multi-Hop QA
		NQ^†	PopQA^⋆	TriviaQA^⋆	HotpotQA^†	2Wiki^⋆	Musique^⋆	Avg.
Basic LLM	Direct Infer^♠	0.165	0.162	0.347	0.361	0.347	0.135	0.253
	CoT^♠	0.399	0.316	0.618	0.436	0.344	0.243	0.393
RAG-Based	Vanilla RAG	0.401	0.275	0.623	0.445	0.376	0.257	0.396
	HippoRAG2	0.465	0.329	0.668	0.501	0.428	0.416	0.468
	LinearRAG	0.436	0.334	0.661	0.487	0.469	0.403	0.465
	HyperGraphRAG	0.375	0.307	0.656	0.376	0.419	0.346	0.413
	RAPTOR	0.382	0.282	0.684	0.409	0.398	0.395	0.425
	GraphRAG	0.423	0.348	0.705	0.388	0.401	0.248	0.419
Training-free Agent	Search-o1	0.476	0.354	0.672	0.439	0.289	0.237	0.411
	GraphSearch	0.483	0.371	0.683	0.425	0.458	0.190	0.435
RL-based Agent	Search-R1	0.473	0.357	0.617	0.371	0.451	0.160	0.405
	Graph-R1	0.415	0.294	0.597	0.468	0.468	0.357	0.433
	Router-R1	0.495	0.418	0.723	0.467	0.502	0.224	0.472
Our Method	GraphRAG-Router	0.525	0.403	0.735	0.589	0.591	0.563	0.568

Table 10: Overall F-1 on six QA datasets.

\spadesuit

B.4 Detail for fine-grained format reward

•

Fatal format violation ( $+1.0$ ): If the output contains no valid tags, includes unclosed or mismatched tags, or contains nested tags, the format penalty is directly set to $1.0$ .
•

Missing reasoning tag ( $+0.4$ ): A penalty of $0.4$ is applied if no <think> tag is present.
•

LLM selection before GraphRAG selection ( $+0.8$ ): If a <llm> tag appears before any valid <graphrag> tag, a penalty of $0.8$ is applied, as this violates the intended hierarchical routing order.
•

Missing GraphRAG selection ( $+0.4$ / $+0.6$ ): If no valid <graphrag> tag is identified, a penalty of $0.4$ is applied. If the model proceeds to <search> without a valid GraphRAG selection, the penalty is increased to $0.6$ .
•

Invalid GraphRAG name ( $+0.2$ ): If a <graphrag> tag is present but its content does not correspond to a valid GraphRAG option, a penalty of $0.2$ is applied.
•

Missing second-stage reasoning ( $+0.3$ ): If a <search> action is produced with fewer than two <think> tags, a penalty of $0.3$ is applied, encouraging an explicit two-stage reasoning process before execution.
•

Missing LLM selection before search ( $+0.3$ ): If the model issues a <search> action without first specifying a valid <llm> tag, a penalty of $0.3$ is applied.
•

Invalid LLM name ( $+0.1$ ): If a <llm> tag is present but its content is not a valid LLM name, a penalty of $0.1$ is applied.
•

Missing search action ( $+0.4$ ): If no <search> tag is present, a penalty of $0.4$ is applied.
•

Invalid search format ( $+0.3$ ): If the content of <search> does not follow the required format, i.e., it does not contain exactly one “:” and one “;”, a penalty of $0.3$ is applied.
•

Invalid answer cardinality ( $+0.3$ ): If the output contains zero or more than one <answer> tag, a penalty of $0.3$ is applied.
•

Empty reasoning content ( $+0.2$ ): If any <think> tag is empty or contains only a placeholder such as “…”, a penalty of $0.2$ is applied.

The final format penalty is clipped to a maximum value of $1.0$ .

Appendix C Overall F-1

The overall F-1 score is shown in Table 10. The results further indicate the effectiveness of our proposed framework.

Appendix D Additional Experiment Results for SFT

We provide the reward curves for both variants. As shown in Figure 4, the variant with SFT exhibits a noticeably more stable training process, demonstrating the necessity of SFT for effective optimization.

Appendix E Ablation on Curriculum Cost-aware Reward

To evaluate the effectiveness of our proposed curriculum cost-aware reward (CCR), we implement a variant that applies the same penalty to all questions, i.e., setting $W_{\mathrm{Difficulty}(q)}=1$ for every query. The EM results are reported in Table 11, and the routing distribution is shown in Figure 5. From the results, we observe that although applying a uniform penalty to all questions can substantially reduce the routing cost, this reduction comes at the expense of answer performance. This suggests that simply imposing a global cost penalty encourages overly conservative routing, whereas our difficulty-aware CCR achieves a better balance between cost efficiency and performance.

Method	NQ	HotpotQA
w/o curriculum	0.391	0.406
w/ curriculum	0.426	0.461

Table 11: Ablation study of curriculum cost-aware reward.

Appendix F Ablation on GraphRAG Routing Pool

In this section, we conduct an ablation study on the GraphRAG routing pool. Specifically, we consider a GraphRAG-only variant in which the router is restricted to selecting a single predefined GraphRAG framework throughout the entire reasoning process. In other words, the model is not allowed to adaptively switch among multiple GraphRAG candidates, while the generator LLM routing remains unchanged. This setting allows us to examine whether the performance gains of GraphRAG-Router truly come from diverse GraphRAG routing, or whether a single strong GraphRAG is already sufficient. The results in Table 12 show that restricting the router to a single GraphRAG consistently underperforms the full routing pool. Although HippoRAG2-only yields the strongest performance among all single-GraphRAG variants, it still falls behind the full model. This suggests that, while certain GraphRAGs can serve as strong default retrievers, no single GraphRAG is sufficient to cover the full diversity of evidence requirements across queries. These results further validate our motivation that no single GraphRAG framework is universally optimal, and that diverse GraphRAG routing is necessary to match the varying evidence requirements of different queries.

Method	NQ	HotpotQA
GraphRAG-only	0.393	0.408
RAPTOR-only	0.376	0.417
LinearRAG-only	0.398	0.433
HyperGraphRAG-only	0.368	0.401
HippoRAG2-only	0.401	0.442
ALL	0.426	0.461

Table 12: Ablation study of GraphRAG routing pool.

Appendix G Additional Results for Ablation Study on Training Strategies

In this section, we further investigate the effectiveness of the two-stage RL training procedure. Specifically, we consider a variant in which the two training stages are merged into a single training process. As Table 13 and Figure 6 indicates, this variant exhibits a performance gap compared with the full two-stage training scheme. We attribute this gap to a less favorable performance–cost trade-off during optimization: when effectiveness and cost efficiency are optimized simultaneously from the outset, the model may prematurely favor cheaper routing decisions before learning sufficiently strong routing policies, thereby sacrificing answer quality. This observation indicates the effectiveness of our two-stage RL training paradigm.

Method	NQ	HotpotQA
Mix	0.405	0.413
Two-stage	0.426	0.461

Table 13: Ablation study of different RL procedure.

Appendix H Prompt Template

In this section, we describe the prompt templates used in GraphRAG-Router for all experimental settings. We designed a set of task-specific prompts to support different stages of our framework.

H.1 Instruction of GraphRAG-Router

We construct a prompt template for GraphRAG-Router, as shown in Figure 7. The prompt guides the router through a structured decision process, where it first selects an appropriate external knowledge base and then chooses a generator LLM to process the retrieved information. It also defines a unified tagged format, including <graphrag>, <llm>, <search>, <information>, and <answer>, to make intermediate decisions machine-readable and reduce output ambiguity. In addition, the prompt supports multiple calls to different knowledge bases and LLMs, enabling GraphRAG-Router to leverage the complementary strengths of heterogeneous retrieval systems and language models.

H.2 Trace Generation Prompt

We provide prompt templates for both general trace generation and self-reflection trace generation. These templates are designed to produce structured reasoning traces for training GraphRAG-Router. The full prompts are presented in Figure 8 and Figure 9.

H.3 GraphRAG Summary Prompt

Figure 10 shows the prompt used to generate concise summaries of candidate GraphRAG frameworks from their original papers. The resulting summaries serve as auxiliary prompt context for routing and trace generation.

H.4 Descriptions of GraphRAGs and Generator LLMs

We provide concise descriptions of all candidate GraphRAG frameworks and generator LLMs, as shown in Figure 11 and Figure 12. These descriptions summarize the main characteristics and capabilities of each candidate component and are used as auxiliary prompt context for routing and trace generation.

Appendix I Case Study

I.1 Case Study of Cost-efficient Routing

We present a case study of GraphRAG-Router with cost-efficient optimization. As shown in Figure 13, the router generates a structured decision trace that explicitly records GraphRAG selection, LLM selection, retrieval results, and final answer generation. This example provides an intuitive illustration of how GraphRAG-Router performs interpretable and evidence-grounded routing.

I.2 Case Study of Routing Strategies

Figure 14 compares the previous one-time routing format with our hierarchical trace format on the same question. The one-time format exhibits two clear limitations, namely inconsistency between reasoning and the actual retrieval action, as well as the inability to revise the answer when the returned information is uncertain or insufficient. In contrast, our format explicitly supports evidence sufficiency checking and second-round rerouting, which enables self-reflection and leads to the correct answer.

Figure 7: Instruction for GraphRAG-Router

Figure 8: General Trace Generation Prompt

Figure 9: Self-Reflection Trace Generation Prompt

Figure 10: GraphRAG Summary Prompt

Figure 11: Descriptions of LLM Candidates

Figure 12: Descriptions of GraphRAG Candidates

Figure 13: Case study of GraphRAG-Router under cost-efficient routing

Figure 14: Comparison between one-time routing and hierarchical routing on the same question