Consistent thought tree with retrieval: An optimization framework for reasoning in large language models

Large language models (LLMs) excel at text generation and complex problem solving but are prone to factual and logical errors in multidimensional reasoning due to internal knowledge limitations. Although retrieval-augmented generation (RAG) mitigates this problem by integrating external knowledge, it introduces knowledge noise, multisource conflicts, and a disconnection between retrieval and reasoning. Structured approaches such as chain-of-thought and tree-of-thought enhance reasoning via explicit intermediate steps, but they rely on internal knowledge without effective fact-checking. Recent methods that fuse RAG with structured reasoning lack coordination in situations involving conflicts between external and internal knowledge, and tree architectures focus locally without ensuring cross-level consistency. To address these issues, this study develops a framework that deeply integrates retrieval with tree-structured reasoning and enforces consistency constraints to guarantee factual accuracy and logical coherence.

This study proposed the Consistent Thought Tree with Retrieval (CTTR) framework, featuring three core designs. First, dynamic retrieval embedding enabled context-aware knowledge acquisition at each tree node for on-demand external knowledge fusion. For each node, a retrieval query was built from the content and context, and the top-k documents were retrieved via cosine similarity. A reading-note mechanism processed the retrieved content through the LLM to ground each branch in accurate facts. Second, knowledge consistency verification resolved conflicts by comparing the semantic consistency among three results: those obtained from internal knowledge, external retrieval, and fused synthesis. Their embeddings were computed, and similarities were ranked; finally, the highest pair was selected as the output. Irrelevant branches were pruned to prevent noise propagation. Third, node consistency evaluation applied majority voting and confidence assessment among nodes of the same level, thereby enhancing reliability via multipath convergence and avoiding single-chain deviations. The algorithm processed each tree layer through retrieval, verification, optimization, and integration to produce the optimal result.

Experiments encompassed four tasks using GLM4-9B and DeepSeek-R1-Distill-Llama-8B: hallucination detection (TruthfulQA with 817 questions, HalluQA Chinese with 452 questions), code generation (HumanEval with 164 problems), the 24-Game (100 problems from 4nums), and creative writing (science fiction and historical narrative evaluated by GPT-4 scoring and human review). CTTR achieved 66.50%/57.78% on GLM4-9B and 91.31%/45.56% on DeepSeek-R1-Distill-Llama-8B for hallucination detection, averaging an improvement of 2.89% over the retrieval-augmented thought tree (RATT). Code generation reached 73.17%/87.20% and 75.61%/88.41% pass@1/pass@5, showing an improvement of 3.96%. Further, the 24-Game success rates of 11% and 24% indicated an improvement of 3.50%. Creative writing scores reached 8.96/8.81 and 9.57/8.45, reflecting an improvement of 1.24%. Ablation studies confirmed that removing knowledge selection (KS) or consistent nodes (CN) degraded the performance, with KS contributing the most; this validated the necessity of dual consistency. Resource analysis showed that compared to RATT, CTTR reduced token consumption by 11% and the time required to 247.67 s by pruning redundant retrievals. Poisoning attack tests demonstrated the robustness of CTTR against malicious noise injection.

CTTR effectively addresses the need to maintain logical consistency and factual accuracy for complex problems. By embedding retrieval into tree-structured reasoning with dual consistency evaluations, it enhances hallucination detection, code generation, mathematical reasoning, and creative writing. The results confirmed that retrieval–tree fusion strengthens complex reasoning, and that dual consistency mechanisms are task-adaptive. KS is prominent in fact-intensive tasks, while CN is valuable in open-domain generation. CTTR improves accuracy while also optimizing resource efficiency. As a training-free framework deployable without fine-tuning, CTTR reduces application costs. Future work will explore the dynamic mechanisms that adaptively switch paths by task complexity, using direct responses for simple tasks and tree reasoning for complex ones to balance efficiency and accuracy.