Knowledge Graphs (KGs) are used to organize and understand vast amounts of information. However, they usually suffer from incompleteness which limits their applicability. Knowledge Graph Completion (KGC) is a critical process for augmenting these graphs and is typically achieved through embedding-based and pre-trained language model based methods. Although Large Language Models (LLMs) have demonstrated significant potential for knowledge extraction and reasoning, they face challenges such as fact knowledge hallucinations which adversely affect their KGC performance. We propose Knit, a novel KG-integrated instruction tuning framework to alleviate LLM fact knowledge Hallucinations in KGC. The proposed framework comprises three key components: (1) KG-integrated information adapter, (2) knowledge prompts, and (3) KG-integrated instruction tuning strategy. These components enhance LLMs’ ability to recognize entity relationships, improve embedding interpretability, and ensure response consistency. The experimental results across four public datasets (WN11, FB13, WN18RR, and YAGO3-10) demonstrate that Knit achieves state-of-the-art performance in KGC with significant improvements in all KGC subtasks.