Target detection and tracking represent key challenges facing miniature fixed-wing unmanned aerial vehicles (UAVs), particularly at high cruising speeds. Therefore, this paper proposes a vision-based target detection and tracking algorithm that systematically couples two mainstream methods, namely, you only look once (YOLO) and kernel correlation filter (KCF) algorithms. This combination enables small fixed-wing UAVs to achieve reliable target detection and rapid target tracking. A customized vision-guidance module is constructed to implement this algorithm, and a dual-thread execution mechanism is developed to ensure that the computational resources are used effectively. A miniature fixed-wing UAV experimental platform is also constructed and evaluated. Flight experiments are performed, and the results demonstrate that the developed algorithm can achieve satisfactory detection and tracking accuracy for stationary and moving ground targets in complex environments.