In this paper, we propose an efficient execution method of edge intelligent model based on low bit shift quantization, aiming at the issues of limited computing power and power supply resources at the edges of transmission lines, as well as slow inference and high power consumption of abnormal target detection models in term of hardware. At first, we construct a low-bit shift quantization strategy, which replaces resource-intensive full-precision floating-point multiplication with shift, addition, and flip operations. Then, we design a training method for an abnormal target detection model using low-bit shift quantization to address the issue of gradient non-differentiability caused by discontinuous weights in low-bit shift quantization. Furthermore, we implement the abnormal target detection model using low bit shift quantization based on PyTorch and have realized a change in low bit quantization convolution operators in various common sizes (including 3 × 3, 5 × 5, 7 × 7, 9 × 9) based on the ZYNQ-7020 FPGA. Finally, we conduct comprehensive experiments on our proposed method, indicating detection precision of our method comparable to the full-precision method, improvement of inference speed of convolutional computations by approximately 6.42-14.21 times, and reduction of power consumption by about 93%–95%.