Digital signal processing (DSP) is an important technology in various research fields. However, mainstream implementations of DSP algorithms either lack flexibility and scalability or have performance saturation and bottleneck, which still have the potential to be optimized. In this work, we proposed DSP-OPU, an FPGA-based overlay processor for digital signal processing. Specifically, we designed an overlay architecture suitable for various DSP algorithms. A unique data path was proposed with multiple computation engines and a reconfigurable pipeline. Additionally, we realized software-hardware co-design for DSP-OPU. On the software side, we achieved excellent scalability with a customized instruction set and a user-friendly compiler. On the hardware side, the reconfigurable data path breaks the dependency between instructions and data, enhancing scheduling ability and transmission efficiency. In addition, we also decreased the performance saturation with increasing engine numbers. Compared to C6678 SoC, our DSP-OPU achieves up to 29 $\times$ speedup and 70 $\times$ higher energy efficiency for different DSP algorithms. Compared to FPGA-based implementations for a single DSP algorithm, we achieved up to 4.5 $\times$ speedup and 4.4 $\times$ better energy efficiency.