It has been widely acknowledged that learning-based super-resolution (SR) methods are effective to recover a high resolution (HR) image from a single low resolution (LR) input image. However, there exist two main challenges in learning-based SR methods currently: the quality of training samples and the demand for computation. We proposed a novel framework for single image SR tasks aiming at these issues, which consists of blind blurring kernel estimation (BKE) and SR recovery with anchored space mapping (ASM). BKE is realized via minimizing the cross-scale dissimilarity of the image iteratively, and SR recovery with ASM is performed based on iterative least square dictionary learning algorithm (ILS-DLA). BKE is capable of improving the compatibility of training samples and testing samples effectively and ASM can reduce consumed time during SR recovery radically. Moreover, a selective patch processing (SPP) strategy measured by average gradient amplitude $|$grad $|$ of a patch is adopted to accelerate the BKE process. The experimental results show that our method outruns several typical blind and non-blind algorithms on equal conditions.