Recently, convolution neural networks based approaches have achieved unprecedented success for image super resolution. However, such methods typically assume a predetermined degradation that deviates from real-world cases, resulting in poor performance frequently. To improve upon this, researchers have proposed multiple methods to handle super-resolution under various degradations. Despite, such methods fail to capture an accurate image prior, which is a crucial part for reconstructing image details. In this work, we propose a novel framework called Decoupled Super Resolver (DSR) with both promising performance and applicability. DSR employs an LR Finer to project a degraded image back to its clean version and a Combinational Super Resolver to retrieve a more comprehensive and accurate prior. The latter module further enables DSR to output high-resolution images by combining both image-specific knowledge and sophisticated external knowledge. Extensive experiments under various degradation settings demonstrate the effectiveness of DSR by setting new state-of-the-arts on multiple benchmarks.