Blind all-in-one image restoration models aim to recover a high-quality imagefrom an input degraded with unknown distortions. However, these models requireall the possible degradation types to be defined during the training stagewhile showing limited generalization to unseen degradations, which limits theirpractical application in complex cases. In this paper, we propose a simple buteffective adaptive blind all-in-one restoration (ABAIR) model, which canaddress multiple degradations, generalizes well to unseen degradations, andefficiently incorporate new degradations by training a small fraction ofparameters. First, we train our baseline model on a large dataset of naturalimages with multiple synthetic degradations, augmented with a segmentation headto estimate per-pixel degradation types, resulting in a powerful backbone ableto generalize to a wide range of degradations. Second, we adapt our baselinemodel to varying image restoration tasks using independent low-rank adapters.Third, we learn to adaptively combine adapters to versatile images via aflexible and lightweight degradation estimator. Our model is both powerful inhandling specific distortions and flexible in adapting to complex tasks, it notonly outperforms the state-of-the-art by a large margin on five- and three-taskIR setups, but also shows improved generalization to unseen degradations andalso composite distortions.