Satellite imagery offers an efficient and cost-effective means of estimating water depth in shallow environments. However, traditional empirical algorithms for calculating water depth often are unable to account for varying bottom reflectance, and therefore yield biased estimates for certain benthic environments. We present a simple method that is grounded in the physics of radiative transfer in seawater, but made more robust through the calibration of individual color-to-depth relationships for separate spectral classes. Our cluster-based regression (CBR) algorithm, applied to a portion of the Great Bahama Bank, drastically reduces the geographic structure in the residual and has a mean absolute error of 0.19 m with quantified uncertainties. Our CBR bathymetry is 3-5 times more accurate than existing models and outperforms machine learning protocols at extrapolating beyond the calibration data. Finally, we demonstrate how comparison of CBR with traditional models sensitive to bottom type reveals the characteristic length scales of biosedimentary facies belts.