Data collection for the accuracy assessment of remote sensing models can be expensive. The careful selection of samples through stratified sampling can improve the confidence in estimates of model accuracy compared to simple random sampling while using the same (or smaller) sample size.
Best practices for sample selection and accuracy assessment for categorical data products have been established, but there is little guidance in the literature for evaluating continuous variables. We demonstrate the utility of stratified sampling with optimal allocation for reducing the variance in estimates of model accuracy. Our method allocates samples based on prediction uncertainty, which can be derived from commonly used ensemble modeling approaches like random forest regression. We also test the effect of using smaller sample sizes on the effectiveness of the various sampling protocols. We show that stratified random sampling with optimal allocation provides the most precise estimate of both the mean of the reference Y and the RMSE of the population. We also demonstrate that all sampling methods demonstrate reasonably accurate estimates on average. Additionally we show that, as sample sizes are increased with each sampling method, the precision generally increases, eventually reaching a level of convergence where gains in estimate precision from adding additional samples would be marginal.
