Recent work on model calibration found that a simple variant of Platt scaling, temperature scaling, is effective at calibrating modern neural networks across an array of classification tasks. However, when negative examples overwhelm the dataset, classifiers will often be biased to producing well-calibrated predictions for negative examples, but have trouble producing well-calibrated predictions for true anomalies. A well-calibrated model – one whose scores accurately reflect the true probability of anomaly likelihood – is an invaluable tool for decision makers.