The well-worn George Box aphorism ``all models are wrong, but some are
useful'' is particularly salient in the cybersecurity domain, where the
assumptions built into a model can have substantial financial or even national
security impacts. Computer scientists are often asked to optimize for
worst-case outcomes, and since security is largely focused on risk mitigation,
preparing for the worst-case scenario appears rational. In this work, we
demonstrate that preparing for the worst case rather than the most probable
case may yield suboptimal outcomes for learning agents. Through the lens of
stochastic Bayesian games, we first explore different attacker knowledge
modeling assumptions that impact the usefulness of models to cybersecurity
practitioners. By considering different models of attacker knowledge about the
state of the game and a defender's hidden information, we find that there is a
cost to the defender for optimizing against the worst case.