We introduce a new class of priors for Bayesian hypothesis testing which we name "cake priors". We will discuss add differentiate Bartlett-Jeffreys-Lindley paradoxes which include the problem of using diffuse priors leading to nonsensical statistical inferences. Cake priors allow the use of diffuse priors (having one's cake) while achieving theoretically justified inferences (eating it too). The resulting Bayesian test statistic takes the form of a penalized likelihood ratio test statistic. By controlling the rate the prior becomes diffuse, cake priors can either be constructed to control type I error (to coincide with frequentist inferences), or to be Chernoff-consistent, i.e., achieve zero type I and II errors asymptotically. Finally, we will look at some extensions using variational Bayes including multiple hypothesis testing, robust hypothesis testing, and prior-data conflict, time permitting.