The problem with sugar is not the calories, but the rapid absorption, quickly raising the glucose levels and a corresponding rapid rise in insulin levels, repeatedly stressing the insulin producing cells in the pancreas (way oversimplified),
The refined grains in the challah you mentioned, primarily starches, take longer to be processed into sugar, so they are somewhat better than sugar itself, but not much.
What you heard about the splenda packets is true but I don’t think that in itself is an indicator for healthfulness. Most vitamins are even more potent.
In my opinion, the majority of ill health in the modern world is due primarily to the overabundance of sugar and refined grains in the diet.