Trust me, we are. In addition to the usual household of computers and half dozen television sets, we have four AC systems (15 - 16 tons combined), three dehumidifiers, more than 200 incandescent lights (most of which my wife leaves on continuously), two electric water heaters (in addition to the one on the boiler), five refrigerators, one freezer chest, and a shop full of machines lugging 3 - 5 HP motors (16" radial saw, 14" table saw, 18" & 32" band saw, 6" and 16" jointers, 3hp heavy duty planer, etc.). Every time one of the AC units kicks on, every light on the property dims to 1/2 intensity. With 6 sub-panels, panel space is actually not one of my issues.
I do think we have a main feed problem, but haven't had time to deal with it. I don't think the inrush of a 5 ton AC unit should be sufficient to cause a significant drop in mains voltage, but that's a separate issue from net capacity.