I agree, it sounds like you've got your fan powered from the Accessory Circuit. With a relay, the power comes direct from the battery, with the switch side of the relay the only Accessory load.