To MScruggs, have you replaced your starter relay? If it sticks it will keep the solenoid energized.
To plow boy, the starter button only energizes the starter relay. When it activates it gives + battery to the starter solenoid, which is basically another relay but can carry a lot more current to the starter. If you look at Jack's pic. above, the large post on the cap both get large battery cables attached to them. One comes directly from the + post on the battery and the other side goes to the starter motor. When the + voltage from the starter relay energizes the solenoid relay the long plunger with the spring,in Jack's picture, gets pulled inward which shorts the 2 large post together. Thus supplying direct battery to the starter motor. So if the the starter relay is stuck in a closed position,with starter button released(not pushed),the starter relay will still energize the the starter solenoid. But when the battery ground cable is removed it kills + voltage to everything.
Sorry for the long post but it's the only way to explain it without being able to point my finger at things(lol)
tourbox