I used a nickel, yes, the one you have in your pocket. Someone told me this, but I didn't try it until I replaced the stock pipes on a Road Star I used to own. The heel rubber (wife's) had been on the pipes for about four years.
The nickel worked like a charm and didn't leave any mark whatsoever. Took me all of 2-3 minutes to get 4 year old, baked in rubber off the pipes.
Of course, these were the stock pipes I'd just removed as I didn't want to try this until I had decided to remove them. I'd do it again though after the results.