AFAIK you can't. Irritates the soup outta me too. About the most I've been able to do is to alter the pallet colors -- tools | options | color (tab) and the bottom set of sixteen "custom colors" are the charting pallet. But do anything fancy like shaded fills or fill with a picture and it all evaporates when you click on any of the field-changing buttons.
Just saw Andrew's post when I "previewed" this. So, looks like we can't stop it from happening, only undo the mess by dragging VBA into it and restoring the formatting that way. yuk