Do you keep your keys in your pants pocket?
I have several keys on my key ring along with another remote for the wife's vehicle. When I first got my LGT I was semi-regularly inadvertantly activating the door lock buttons and occasionally the trunk release, and even once the panic button.
A running change was made later in the '05 MY to key fobs so that they have the buttons more recessed to prevent accidental activation. If you have the older style, there are a couple easy mods to the innards of the fob that people came up with so that they will usually only activate with a deliberate press of a button. I think some have gotten Subaru to replace the older style with the newer one under warranty if you want to try that route.
These threads discussed the problem and home-made solutions...