Here's the DIY http://legacygt.com/forums/showthrea...+panel+removal
Trust what it says I pulled my passenger side off to replace my mirror assembly. While your doing it it's a bit scary because you have to pull a bit harder than you would think.
Screwed the handle to armrest?? I'd pull a panel off 1st and see whats going on with it, you'll know more once you do.
After you do, if you still think you need a new one check the FS threads for newer panel's I've seen them come up before.
Sometimes the easiest way to find out info is to Google 2005 Legacy GT and your question, legacygt.com threads will pop-up.