Partywagon, before you go out to buy a 2nd car, have you asked yourself why you want to have a 2nd car? I was in the same position back in late 2007 after I had paid off my LGT wagon. I wanted a 2nd car mainly b/c I wanted something different - it was then I decided that I wanted a 2-door sports car that had a manual transmission and was RWD. I found a 1999 BMW M3. By the time I got the funds together to get it, it was sold! I was devastated and then I decided maybe I didn't want two fast cars. Turning 180 deg around, I got a Golf TDI. I had wanted a Jetta wagon TDI before the LGT wagon came to crash the party.
I have no regrets with the TDI - I think both cars satisfy my motoring needs very well. I would get a new one to replace the current one if I have the means.
It's sounds like you want to get a SVX b/c you have always wanted one. I hope you realize at the end of the day, it's a 18 year old car. And with cars that age, the purchase price is NOT the only $$$ you would be spending on the car. The SVX is not a sports car... it's a GT. Comfortable and luxurious for its time. I sat in one when it first came one. I thought it was really cool. The ONLY thing that bothered me about that car is its "automated mouse" seat-belts. Don't know why Subaru did that b/c it came with airbags. Those seat belts could be a point of failure amongst other things.
So I am going to vote no on this b/c the SVX doesn't offer a whole lot more than your LGT. It's a one generation run car so some parts are going to be tough to find. It's too big and heavy to be a sports car. The H6 engine is not special enough. It is not 911 flat-six special. If you are still paying on your LGT, I'd say use the money to pay it off or use it for more mods. Or save up the money and wait for the 2door sports car that Subaru and Toyota are collaborating on. Hopefully the Subaru version comes with brakes!
Now if you were considering an older air-cooled 911 or a used Miata for track use, then I would whole-heartedly say yes.