Probably because people wanting to step up from a WRX have to go to another brand. I really liked my '05 Legacy GT, but after 2012 I had nothing to replace it. That's why I went with a Kia Cadenza.
Aside from the WRX & STI, Subaru just does not prioritize performance.
I think the H6-equipped models didn't sell well because they were never performance-focused, so performance-focused drivers went elsewhere.
When the Legacy 3.0R came out, it was never really 'R' material. It was hardly even 'S' material. The Legacy GT had the performance edge with a smaller engine.
When the 3.6 came out, people were saying it would be easily capable of 300+ hp, but Subaru never changed from the 250 hp that was already available in the 3.0. Sure we got more torque, but hp sells, not torque.
I can understand Outback buyers not caring much (though there was interest when those Outback STI pics surfaced), but Legacy owners cared more.
My Cadenza makes a pretty good successor to my Legacy GT, and it's not really supposed to be a sporty car. It has 293 hp & 255 ft-lbs of torque with 6-spd auto w/ manual mode. Handling is crisp for a semi-luxury car. Sure, I wish it had a real manual like my Legacy GT, but it's pretty good for what it is.
I might move to a Stinger next. I think that's the real successor to the lost Legacy performance line.