+1 for what Colby said. I have a 97 Legacy 2.5GT that had the same symptoms. I changed the clutch and at the same time looked closely at the input shaft housing. It was grooved/worn so that the throw-out bearing would not engage the clutch pressure plate at a perpendicular 90 degree angle.
the following link from Subaru Legacy International Forum gives more info.
If you are going to do clutch take a close look at the input drive shaft housing on the clutch.