If you go to enough thrift stores over a long enough time, you can find brilliant records for very cheap. However, it requires a lot of patience and time. It may be harder than it was two or three years ago, but my experiences (such as finding two copies of Quadrophenia in good condition at one Goodwill) are no more out of date than that.

I do recommend garage sales and thrift (like, real thrift - Goodwill, Salvation Army) stores if you're going to go this route though. Any records at a farmer's market have already been sourced this way and up-priced. Garage sales are going to be hit or miss but if you find someone selling what was a great collection (hopefully of an uncle's/aunt's or grandparent's; a collection they're not too attached to), you've got a goldmine in front of you that's usually going to be pretty cheap too.

Of course, it is easy to find records online and you can usually get them there for cheaper than you would at a farmer's market or a shop that specializes in records. I got Harvest for about $7-8 online, I suspect it would go for about $20-25 if I'd find a place and bought it in person.