I really dislike the nostalgia thing, it's a cop-out and a reason to give for liking something when you don't have a real reason to do so. The only time I find it acceptable is if it's something you find to be bad but still listen to it because it reminds you of something. That's it. You can still find some stores that sell good, old records for very cheap. They're just few and far between.
If you go to enough thrift stores over a long enough time, you can find brilliant records for very cheap. However, it requires a lot of patience and time. It may be harder than it was two or three years ago, but my experiences (such as finding two copies of Quadrophenia in good condition at one Goodwill) are no more out of date than that. I do recommend garage sales and thrift (like, real thrift - Goodwill, Salvation Army) stores if you're going to go this route though. Any records at a farmer's market have already been sourced this way and up-priced. Garage sales are going to be hit or miss but if you find someone selling what was a great collection (hopefully of an uncle's/aunt's or grandparent's; a collection they're not too attached to), you've got a goldmine in front of you that's usually going to be pretty cheap too. Of course, it is easy to find records online and you can usually get them there for cheaper than you would at a farmer's market or a shop that specializes in records. I got Harvest for about $7-8 online, I suspect it would go for about $20-25 if I'd find a place and bought it in person.
You used to be able to find good records in thrift stores for a dollar. Now it's all Lawrence Welk and shitty gospel groups with bad haircuts. Of which I've bought a couple for my own purposes. I'm not complaining it's just interesting how vinyl went from trash to treasure. The best thing no one has mentioned is the art. 12x12 is so much better than any other format for a cover: