I agree that that is the best way to shop for records, let alone by far the most fun. What I'm talking about now is essentially defining the "essential" records to own, with the limit of $100 worth of essential-ness. Beatles would be sick.
I'll second The Clash - London Calling. Here's some others I would consider "essential", any combination of the following and yes there's plenty I'm missing. Bold is for emphasis. Without further ado, I present bfx's big and wholly incomplete list of badass records: The Rolling Stones - Let It Bleed Pink Floyd - Dark Side of the Moon The Beatles- Abbey Road The Beatles - Revolver Joy Division - Unknown Pleasures or Closer Neil Young - Everybody Knows this is Nowhere The Velvet Underground - The Velvet Underground & Nico Fugazi - Repeater The Descendents - Milo Goes to College The Stooges - Raw Power Husker Du - New Day Rising Sex Pistols - Never Mind the Bollocks Here's the Sex Pistols Television - Marquee Moon The Replacements - Sorry Ma, Forgot to Take out the Trash Mission of Burma - Vs. Pavement - Crooked Rain, Crooked Rain Arcade Fire - Funeral Boards of Canada - Music Has the Right to Children Miles Davis - Kind of Blue Miles Davis - Bitches Brew Dave Brubeck Quartet - Time Out Depeche Mode - Music for the Masses New Order - Substance Herbie Hancock - Head Hunters The Band - Music from Big Pink The Beach Boys - Pet Sounds Big Star - #1 Record (I don't know how available this one is but it is the essential Power Pop record) David Bowie - Ziggy Stardust The Byrds - Sweetheart of the Rodeo Nirvana - Nevermind Nick Drake - Pink Moon Fleetwood Mac - Rumours Marvin Gaye - What's Going On Elton John - Goodbye Yellow Brick Road Van Morrison - Astral Weeks Prince - Purple Rain Public Enemy - It Takes a Nation of Millions to Hold Us Back Radiohead - The Bends Simon & Garfunkel - Bridge Over Troubled Water Stevie Wonder - Innervisions