It's tactile, it's multimedia (audio + visual as opposed to just audio), it provides a different, more long form experience, there is a lot of old media that never made beyond vinyl, old classics are cheap, and new vinyl releases are usually premium items that pay a lot of attention to the visual aspect (and give you a digital download as well often). Vinyl shops have always existed, but I don't think it's a coincidence that they are now thriving when digital streaming and downloads are in ascendence.
Yeah, I wasn't very clear. Modern vinyl is a premium product that gives you a digital download and doesn't neglect the visual component generally. Contrast that with vintage vinyl where you can score some awesome albums for not much at all, even with the explosion in popularity. But yeah, no such thing as brand new vinyl that isn't expensive as far as anything I've seen.