The comments section, Disqus, Rotten Tomatoes Audience Score, YouTube's thumbs up:thumbs down ratio, upvotes, downvotes, reviews... These measures of the great masses' opinions form an integral and visually unavoidable aspect of the internet. My question is, what weight do you place on them?
On the one hand, the YouTube comments section (xkcd comic) is notorious for being the saddest cesspool on the internet. On the other hand, I trust implicitly the ratings accorded to different products on Amazon. Yet the populations can't be all that different. I confuse myself when some part of me is disappointed that a book a friend recommends is less than 5 stars on Amazon. It's patently ridiculous that I'd trust 600 complete strangers rather than a close friend, but I can't find a satisfying answer as to why I check the pulse of the crowd.
Do you ignore the comments entirely? Are some sites better than others? Do you find yourself trusting, say, the discussion underneath The Atlantic more than The New York Times? Or is hubski the high bar you set?
You can't assume that all aggregates are created equal. Amazon reviews, for example, are stupidly easy to buy. Youtube comments are generated by people who think commenting on Youtube videos is a form of expression; Yahoo! Answers are mostly Something Awful goons trolling each other. You have to understand the motivation of reviewers, as well as their investment in their reviews. That pretty much makes Rotten Tomatoes wholly and absolutely useless (much like all movie reviews - ask me about the time I "wrote" a movie review for The Stranger). Tech reviews on Newegg or Amazon, on the other hand, can be evaluated on face value - "I hate this product because it failed this explicit, technical task that it had every reason to pass" vs "WORST PRODICT EVAR WOOD GIVE ZERO STARZ IF i COULLD" self-polices. You trust the aggregate because you have no insight into how it is created. Therefore the big number matters. Sometimes, I use that approach: "I need USB cables. Here's a set with 8,000 mostly 4-star reviews. They probably don't suck too hard." However, my buddy is looking for fuel injection for his truck and I got a recommendation from someone who has put FAST systems on no less than 8 jeeps. Do I give a fuck that it's got like 3 stars most everywhere? No. No fucks given. Gearheads are fucktards and gearheads attempting to use computers are shit-throwing monkeys.
I intuit that there's a spectrum that aggregates fall along. But it's hard to typify the aggregates with a high signal-to-noise ratio, because it's not merely that they're "moderated" or un-self-interested or a function of how deeply the participants care. I'm a Rotten Tomatoes snob (meaning, I'm highly influenced by critics aggregate and won't see movies if they fare too lowly). I've noticed that in the last five or so years, I've watched fewer movies. I figured that it was mainly because I'm busier, but in thinking about it, I realize that I just exported my decision making and film going is less enjoyable. Watching a movie is now a weird confirmation of the critics score instead of a single-entendre act of watching a movie and enjoying it (or not). What makes Rotten Tomatoes useless in your eyes?
Dude. I like you enough that I'm going to ignore this comment after a slight shaming, but I can't let it slide. Crowdsourcing is perhaps the worst way of deciding what's good (even when the crowd is critics--especially when the crowd is critics, who are useless jerkoffs who spend time sitting in judgement). By this measure Brittany Spears was the best musician of 1997 (or whenever) because everyone wanted to fuck a 16 yr old in tight red vinyl. Crowds are full or morons, and aggregating a crowd's opinion guarantees nothing but mediocrity. Rotten Tomatoes is the internet version of actual human garbage and raw sewage. As an aside, if you want to know how worthless critics are generally, go back and read reviews of 2001 from 1967 and then from decades later, after time had proven it ot be among the greatest movies of all time. Many of the same critics changed their minds miraculously. Critics generally are scum. I except Roger Ebert from that group, because he was a very talented writer who wrote on many topics with great insight, and even when he hated a movie he was very good at putting it in context. I would recommend never reading a review until after you've seen a movie, because perhaps you'll gain some better insight. You're only making yourself seem foolish by using the opinion of others as a proxy for your own.I'm a Rotten Tomatoes snob.
No, you're right. I feel a vague sort of laziness when I check Rotten Tomatoes. Like I'm relinquishing responsibility for developing a personal taste. It started because of the expediency -- if you don't but have one evening for movie watching, you want to make sure the experience is worthwhile. But it's grown into a way larger thing than I'd like to admit. I'm resolving to stop checking RT before seeing a movie.
1989: the year Keyboard Magazine cited Jane Child as "Debut Album of the Year."
Play this game for me: 1) List, oh, five movies you know really well. Like, don't like, doesn't matter. If you're really into it, list ten. Doesn't matter though. 2) Next to each one, write down what you think of that movie out of ten. Good, bad, indifferent, write it down. 3) Go look those movies up on Netflix. What does Netflix predict you'll think of those films? How's the correlation? 4) Go look those movies up on Rotten Tomatoes. How's the correlation now? See, I'm educated about film. I'm a film professional. I'm surrounded by film professionals. And if you think you and your friends have differing views about films... I mean, my best friend since 8th grade has opinions so different than mine that I deliberately avoid any movie he likes. He's got an MFA in this shit. I've gotten into flame wars over Network and Crash. I know the guy who passed on Being John Malkovich for HBO... and we nearly came to blows over Children of Men (and I happen to hate BJM). And everybody on all sides can make erudite, intelligent arguments about why they're objectively right and why you're objectively wrong and neither one of us has a leg to stand on because at the end of the day, it's art. Ever looked up the contemporary reviews of, say, Empire Strikes Back? Blade Runner is just as bad. How 'bout A Christmas Story? Even individual reviewers often restate their opinions. And while I usually agreed with Roger Ebert, he was offensively, stupidly wrong so often that even a reviewer you trust can often be completely full of shit. So. There's probably no correlation between what you think and what any one reviewer thinks. There's demonstrably no correlation between what you think and what every reviewer thinks. So why give the mantrolls of Rotten Tomatoes the power to tell you that Ghostbusters sucks because it's full of GURLZ? I mean, if you care, see it for yourself and determine for yourself why it sucks (or not).
OK, listing five movies I love and five I dislike, with they x/10 score I believe they contemporaneously received. 1) The Shawshank Redemption (10/10). Instantly recognized masterpiece. 2) Tommy Boy (8/10). Hysterical, even though retard jokes don't age well out of the 90s. 3) Children of Men (10/10). Cinematic marvel, thought-provoking sci-fi at some of its best. 4) Michael Clayton (9/10). Underrated thriller. 5) The Big Lebowski (10/10). They must have known this would be a cult classic. 6) Enough Said (4/10). I just remember wanting two hours back. 7) Wild (probably got 10/10). I disliked the movie almost as much as I disliked the first 50 pages of the book I read. 8) American Sniper (probably got an 8/10). I was distracted by the raw propaganda of this movie. I've been told that if I hated this movie, I should stay away from Michael Bay's 13 Hours. 9) Ex Machina (8/10). Twice I could not get more than halfway through this movie. 10) .... 2001 A Space Odyssey. (10/10). I know I will be called a stupid, unappreciative millennial. But I spent a month reading Anna Karenina everyday that went by faster than this movie. I can almost feel how wrong I am on this one, but I just can't force myself to like it. Alright, firing up Rotten Tomatoes. 1) The Shawshank Redemption (10/10). 91% 2) Tommy Boy (8/10). 44% (LOL) 3) Children of Men (10/10). 92% 4) Michael Clayton (9/10). 90% 5) The Big Lebowski (10/10). 81% 6) Enough Said (4/10). 96% (Holy Jesus what the fuck?!) 7) Wild (probably got 10/10). 90% 8) American Sniper (probably got an 8/10). 72% 9) Ex Machina (8/10). 93% 10) .... 2001 A Space Odyssey. (10/10). 94% Ok, so I wasn't totally off. I was generally within 10%, except for the times I wasn't. I don't know if my aim is good or not, or if I even want it to be good. But certainly there's an effect of priming: if a movie is rated poorly, I brace myself to enjoy it less, which is a wholly unnecessary waste of 12 bucks. I don't need to give that power away.
Ahh, but what did Netflix think? Because Netflix's algorithm is refined to your inputs. It uses affinity to track how well it thinks you will like something. It's our magic new data driven studio that creates content based on what people really want! And it gave us Orange is the New Black! And House of Cards! And Marco Polo. And that's my point: at the end of the day, the best curve fit is still an approximation of what you'll think, and it doesn't know things like "kleinbl00 hates watching OITNB Season 1 because the sound is shit, and Season 2 because you guys have never asked me how I feel about fat lesbians getting naked together." Some things it doesn't track well. The Napoleon Dynamite Problem has been solved for like eight years now and Netflix still gives four stars to shit that I hate, and two stars to shit that amuses the hell out of me (Netflix doesn't get irony). And never mind the averages. Of your ten movies, you gave perfect scores to half of them. Of those four, three of them effectively got an A-. One of them got a B-. Doesn't mean they're wrong, it means they aren't tracking your tastes. Which is pretty much this discussion in a nutshell. Again - I'll use reviews that illuminate what, exactly, is broken about something. I didn't buy a phone system because a couple reviewers said "yeah, it supposedly does this thing, but we haven't gotten it to work." I got the more expensive version because everyone said "this thing is a total piece'o'cake to set up." It totally isn't, by the way... but tech support is super helpful so I'm happy with my choice. It's all in how you use it.
I've done a lot of research into reviews (because I was tasked once with designing an infallible reviewing system), and, conservatively speaking, 70% of all the reviews you see on the major sites are fake. (Where "fake" means purchased, biased, trade-for-product, etc., anything where it isn't someone spending their own money, trying the product out, and then reviewing it.) And honestly, the entire model of crowd-sourced reviews is completely fucked anyway. How many beds will you own in your lifetime? 5? 6? And the store has 30 models that are renewed every 2-3 years? Your review means less than nothing because you do not have the wide range of experience necessary to make an accurate assessment of the actual features of any bed. Now, if you are a professional car or motorcycle reviewer, your opinion is also worth exactly nothing. Because all you ever do is drive new cars, and - at the absolute best - only 50% of car sales in the US are new cars, so your entire skillset is completely worthless for half of the people buying cars. (And that's best case. The real number seems to be more like only 15-20% of car sales are new cars. All the other 30+ million vehicles sold every year are used.) Go through any 25 Amazon products quickly. Scroll down to the reviews section. Scan the first 10 reviews. Jump to the next product. Scan the first 10 reviews. Repeat. You will see the pattern pop out pretty quickly. There will be one genuine review for each product (probably) and all the others will be fake. Over-general, vague, or even say in the copy "I got the product for free so I would write this review, but I don't care! I'd still buy it!" Anyone who reads the comments/reviews for any reason other than entertainment, is simply fooling themselves.
I've seen so many beautiful/informative comments on YouTube. I suppose if you are watching common, controversial or popular stuff that you'll be exposed to the masses and their course discourse. I feel like people who hate YouTube and Twitter must be doing it wrong, it's like they hate resturants because they had a bad time at a McDonald's on the wrong side of town. On Amazon I look at the worst reviews first to see what can go wrong with a product. Half of the worst reviews are often people who didn't buy the right product or bought the wrong size. The other half give a pretty good idea about the how poor design or quality might let you down. As far as aggreggate star or thumbs up or down goes, and extremely bad score with a lot of votes is a quick dismissal, the more reviews the better and anything three or more is probably worth a look if there aren't many products in it's class. For stuff like electronics I'm way more likely to check out what The Wirecutter suggest than wade through a bunch of comments. Any time there is a freak with a blog that does something like review every pourover kettle, I'm likely to weight his score more than the random commentators.
Can you share any such freaks?Any time there is a freak with a blog that does something like review every pourover kettle, I'm likely to weight his score more than the random commentators.
I never read the comment section of any video, or news article. Most people have legit no clue what they are talking about. I actually learned that from Reddit. The number of times you go into the comment section, and see the top comment is full of shit is staggering. This is usually about politics and other stuff though. In terms of product review, it honestly depends on the product I'm looking for if it's something non-tech I will usually just take whatever has a decent rating-to-price ratio. If it is highly technical, then I usually have to consult a lot more than just the reviews, ratings, and critics. For example I did extensive research on all things involving computer components, before I built my first tower. I knew the essentials of every piece and why spending a hundred to two hundred dollars on what people considered "non-essential" or "high-end" was actually worth it for the amount of life you can get out of those components. Rotten Tomatoes I have to say I often look to. In my defense I will say that if the critic score is shitty, but the audience rating is high I will still see a movie.
On hubski there was a discussion around a relevant observation, the Murray Gell-Mann Amnesia effect: Briefly stated, the Gell-Mann Amnesia effect works as follows. You open the newspaper to an article on some subject you know well. In Murray’s case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward-reversing cause and effect. I call these the “wet streets cause rain” stories. Paper’s full of them. In any case, you read with exasperation or amusement the multiple errors in a story-and then turn the page to national or international affairs, and read with renewed interest as if the rest of the newspaper was somehow more accurate about far-off Palestine than it was about the story you just read. You turn the page, and forget what you know. That is the Gell-Mann Amnesia effect. I’d point out it does not operate in other arenas of life. In ordinary life, if somebody consistently exaggerates or lies to you, you soon discount everything they say. In court, there is the legal doctrine of falsus in uno, falsus in omnibus, which means untruthful in one part, untruthful in all. But when it comes to the media, we believe against evidence that it is probably worth our time to read other parts of the paper. When, in fact, it almost certainly isn’t. The only possible explanation for our behavior is amnesia.Media carries with it a credibility that is totally undeserved. You have all experienced this, in what I call the Murray Gell-Mann Amnesia effect. (I call it by this name because I once discussed it with Murray Gell-Mann, and by dropping a famous name I imply greater importance to myself, and to the effect, than it would otherwise have.)
I can honestly say that I pay little to no attention. I do peruse written reviews, because you can usually get some glimpses of truth out of the patterns in them. But, if something doesn't have 2 stars with a lot of votes, then I don't fret anything ranging between 3-5. I mean, look at this shit: https://www.amazon.com/Power-Secret-Rhonda-Byrne/dp/1439181780/ref=sr_1_9 I recently made the mistake of booking a Travelodge for a trip that thenewgreen and I took together, because it had the most stars (3) for anything with a comparable price. I should have known nothing at that price could be 3 stars.
It was, without question, the shittiest hotel I have ever stayed in. And I have stayed in some shitty hotels back in the day. What a fun trip though. All part of the journey.
Those are reviews. What about comments? Under the article, not just hacker news or a favorite subreddit. Is there such a site?
Hey - that's 1452 people who got exactly what they paid for. Was it Flagstaff HoJo bad?