|
|
|
|
j_ung
Jan 20, 2006, 5:28 PM
Post #1 of 36
(2760 views)
Shortcut
Registered: Nov 21, 2003
Posts: 18690
|
For background on this thread, click here: http://www.rockclimbing.com/forums/viewtopic.php?t=104905
Hi all,
OK, after considering many of the suggestions that came to light in the above referenced thread, I've decided to initiate a few changes to the way RC.com handles editorial reviews. Not all of these have come into play yet, but they will.
1. A disclosure statement will be added in bold to the top of every editorial review. This statement will tell you whether or not the reviewer received the gear free or at a discounted price from the manufacturer. All RC.com reviews will always be unbiased, regardless of where the gear originated, but now you'll know right off where it came from. The disclosure statement will also tell you whether or not the manufacturer is an advertiser on RC.com at the time the review is published.
2. The policy of not allowing a negative review to be the feature unless the product is significant enough to warrant it will be discontinued immediately. I'll decide before the product is ever reviewed whether or not it will be a feature. We will continue to write our reviews to be as detailed as possible: What's it good for? What's it not good for?
3. Editorial reviews will no longer carry a highly subjective and poorly defined biner rating. This means that RC.com reviewers will no longer have any influence on any product's average rating. Until coding is figured out to make this change, I'll try to make them all 3... nice and neutral.
4. I'll try to be more careful with blurbs on the FP so that they better match the actual reviews. I realize that, often, people only see the FP blurb and that, even if the review itself is negative or lukewarm, a positive blurb can create the appearance of bias. For the record, I write those blurbs, not the reviewers.
5. I realize that without side-by-side comparisons, a product review has limited value to the discerning climber. While, for logistical reasons, I can't commit to always posting only side-by-side comparisons at the feature spot, I'll try to incorporate more of them and a few other nifty, alternative, gear-related features.
Those are all the changes for now. If you followed the above referenced thread, you know that another suggested change -- elimnating the "Users review their stuff" section on the FP -- also came up. I decided not to incorporate that one. By clicking on the YES or NO at the bottom of every review to vote whether or not it was helpful, you -- the users -- affect what does and does not make it onto that list. I have never exerted control over that feature, and will continue not to.
Thank you everybody for a very useful thread. I remain open to your suggestions regarding any and all editorial content on RC.com, including the Gear Guide features. Start a thread to start a discussion. If I don't respond, it means I never saw it. PM me a virtual ear flick. :wink:
Climb smart,
Jay Young
Content Editor
RC.com
|
|
|
|
|
shakylegs
Jan 20, 2006, 5:32 PM
Post #2 of 36
(2760 views)
Shortcut
Registered: Aug 20, 2001
Posts: 4774
|
Thanks Jay.
Another suggestion: Could you not put trip reports to trade shows in the gear review section?
|
|
|
|
|
j_ung
Jan 20, 2006, 5:42 PM
Post #3 of 36
(2760 views)
Shortcut
Registered: Nov 21, 2003
Posts: 18690
|
One step ahead o' ya; I was planning to make it the Featured Article, instead. I'll be back, folks. I have a lunch break comin'. Think I'll go boulder. :D
|
|
|
|
|
crackers
Jan 20, 2006, 7:27 PM
Post #4 of 36
(2760 views)
Shortcut
Registered: Apr 20, 2005
Posts: 416
|
I've got some thoughts on this:
1) Having negative reviews is a good idea. As mentioned in the original thread having them be a featured review isn't unless something is done to make it absolutely clear that the review is bad. I thought about this, because i like the idea of editorial consistancy, and the following ideas came up: a) an obvious headline or b) different colors (green good, red bad?). At this point though, i'm just brainstorming.
2) I'd rather not have a bold paragraph; i already skim the bold 'you're going to die' paragraphs that come with gear. Why not have a simple well designed header format that is on top of every page indicating the information?
3) Honestly, as both a manufacturer and a consumer of gear, I'd like to see something like CU. How about you stop giving gear away to your reviewers? Instead, and I know this might be unpopular, tell the reviewer at the beginning that they have the right but not the obligation to purchase the gear (at or around the prodeal price) at the end of the test with the proceeds going to an access fund affiliate or SAR team of their choice. And of course, this should be disclosed. If they do not purchase the gear, then auction the gear and donate the proceeds to the access fund or whatever.
just my $0.002
|
|
|
|
|
dingus
Jan 20, 2006, 7:36 PM
Post #5 of 36
(2760 views)
Shortcut
Registered: Dec 16, 2002
Posts: 17398
|
In reply to: How about you stop giving gear away to your reviewers? Instead, and I know this might be unpopular, tell the reviewer at the beginning that they have the right but not the obligation to purchase the gear (at or around the prodeal price) at the end of the test with the proceeds going to an access fund affiliate or SAR team of their choice. And of course, this should be disclosed. If they do not purchase the gear, then auction the gear and donate the proceeds to the access fund or whatever. just my $0.002
You may as well go back to the old model of anyone reviewing anything they please.
I'm not taking assinments where I bloody well have to pay to publish. I'm not buying any gear for the sole purpose of reviewing it. I got one bloody friend (I didn't need) to test for the Friend review. Wholesale value? 15 bucks. And now you propose I PAY THAT??? For the privilege of gettingt slagged?
Flat out... no. If that becomes policy my review days are over. I frankly don't like the implication and find it insulting.
I won't do it.
DMT
|
|
|
|
|
dingus
Jan 20, 2006, 7:37 PM
Post #6 of 36
(2760 views)
Shortcut
Registered: Dec 16, 2002
Posts: 17398
|
In reply to: How about you stop giving gear away to your reviewers? Instead, and I know this might be unpopular, tell the reviewer at the beginning that they have the right but not the obligation to purchase the gear (at or around the prodeal price) at the end of the test with the proceeds going to an access fund affiliate or SAR team of their choice. And of course, this should be disclosed. If they do not purchase the gear, then auction the gear and donate the proceeds to the access fund or whatever. just my $0.002
You may as well go back to the old model of anyone reviewing anything they please.
I'm not taking assinments where I bloody well have to pay to publish. I'm not buying any gear for the sole purpose of reviewing it. I got one bloody friend (I didn't need) to test for the Friend review. Wholesale value? 15 bucks. And now you propose I PAY THAT??? For the privilege of gettingt slagged?
Flat out... no. If that becomes policy my review days are over. I frankly don't like the implication and find it insulting.
I won't do it.
DMT
|
|
|
|
|
crackers
Jan 20, 2006, 8:02 PM
Post #7 of 36
(2760 views)
Shortcut
Registered: Apr 20, 2005
Posts: 416
|
In reply to: Flat out... no. If that becomes policy my review days are over. I frankly don't like the implication and find it insulting.
So dingus, you're saying the gear isn't worth it? Because that is the implication.
Or are you saying that you should be compensated for your work? I think reviewers should get compensated for your reviews if they get published, but I don't think reviewers should be compensated with gear. The whole thing gets muddied then, from beginning to end.
In a nutshell, this is my ethical problem with the whole system. I've kept tons of crap that I reviewed and then ebayed the stuff or sold it at the crag. I think it's unethical to get paid in the stuff you review. On the other hand, nobody's going to do it for free, and rockclimbing.com probably doesn't have the money to pay dingus $250 for a five hundred word review.
I honestly don't know what would work in this circumstance. To date, I've not heard of anybody trying to do something like Consumers Union or Cooks Illustrated, where they buy, test, and donate the products under consideration. People test gear because they are gear heads and covet gear...
aaaaiiiiii!
|
|
|
|
|
j_ung
Jan 20, 2006, 8:18 PM
Post #8 of 36
(2760 views)
Shortcut
Registered: Nov 21, 2003
Posts: 18690
|
Ideally, I'd like for us to be an a situation where we can pay cash for reviews. Perhaps someday we will be, but we're not there yet. For now, I want to give the above changes a chance to work a little and see what happens. I won't rule anything out... except charging reviewers for the review. :wink:
|
|
|
|
|
dingus
Jan 20, 2006, 8:19 PM
Post #9 of 36
(2760 views)
Shortcut
Registered: Dec 16, 2002
Posts: 17398
|
In reply to: In a nutshell, this is my ethical problem with the whole system.
Asking the reviewer to pay for gear (that has to be shit canned otherwise, from an ethical standpoint) for the privilege of having that review published? For gear she perhaps doesn't really need?
I suppose that would be a good deal for someone...
but not me, sorry. I don't like the insinuation either.
Cheers though!
DMT
|
|
|
|
|
maculated
Jan 20, 2006, 8:29 PM
Post #10 of 36
(2760 views)
Shortcut
Registered: Dec 23, 2001
Posts: 6179
|
In reply to: Or are you saying that you should be compensated for your work? I think reviewers should get compensated for your reviews if they get published, but I don't think reviewers should be compensated with gear. The whole thing gets muddied then, from beginning to end.
I'm with Dingus on this one. It does not muddy the reviews. I did a stick clip review that did not make the front page because it was an AWFUL stick clip. I got to keep that, and I spent $10 getting it a pole. I don't know where it went to. I couldn't GIVE it away. I'm sorry for the guy I had to do it for.
If I like a bit of gear, heck yeah I'll keep it, and I will also expound its glory to anyone around me (Max Cam?) which means I'm doubly helping the company that was hoping for good word from RC.com - but if I don't, I do my review and I find a better home for it. And I'll continue to critique it if I encounter it. That's what the companies need and want - good word of mouth not only published but in practice. There's no point in being forced to sell something and then buy it again with the money you make if you truly like it.
It's one thing if Jay says, "Pick things you'd like to review," but he doesn't. He says, "Want to review this product?"
|
|
|
|
|
crackers
Jan 20, 2006, 8:31 PM
Post #11 of 36
(2760 views)
Shortcut
Registered: Apr 20, 2005
Posts: 416
|
How about if you put all the gear back into a pool, reviewers got numbers and you picked your top three choices. Then mr mgt picks numbers out of a hat and you get something. Maybe but not necessarily what you reviewed?
|
|
|
|
|
caughtinside
Jan 20, 2006, 8:48 PM
Post #12 of 36
(2760 views)
Shortcut
Registered: Jan 8, 2003
Posts: 30603
|
I've got no problem with a reviewer getting an item free. Preferably, it'd be something they might use in the type of climbing they ordinarily do.
I also think Jay's claim that every review will be 100% unbiased is wrong, but that doesn't bug me. A review is a review, one person evaluating something.
Personally, I'm not going to make any purchasing decision based on just one review anyway, whether it be on RC or in the mag. So, I'm pleased when I get a variety of reviews, so I have more opinions and information, both on the one product and competing products.
I think the best gear reviews are actually forum discussions anyway. Sure, there's a lot of nonsense. But, you'll read tons of opinions, which often represent the opinion of the poster, and whatever they've heard from their climbing partners, etc. It's kind of fun considering these different opinions, and either adopting them, or discarding them as n00b ramblings.
Carry on, Booty Crew.
|
|
|
|
|
edge
Jan 20, 2006, 9:01 PM
Post #14 of 36
(2760 views)
Shortcut
Registered: Apr 14, 2003
Posts: 9120
|
I must add that I have been asked/given the opportunity to review a ton of gear in my time. It was always agreed upon in advance that I would give an honest assessment, regardless of how I obtained the gear. A reviewer who is in someone's back pocket is about as useful as the proverbial teats on a bull.
I have both lauded and crapped on gear that I have received for free, but always made an effort to be constructive. I see the same with the RC.com reviewers. If a piece of gear is worthy, then it deserves to be shown to the entire climbing community. If it sucks, however, then the manufacturer should be told as much, so that they can look outside their little box and refine or replace the errant object. I think they actually appreciate this sort of response, even though they secretly wish against it.
I have traded many an e-mail with manufacturers who have listened to my complaints and have improved (to my way of thinking) their product; I have also been told to blow chunks by others who prefer to place themselves above the fray. All I can personally promise is to tell the truth as I see it. As someone who began climbing with 3/8" goldline, steel ovals, and RR's, I have pretty much seen the whole gammut of climbing technology, and I think modern climbers would do well to learn of their equipment heritage before clicking on an internet link for their next nylon lifeline.
A big thank you to the RC.com testers. I for one appreciate all of your efforts!
|
|
|
|
|
veganboyjosh
Jan 20, 2006, 9:13 PM
Post #15 of 36
(2760 views)
Shortcut
Registered: Dec 22, 2003
Posts: 1421
|
In reply to: I have both lauded and crapped on gear that I have received for free, but always made an effort to be constructive.
this is what i find most useful about reviews i read, and least helpful when it's missing. if something sucks, don't just tell us it sucks. tell us how/why it could be better...
|
|
|
|
|
crackers
Jan 20, 2006, 9:17 PM
Post #16 of 36
(2760 views)
Shortcut
Registered: Apr 20, 2005
Posts: 416
|
Let me try to clarify something:
I think that the reviews i've read on rc.com have been good. I trust the reviewers as much as i trust any other reviewer and i welcome their wit, insight and willingness to express their thoughts. And I think they are doing a good service to the site which is both hard and to be honest, almost totally uncompensated. I genuinely appreciate everyone's efforts.
As a manufacturer, i write off the value of all goods i give for reviews from my taxes and i'd love it if everybody who reviewed my goods decided to keep them.
As a critically thinking person, I expressed my views of the ideal. Obviously, that's neither close nor feasible. But that reality does not disuade me from trying to reach the best feasible solution or offering an opinion that I thought might add value to the future.
|
|
|
|
|
maculated
Jan 20, 2006, 9:17 PM
Post #17 of 36
(2760 views)
Shortcut
Registered: Dec 23, 2001
Posts: 6179
|
I think that's the good thing about RC.com. I remember telling someone once that i refused to write reviews for gear I wasn't going to be able to actually use - I know it's standard practice in all industries to simply rewrite press releases. THAT is dishonesty, and THAT occurs all the time.
I agree with Dave, the discussions are the most useful, but someone going out and actually TESTING with that in mind is good, too. Better ifyou can see a photo of them doing it so you know they aren't just blowing smoke.
I would still love to see an RC.com link cam review - used in all scenarios including aid.
|
|
|
|
|
tim
Jan 20, 2006, 9:21 PM
Post #18 of 36
(2760 views)
Shortcut
Registered: Apr 4, 2002
Posts: 4861
|
In reply to: 3. Editorial reviews will no longer carry a highly subjective and poorly defined biner rating. This means that RC.com reviewers will no longer have any influence on any product's average rating. Until coding is figured out to make this change, I'll try to make them all 3... nice and neutral.
Don't worry too much about that. I'm checking in that change now.
In reply to: 4. I'll try to be more careful with blurbs on the FP so that they better match the actual reviews. I realize that, often, people only see the FP blurb and that, even if the review itself is negative or lukewarm, a positive blurb can create the appearance of bias. For the record, I write those blurbs, not the reviewers.
Personally, I always find it more amusing when it's an outright lie... but then again, I think the Something Awful Guys are the pinnacle of online journalism. You probably shouldn't listen to anything I say ;-)
In reply to: If you followed the above referenced thread, you know that another suggested change -- elimnating the "Users review their stuff" section on the FP -- also came up. I decided not to incorporate that one. By clicking on the YES or NO at the bottom of every review to vote whether or not it was helpful, you -- the users -- affect what does and does not make it onto that list. I have never exerted control over that feature, and will continue not to.
Thank god. Someone has to stand up to the tinfoil hat crowd now and then.
In reply to: I remain open to your suggestions
I suggest that you run a really negative review of something Five.Ten makes :twisted:
Keep up the great work, Jay. This site would be boring as shit without you managing the content and the FP changes.
|
|
|
|
|
j_ung
Jan 20, 2006, 9:30 PM
Post #19 of 36
(2760 views)
Shortcut
Registered: Nov 21, 2003
Posts: 18690
|
In reply to: Let me try to clarify something: I think that the reviews i've read on rc.com have been good. I trust the reviewers as much as i trust any other reviewer and i welcome their wit, insight and willingness to express their thoughts. And I think they are doing a good service to the site which is both hard and to be honest, almost totally uncompensated. I genuinely appreciate everyone's efforts. As a manufacturer, i write off the value of all goods i give for reviews from my taxes and i'd love it if everybody who reviewed my goods decided to keep them. As a critically thinking person, I expressed my views of the ideal. Obviously, that's neither close nor feasible. But that reality does not disuade me from trying to reach the best feasible solution or offering an opinion that I thought might add value to the future.
I love you, crackers. Friend way. :)
|
|
|
|
|
dingus
Jan 20, 2006, 10:06 PM
Post #20 of 36
(2760 views)
Shortcut
Registered: Dec 16, 2002
Posts: 17398
|
In reply to: How about if you put all the gear back into a pool, reviewers got numbers and you picked your top three choices. Then mr mgt picks numbers out of a hat and you get something. Maybe but not necessarily what you reviewed?
I think you should use what you review. I only use a haulbag for maybe a climb a year. I'm not the person who should be doing that review. I know most climbers have never had the occasion to use one at all.
Cheers!
DMT
|
|
|
|
|
dingus
Jan 20, 2006, 10:12 PM
Post #21 of 36
(2760 views)
Shortcut
Registered: Dec 16, 2002
Posts: 17398
|
In reply to: I think the best gear reviews are actually forum discussions anyway.
The recent DMM thread being a great example. I agree. A feature gear review automatically linked to the subsequent discussion? That seems in keeping with an open forum, open source site?
DMT
|
|
|
|
|
crackers
Jan 20, 2006, 10:16 PM
Post #22 of 36
(2760 views)
Shortcut
Registered: Apr 20, 2005
Posts: 416
|
In reply to: The recent DMM thread being a great example. I agree. A feature gear review automatically linked to the subsequent discussion? That seems in keeping with an open forum, open source site?
i think that's genius.
|
|
|
|
|
caughtinside
Jan 20, 2006, 10:17 PM
Post #23 of 36
(2760 views)
Shortcut
Registered: Jan 8, 2003
Posts: 30603
|
In reply to: In reply to: I think the best gear reviews are actually forum discussions anyway. The recent DMM thread being a great example. I agree. A feature gear review automatically linked to the subsequent discussion? That seems in keeping with an open forum, open source site? DMT
Yep, the reviews often do a nice job of spurring discussion. Plus, the reviewer often gets involved in the discussion, allowing for an 'interactive' review. Jay's gearland thread was a good example.
|
|
|
|
|
aryemanhattanforme
Jan 20, 2006, 10:32 PM
Post #24 of 36
(2760 views)
Shortcut
Registered: Nov 10, 2004
Posts: 29
|
In reply to: I suggest that you run a really negative review of something Five.Ten makes :twisted:
Hey wait a minute . . . :shock:
|
|
|
|
|
tim
Jan 20, 2006, 10:51 PM
Post #25 of 36
(2760 views)
Shortcut
Registered: Apr 4, 2002
Posts: 4861
|
In reply to: In reply to: I suggest that you run a really negative review of something Five.Ten makes :twisted: Hey wait a minute . . . :shock:
Heheheh. Long story. Probably will never happen, though, so rest easy.
|
|
|
|
|
|
1
