|
rrrADAM
Feb 27, 2004, 5:46 PM
Post #27 of 78
(7259 views)
Shortcut
Registered: Dec 19, 1999
Posts: 17553
|
Not sure, but I think the original code used to have all Highs @ 9.00, Meds @ 8.00, and Lows @ 6.00 when they first are approved.
As for trying to minimize vote bombing when mediocre pics get 10's for a day, what do you think about setting pics to those defaults, then after 4 votes recalcing it to it's propper place ??? If we do it this way, we can either remove the 24 hr recalc, or reduce it to a 4 hr interval.
Just some thoughts.
Of course keeping the "out liners", or whatever it's called is a great idea. Tossing the bottom and top 10% would leave the middle 80%, and that should be a pretty true representation of what the general public thinks of the pics... This would remove bomb votes, as well as friend votes. :wink:
|
|
|
|
|
hardmanknott
Feb 27, 2004, 5:46 PM
Post #28 of 78
(7259 views)
Shortcut
Registered: Jan 15, 2003
Posts: 228
|
In reply to: The problem wasn't really people bombing them, it was that somone voted 10 initally, then other people who think that your photo doesn't deserve a 10, overcompensate.
Can someone please explain to me why people are allowed to vote
on their own photos? This seems to be the heart of the problem.
It explains all the mediocre 10's on the Photo Page. Uhh, hello?
In reply to: I am trying to think of a way to avoid this, but it seems hopless.
It might be easier than you think...
Hardman Knott
|
|
|
|
|
rrrADAM
Feb 27, 2004, 5:54 PM
Post #29 of 78
(7259 views)
Shortcut
Registered: Dec 19, 1999
Posts: 17553
|
They can't, as votes by the submitter are ignored.
How did you read that in your quote of Biff ???
|
|
|
|
|
hardmanknott
Feb 27, 2004, 6:02 PM
Post #30 of 78
(7259 views)
Shortcut
Registered: Jan 15, 2003
Posts: 228
|
In reply to: They can't, as votes by the submitter are ignored.
Is this a recent change? How do you account for all the pics that seem
to be 10's right out of the gate? How do they get a 10?
It sounds like a conspiracy, man....
Hardman Knott
|
|
|
|
|
rrrADAM
Feb 27, 2004, 6:18 PM
Post #31 of 78
(7259 views)
Shortcut
Registered: Dec 19, 1999
Posts: 17553
|
About a year old I think... At least 6 months old, but Biff can tell you for sure, as he wrote much of the newer code for the Gallery.
Not sure about the 2nd... I would have to guess that many users send friends PMs with links to the pics. Please see my above post regarding "...bomb votes & friend votes...", which are just as worthless for a true voting average as are the bomb votes.
As for the conspiracy thing... Pay no attention to the man behind the curtain, there is nothing to see here, move along. :wink:
Oh yea... And OJ was innocent !!! :lol:
|
|
|
|
|
jt512
Feb 27, 2004, 7:04 PM
Post #32 of 78
(7259 views)
Shortcut
Registered: Apr 12, 2001
Posts: 21904
|
In reply to: Really it comes down to this .. when there are 2 votes on opposite sides of the spectrum .. how do you decide which is honest?
For the two-vote problem I would say, either:
1. Give greater weight to the higher vote, or
2. Give greater weight to the earlier vote.
(1) seems more appropriate if you think that the higher vote is usually the more honest one, regardless of whether it is the first or second vote, whereas (2) seems more appropriate if you think that the later vote is likely to be an over-reaction to the earlier vote.
In general, I believe that no vote should be given a weight of 0. I see the votes in terms of a model in which all votes have a probability of being honest, no matter how disparate they are. However, the more a vote differs from the mean, the more likely it is to be dishonest. In theory, then, an outlying vote's weight should equal its probability of being honest. Assigning weights is, thus, a two step process. First, outliers need to be identified; and second, an appropriate weight needs to be assigned.
This won't solve the 2-vote problem, but once you have enough votes, you can use the number of standard deviations (SD) a vote is from the mean to determine whether the vote is an outlier. I suggest using the SD because it is a true measure of how outlying a vote is. Simple distance from the mean is not, which I can I demonstrate with a counterexample by PM if you are interested. For simplicity, I would suggest using two cutpoints, beyond which reduced weights would be assigned. You could, for instance, assign a weight of 0.5 to votes that are more than ± 1.65 SD from the mean, and a weight of 0.25 to votes that are more than ± 2 SD from the mean. You could pick the weights and the cutpoints arbitrarily, but it sounds like you have a good enough feel for the data to make educated estimates.
BTW, after hearing your reasoning behind the delay, I've changed my mind, and now agree with you.
-Jay
|
|
|
|
|
the_pirate
Feb 27, 2004, 7:23 PM
Post #33 of 78
(7259 views)
Shortcut
Registered: May 20, 2003
Posts: 3984
|
I shall, once again suggest that the votes and voters be shown as a way to encourage honesty and limit bombing.
At least bring back the sort by user's votes feature. That was fabulous.
|
|
|
|
|
drkodos
Feb 27, 2004, 7:32 PM
Post #34 of 78
(7259 views)
Shortcut
Registered: Dec 21, 2002
Posts: 2935
|
The whole site is political.
Move away from a statistical approach. It is art. Photography is an art form.
Appoint a board of editors. Have them decide. Alow people trained in the field with more expertise to make decisions based on a consensus view of a panel of 5 to 12 (or some such number). In the same way I must concede that a statistician knows more about numbers than I, it is time to realize there are some here that have a better eye and talent for judging photo quality and these people should be appointed the task igf they are willing.
We already have a crew of photo editors and we allow them to decide what photos get accepted so arguments against improving their editorial decision skills to look for certain types of photos, maintain variagation, and keep things from being monoplozied by just several contributers (as is the case now).
There are many ways to go and I am sure it will be improved. It is just a shame that so many times improvements are kept under wraps here with a veil of secrecry as if it were a Romulan Cloaking device.....
|
|
|
|
|
crotch
Feb 27, 2004, 8:07 PM
Post #35 of 78
(7259 views)
Shortcut
Registered: Jan 16, 2003
Posts: 1277
|
In reply to: The whole site is political. Move away from a statistical approach. It is art. Photography is an art form.
And art is a subjective media.
In reply to: Appoint a board of editors. Have them decide. Alow people trained in the field with more expertise to make decisions based on a consensus view of a panel of 5 to 12 (or some such number).
Why have someone else tell me what I like? My subjective opinion is as valid as anyone else's, I submit.
|
|
|
|
|
biff
Feb 27, 2004, 8:36 PM
Post #36 of 78
(7259 views)
Shortcut
Registered: Nov 5, 2001
Posts: 851
|
JT .. I'm glad I convinced you that overneight calculation is good .. but now I have come up with a system that I think might be even better. (read below)
Kudos: Keep the cloaking device quiet .. I sold that to the CIA yeas ago .. how else do you think I paid for my education / rack / photo equimpment?
honestly .. there is no secret code, there is just soo much stuff in these forms that to find where the specs are released is hard. I have written in about 4 threads about how the outlier removal works, good luck finding it though.
So After reading the pots here .. I have a proposal to y'all
1) Only show rating to owner if it has less than 4 votes (but still use the [calculated but not shown] rating to display photos when sorted by rank)
2) Only calculate rating overnight if there is less than 4 votes on the photo. Otherwise recalculate immediatly.
This will:
1) make the first 4 votes more honest (hopefully)
2) Still make it difficult for one person to bomb a photo from the first page of top results (it would take 3 more votes to accomplish)
3) Quickly get rid of photos from the first page of results if they don't deserve to be there.
I think it would work .. can anyone see a problem with it?
Edited for clarity
|
|
|
|
|
popol
Feb 27, 2004, 8:37 PM
Post #37 of 78
(7259 views)
Shortcut
Registered: Nov 9, 2003
Posts: 390
|
Just exploring the editor idea:
Why not including 3 (?) editor's votes in the approval procedure? That way, less boosted 10's will appear, and bombing pictures will have less effect due to the statistical algorithms already in place. Also the 2-vote problem is solved that way.
|
|
|
|
|
ambler
Feb 27, 2004, 8:49 PM
Post #38 of 78
(7259 views)
Shortcut
Registered: Jul 27, 2002
Posts: 1690
|
In reply to: In reply to: Really it comes down to this .. when there are 2 votes on opposite sides of the spectrum .. how do you decide which is honest? For the two-vote problem I would say, either: 1. Give greater weight to the higher vote, or 2. Give greater weight to the earlier vote. (1) seems more appropriate if you think that the higher vote is usually the more honest one, regardless of whether it is the first or second vote, whereas (2) seems more appropriate if you think that the later vote is likely to be an over-reaction to the earlier vote. In general, I believe that no vote should be given a weight of 0. I see the votes in terms of a model in which all votes have a probability of being honest, no matter how disparate they are. However, the more a vote differs from the mean, the more likely it is to be dishonest. In theory, then, an outlying vote's weight should equal its probability of being honest. Assigning weights is, thus, a two step process. First, outliers need to be identified; and second, an appropriate weight needs to be assigned. This won't solve the 2-vote problem, but once you have enough votes, you can use the number of standard deviations (SD) a vote is from the mean to determine whether the vote is an outlier. I suggest using the SD because it is a true measure of how outlying a vote is. Simple distance from the mean is not, which I can I demonstrate with a counterexample by PM if you are interested. For simplicity, I would suggest using two cutpoints, beyond which reduced weights would be assigned. You could, for instance, assign a weight of 0.5 to votes that are more than ± 1.65 SD from the mean, and a weight of 0.25 to votes that are more than ± 2 SD from the mean. You could pick the weights and the cutpoints arbitrarily, but it sounds like you have a good enough feel for the data to make educated estimates. |
|
|
|
|
drkodos
Feb 27, 2004, 8:51 PM
Post #39 of 78
(7259 views)
Shortcut
Registered: Dec 21, 2002
Posts: 2935
|
In reply to: In reply to: The whole site is political. Move away from a statistical approach. It is art. Photography is an art form. And art is a subjective media. In reply to: Appoint a board of editors. Have them decide. Alow people trained in the field with more expertise to make decisions based on a consensus view of a panel of 5 to 12 (or some such number). Why have someone else tell me what I like? My subjective opinion is as valid as anyone else's, I submit.
Agreed.
But if your subjective opinion is "offered" at "strategic" times druing the process it has more or less weight.
I say still have a rating porcess, but why have it effect what is on the cover? Covers are editorial decisions. It already is an editorial decision in that someone decided where the current cut-off point was.....
So my point is to refine the present system. Voting is a popularity contest. If that is what the FP cover shot is, then I will be forced into the obsequious acquiescence of that process.
However, I prefer to move away from the "People's Choice Awards", White-Trash mentality of pop-culture, political popularity contests and allow it to be a purely political process amoung a group of editors in much the same way it works in other media outlets.
Thus, the subjective art form would be subjected to the subjective whims of a few select subjects.
|
|
|
|
|
biff
Feb 27, 2004, 9:07 PM
Post #40 of 78
(7259 views)
Shortcut
Registered: Nov 5, 2001
Posts: 851
|
Quick reply on editors:
We have 10 photo editors, they approve about 50 photos a day maybe more.
They work very hard, I only want to make their job easier, having many more editors would be a pain to manage for the administrators.
I think the current system works well enough that we don't need to change it, if that change means making more work for the editors.
The idea of approving for Front Page has beed discussed before, but we have found that the current system works very well, and doesn't require editor intervention.
To discuss what goes on the front page (that is the rockclimbing.com home page, not the first page of top photo results) continue one of the 7 or 8 threads about that. Lets try to keep to the topic about preventing photos from being bombed .. thanks.
|
|
|
|
|
climbsomething
Feb 27, 2004, 9:13 PM
Post #41 of 78
(7259 views)
Shortcut
Registered: May 30, 2002
Posts: 8588
|
In reply to: Appoint a board of editors. Have them decide. Alow people trained in the field with more expertise to make decisions based on a consensus view of a panel of 5 to 12 (or some such number).
And putting it to actual knowledge of photography is just going to piss people off too. I have no problem giving a poorly exposed snapshot a 5 and saying so (poorly framed, lame perspective, soft focus, bad lighting, whatever) but plenty of people come back with "waaaa, what about how it makes me feeeeel, it's not all about technical aaaaaspects." Try to cut the fluffy "but it makes me say oooooooooh" photos and that's going to tank.
Grumble.
|
|
|
|
|
climbsomething
Feb 27, 2004, 9:27 PM
Post #42 of 78
(7259 views)
Shortcut
Registered: May 30, 2002
Posts: 8588
|
In reply to: Lets try to keep to the topic about preventing photos from being bombed .. thanks.
If we want to cut out bombing and/or homie votes, then we need to cut votes period. I think that is the only guaranteed way to keep shady votes from happening.
Kinda like when your mommy takes away your new toy because you can't play nicely with it, and just poke your brother's eye out with it instead.
|
|
|
|
|
hardmanknott
Feb 27, 2004, 10:10 PM
Post #43 of 78
(7259 views)
Shortcut
Registered: Jan 15, 2003
Posts: 228
|
Here's another thought. Are Photo Editors allowed to vote?
If so, are they above letting their personal feelings--good or bad--influence
the vote they give when approving a photo? The reason I ask is that
I have ruffled a feather or two--and I think my photos might suffer for it.
As an example, please explain how/why this photo appeared as a 6.
A professional photographer friend of mine really liked it, and he tends
to be very critical of my "work":
http://www.oceandave.com/Blurboy.jpg
Anyway, I was so disgusted by this apparant conspiracy that
I immediately deleted it from the site in true drama queen fashion.
So I guess a better question might be: How do I achieve hero
status--where I have a posse of sycophants voting my photos into
the stratosphere?
{both edits for semi-literate grammar and spelling}
Hardman Knott
|
|
|
|
|
jt512
Feb 27, 2004, 10:12 PM
Post #44 of 78
(7259 views)
Shortcut
Registered: Apr 12, 2001
Posts: 21904
|
In reply to: Wott a kludge! I'd been hoping you'd propose a more elegant solution, Tukey's biweight or a Huber function, mebbe just a trimmed mean, or even -- KISS principle -- a median. There are standards, criteria, a whole literature about such things after all. Not that Rockclimbing.com needs them, but what the hell, it's Friday. 8)
When the issue first came up, I suggested just using the median, but someone (I thought, you, actually) shot down the idea. At that time, I looked at several trimmed means, but none of them seemed well-suited for our purposes although I don't recall now exactly why I thought that, but off the top of my head, they give 0 weight to some of the data, and they wouldn't work well for small samples. I like the idea of using the z-score as a measure of how unusual a vote is, because, well, that's what it is. And, finally, I think weighting the votes based on an estimate of their probability of being honest is about as elegant as it gets.
-Jay
|
|
|
|
|
sauron
Feb 27, 2004, 11:22 PM
Post #45 of 78
(7259 views)
Shortcut
Registered: Oct 15, 2001
Posts: 1859
|
Remove photo votes, which also takes care of removing all the problems associated with photo votes...
(See earlier discussion about user Q votes, et al)
Oh, and if you want drama about voting on images, visit www.photo.net...
- d.
|
|
|
|
|
dsafanda
Feb 27, 2004, 11:47 PM
Post #46 of 78
(7259 views)
Shortcut
Registered: Mar 6, 2002
Posts: 1025
|
That's a great shot Dave. I would have given it a 9. I wouldn't put too much stock in the opinion of the masses. There's no accounting for taste. Everyone thinks they have good taste but if that was actually the case, how do you explain the existence of items such as Elvis paintings on black velvet or tie dye t-shirts? As long as the voting is in the hands of the public at large you're going to have a chaotic mix of opinions that ultimately trends to the lowest common denominator. There's nothing wrong with that. I would prefer that RC.com avoid trying to engineer the photo rating system. You would simply be battling human nature.
If RC.com ever decided that is was important to have professional quality photos on the home page then some sort of editorial staff would make the most sense. However, as long as the photo are simply a way for users to share and enjoy the photos of other RC.com users I think the system should be kept as simple and untouched as possible.
Now I'm going to get 1's from all the people who like tie dyes. :)
|
|
|
|
|
jt512
Feb 27, 2004, 11:52 PM
Post #47 of 78
(7259 views)
Shortcut
Registered: Apr 12, 2001
Posts: 21904
|
In reply to: As an example, please explain how/why this photo appeared as a 6. A professional photographer friend of mine really liked it, and he tends to be very critical of my "work":
That's a great shot, one of the best bouldering shots I've seen on the site. A "6"? That's ridiculously low.
|
|
|
|
|
vertical-rockrat
Deleted
Feb 28, 2004, 12:00 AM
Post #48 of 78
(7259 views)
Shortcut
Registered:
Posts:
|
Just a thot maybe?! how bout like in the olympics Throw out the two highest and lowest and average what is left. Set the alogarythim to set all pics from start as a 8 then after the 5th vote the alog can throw out the highest and lowest 2 each and use the middle one ... keep this up so its always the 2 highest and lowest ones thrown out and then averaged .. mabye set it so later on say after 20 votes it throws out 3 each, not sure if that is possible but if you set your system up some how and do not post the alog for those to see they will not know how many votes are being thrown out and how many used... This is only a brain storming idea just thot id throw it out to you.
|
|
|
|
|
orangekyak
Feb 28, 2004, 12:07 AM
Post #49 of 78
(7259 views)
Shortcut
Registered: Jun 30, 2002
Posts: 1832
|
INCREASE THE LAG!
One week lag from when a photo is posted. No votes show until lag is over. Instead, the message "Vote on this photo now. Its rating will appear in X days."
And for the record, I still think comments should be required with all votes, even if the comments are radio buttons that tally, like "great action," "cool shot," "good shot with average composition" ...
|
|
|
|
|
melekzek
Feb 28, 2004, 1:34 AM
Post #50 of 78
(7259 views)
Shortcut
Registered: Nov 16, 2002
Posts: 1456
|
I think any algorithm looking only to the votes given on an image is bound to fail. For a more accurate voting schema, it looks like we need to track peoples individual voting behaviours. Something like a credibility score.
Everybody starts with a medium level credibility. You can keep track of peoples votes, and compare the votes to the average rating of the image. Depending of the difference, you can increase or decrease the credibility of the voter. The average of the image is calculated by weighting individual votes with the voters credibility score. Anybody constantly hate voting will receive a lower credibility, decreasing his votes effects. For a succesful hatevote, a user should vote correctly for some time, raising his credibility, and than hatevote, which will decrease his credibility again.
:twisted:
it might be too much/unnecessary work though.
I guess I can live with current voting scheme :roll:
|
|
|
|
|
|
|
|
|
