Is this bad poll design?



  • In [url=http://perlmonks.org/?node_id=645459]this Perlmonks thread[/url], someone brought up a problem with their polling system which I think is a WTF.  In short: Votes are stored by IP, so it's possible to vote twice even if you're a registered user, as long as you switch computers.  It's also possible for a legitimate vote to get denied if it comes from the same IP as another user.  No one there seems to think it's a problem, or they think it's too hard to fix.  Am I crazy or is this a really amateurish design flaw?



  • I guess it depends on whether or not you have to log in to vote.  If you don't have to log in, it's flawed anyway, because people with dynamic IPs can vote as much as they want.  I suppose even with accounts people can create new accounts to vote, so the whole idea is probably flawed from the start.



  • The idea is not flawed at all. It just depends on the honesty of the voters. That might be perfectly acceptable as long as it's not some kind of important election. ;)



  • @wittgenstein said:

    The idea is not flawed at all. It just depends on the honesty of the voters. That might be perfectly acceptable as long as it's not some kind of important election. ;)

    And in fact, it's all in jest anyway... nobody takes the "poll" seriously.  It's just a place to look at various-length bars, and then comment on what you think the choices should have been. :)



  • The idea that you can get a statistically valid sample through a voluntary poll on the Internet is itself a WTF. (Go look up "selection bias" and "response bias" in a statistics textbook.) Attempting to correct such a poll by changing the vote-recording mechanism gets twice as many WTF points on the "waste of time" heading, but only half as many on the "importance" heading.



  • I designed a poll system for a site and ran into the same problem. Some of the polls are done in jest, but most others will be used to make a decision.  My solution was to allow the moderator to create a poll and specify the voting system:

    1) Anybody can vote (one vote per IP)

    2) Only registered users can vote (doesn't log IPs, but logs their UserID)

    Choosing option 1 does indeed block the second person in a household from getting a vote, even if his registered account proves that he is a different person (I know there are brothers who use the site). Since system 1 is generally reserved for polls done "in jest", nobody is really that concerned about it.

     

    If you really wanted to be thorough, you could log their vote along with their IP and UserID (if logged in).  If they try to vote again and their IP already voted but they have a different UserID, then the vote can be allowed.  I could have done this, but it didn't seem worth the effort.  I'm not surprised that the administrators of your website don't care, either.



  • @Pap said:

    If you really wanted to be thorough, you could log their vote along with their IP and UserID (if logged in). If they try to vote again and their IP already voted but they have a different UserID, then the vote can be allowed. I could have done this, but it didn't seem worth the effort. I'm not surprised that the administrators of your website don't care, either.

    And there's nothing to stop them from creating a new user.

    When you get down to basics, it doesn't matter how badly you want to solve this problem - it cannot be solved. There is simply no way to design a non-trivial system that relies on some concept of "unique identity" that doesn't permit cheating. Any possible mechanism that you can design can be defeated. We cannot prevent election fraud. 

    The solution is, of course, to find a more sensible way to make a decision. The majority opinion is likely to be wrong more often than it's right, anyway.
     



  • Of course it will never be completely cheater proof, but I think it's a pretty bad issue when people can cheat [i]by accident[/i].  And it's even worse that if there's a bunch of registered users behind a NAT, only one of them can vote.



  • There's a gaming website that runs a monthly poll to decide which game is the best and is the most active too. It enforces this very kind of vote: one daily vote per one IP, "cheaters will be banned and their votes removed". There are multiple ads served on each page and you have to go through three pages to know whether your vote passed.

    Now, imagine (as it regularly happens) two games are very close on. What's stopping you from disconnecting from the net, reconnecting to the net and getting another vote? Dynamic IPs for the win!

    So, it regularly happens that the contest is won by a community of maybe 100 abitual gamers that rack up 100'000 votes a month.

    As you can see, this is a perfectly valid approach, as it looks legit, allows for very easy cheating and gives the webmasters ca. 400'000 pages' worth of ads from the voting activity alone.

    To recap:

    • The occasional visitor is happy:

    1. votes counts are reset every month
    2. winners can't vote for 12 months, so the second coming game gets another try
    3. therefore, the votes look legit and he might even try the "game of the month".
    • The hardcore gamer is happy:
    1. you can cheat very well -- hell, it's more a cheating contest than anything else.
    2. you get free pubblicity for your game if you cheat enough -- this means a larger playerbase (and, most of the times, more ads served).
    3. cheating is "fun" as most of these online games ban cheaters.
    • The advertisers are happy:

    1. ads are shown a LOT of time (to voting bots, that is).
    2. the site looks legit so the ads look cool as well.
    3. ads aren't very expensive (or so it is stated on the conveniently-located Advertising page), so it's even more appealing.
    • The webmasters are happy:
    1. loads of hits.
    2. loads of ads.
    3. loads of revenue.

    Of course, the results are immensely f*cked over, but nobody cares...



  • what comes to my mind is that it shouldn't be impossible to use both mechanisms. record both IP and authenticated user for the vote, and allow only one anonymous vote per IP, and only one authenticated vote regardless of the IP. still not perfect, as a registered user can also logout and vote once anonymously, and it still doesn't allow multiple NAT-ed anonymous votes, but still better than IP-only or user-only voting.


Log in to reply