21 Replies Latest reply on Jan 29, 2019 2:59 PM by lui_gough

    RoadTest Rule Update: Submitting Identical RoadTest Applications Under Different Names

    rscasny

      I recently received two applications for a roadtest where the answers to the questions were identical for each application, but were submitted under different names. I'm not sure what the motivation is for doing this, or whether the applicant has two accounts, or any other reasons that I'm not aware of. My new rule is when I see this, both applicants will not be recommended as an official roadtester. If you would like to comment on this new rule, please enter your comments below. Thanks.

       

      Randall Scasny

      RoadTest Program Manager

        • Re: RoadTest Rule Update: Submitting Identical RoadTest Applications Under Different Names
          colporteur

          Mans ingenuity is inexhaustible when it comes to finding ways to gain an advantage.

           

          It could be argued since it wasn't in the rules no foul. Ethics and integrity suggest just because it doesn't say it, doesn't make it right.

           

          Good catch on your part to recognize the duplication. Bad on the part of the conspirator to think it was acceptable to improve their odds of selection by duplicating the application process. Doubly bad on their part for not even modifying the two applications. I suspect he didn't do well in cheating on exams. The dupe and his compatriots all provided the same wrong answers on the test and wondered how they got caught.

           

          I speculate, if the candidate was selected you would not have gotten a RoadTest Review. Yes there are some nice products in the RoadTests I would like to have in my shop but duplicity to achieve such a goal is short sighted.

           

          I don't believe your punitive response is adequate. I feel a year suspension from participating in RoadTests is more appropriate. If found in a subsequent offence then exclusion from participation. Unfortunately, it increases the your logistics of policing the activity.

           

          Sean

          3 of 3 people found this helpful
          • Re: RoadTest Rule Update: Submitting Identical RoadTest Applications Under Different Names
            jomoenginer

            I suppose if it were not posted that one could not send in multiple entries under different names, then it is tough to fault them. But, then again, it really depends on their motivation.  One option would be to contact the individual, or individuals, and ask them which entry they prefer since one would be dropped.   However, there is an X-Files chance that the entries are really from two different people yet sharing the same brain.

            • Re: RoadTest Rule Update: Submitting Identical RoadTest Applications Under Different Names
              peteroakes

              Of course now they will just tweak then a bit to make them different,

               

              I know this is more work, but perhaps if everyone who wants to register for a road test must have their full address in their BIO, I know I do

               

              Then even with different accounts and submissions, perhaps the address will help sort out the "Cheaters", it is less lightly someone will have a different address for each account, this could be required at registration time for a road test, not after being selected.

               

              I know it is still not perfect but it may help.

               

              This certainly irks me when I hear stuff like this is going on within a supposedly great community (Which it is), the few spoiling it for the many

              5 of 5 people found this helpful
              • Re: RoadTest Rule Update: Submitting Identical RoadTest Applications Under Different Names
                Fred27

                I'm guessing they weren't particularly good applications anyway. You've mentioned in the past that some people just put the bare minimum like they're just entering a sweepstake for free stuff. This sounds like someone thinking they're doubling their chances of a "win".

                2 of 2 people found this helpful
                  • Re: RoadTest Rule Update: Submitting Identical RoadTest Applications Under Different Names
                    kas.lewis

                    I think you hit this one square on, They are looking to double their chance to WIN. A number of us here have for a while felt the wording for the roadtests does not properly reflect what they are. By saying get a product for FREE people feel it's a lottery or a give away or just fill in your info and get a free product (like the AVNET email about the IoT product).  

                     

                    This is not the case though. They are anything but free. Do I get a product, YES, do I have to pay money, NO, do I have to spend valuable time working with and reviewing this product YES. So no this is not FREE I pay for it with my time and my (hopefully) quality review.

                     

                    By saying its FREE or saying its a GIVE AWAY people will treat it as such. Fill in my credentials and expect that FREE product to arrive in the main with no obligations or strings attached. All you need from me is a name, email address and mailing address and I will get a FREE product.

                     

                    Kas

                    4 of 4 people found this helpful
                  • Re: RoadTest Rule Update: Submitting Identical RoadTest Applications Under Different Names
                    DAB

                    Henry Ford said it best "You cannot make anything fool proof because fool's are too ingenious!"

                     

                    I support your position, if someone is going to that much trouble to game the system, then they are probably in the grab and run category.

                     

                    I do not feel it is element14's interest to take a risk on this type of person.

                     

                    DAB

                    1 of 1 people found this helpful
                    • Re: RoadTest Rule Update: Submitting Identical RoadTest Applications Under Different Names
                      jomoenginer

                      What if it were really two separate persons and one just copied what the other wrote since the other was more versed in the writing arena? 

                      • Re: RoadTest Rule Update: Submitting Identical RoadTest Applications Under Different Names
                        Gordon Margulieux

                        To Whom It May Concern,

                         

                        Without additional information it is hard to judge one's motive in this situation.  So, for now I would just chalk it up to someone's honest mistake.  It would be interesting to check with the individual or individuals for what they were thinking.  It would be sad to work so hard on a proposal and have someone else steal their idea, then have both entries disqualified.  On a positive note, by adding the restriction of only one entry per E14 member to the official RoadTest rules makes it clear and improves the process.

                         

                        Gordon

                        • Re: RoadTest Rule Update: Submitting Identical RoadTest Applications Under Different Names
                          lui_gough

                          I find it rather difficult to believe two applicants would come up with verbatim applications independently, so I think if you do find duplication like this, it is not unintentional and likely represents a "bad actor" of sorts. I wouldn't be surprised if those who don't know anything about the program perhaps try to take it to the extreme as with other lottery competitions - automating account sign-ups and spamming duplicate entries by the hundreds of thousands.

                           

                          I think that any duplicate entries should be disqualified, particularly if it is found that the accounts are recent sign-ups or perhaps the two accounts have been logged in via the same IP or have been signed up on the same/nearby days. Regardless, it is highly suspicious behaviour and I think the RoadTest terms and conditions should cover this in:

                          1.4 Multiple applications are not permitted.

                          1.5 Applications may not be submitted by an agent whether acting on behalf of an undisclosed principal or otherwise.

                          2.3 All applications submitted to this RoadTest must meet the following criteria:

                          2.3.1 Applicants must be the author, creator and owner of the proposed review idea. Applicants must not submit someone else’s idea.

                          I suspect maybe that's not clear enough that it refers to the individual rather than an account - but as someone already mentioned - addresses, phone numbers, e-mails are ways of weeding duplicates. But already, given the above conditions, there really doesn't seem to be an excuse for a verbatim-duplicate application. If one had been improved in some way - perhaps this is a sign that someone didn't know how to edit their application post-submission. But there shouldn't be any way for unprivileged members of the site to view other people's applications (short of a few bugs in the early days), so foul play in the way of copying other people's applications can be excluded.

                           

                          While I think excluding them from recommendation is a relatively light punishment, I would be in favour of excluding them from selection for that RoadTest at a minimum. Perhaps even excluding them from the RoadTest program as a whole would not be too unfair, as it seems very likely that the applicant wanted to have some level of unfair advantage and probably didn't even read or understand the terms and conditions of the RoadTest they were applying for. I suspect such candidates are unlikely to be quality applicants anyway - if they were, perhaps you could reach-out to them for an explanation, but I probably wouldn't go so far.

                           

                          - Gough

                          2 of 3 people found this helpful
                            • Re: RoadTest Rule Update: Submitting Identical RoadTest Applications Under Different Names
                              jomoenginer

                              Some folks collaborate on projects and even with RoadTests so it might be possible where one person comes up with the idea and another publishes the proposal.

                               

                              Besides, how is it possible to prove any of this?

                               

                              Making the submitter process more complicated may discourage more than just the RoadTest abusers.

                                • Re: RoadTest Rule Update: Submitting Identical RoadTest Applications Under Different Names
                                  lui_gough

                                  If so, you still wouldn't get verbatim applications. Also note terms where applicants must not submit someone else's idea nor submit on behalf of another. That is clear and already existing in the terms.

                                   

                                  If you collaborate, submit one proposal from one person then do the project together. Perhaps name collaborators in proposal rather than submitting identical proposals under different names.

                                   

                                  Proof for some things could be found in the metadata of the submission that site admins probably have some level of access to. IP addresses, login times, cookie tracking, user agents, screen resolution, OS, languages installed, etc. There are many ways to fingerprint a browser that makes random coincidental results fairly unlikely - e.g. demonstrated by Panopticlick. Just testing my browser today showed:

                                   

                                  Making it more complicated may discourage people, but this is not about making the application process more complicated or difficult. The terms I listed are already the currently operating terms of RoadTests. The new rule is more like a warning to potential repeat submitters, but mainly only affect how RoadTests are vetted - that affects the staff. If people fail to read and understand this, that's not exactly the fault of element14, nor should we condone mindless enrollment. The issue is ensuring quality, legitimate RoadTest proposal submissions and some level of fairness, transparency in selection, in part to avoid needless waste of time on the submitter and judger's behalf.

                                   

                                  As it stands, a RoadTest review is a commitment at least 20-times greater than that of a proposal alone. If a potential reviewer isn't willing to pony up a decent proposal, it seems unlikely they would be able to put in even greater effort to deliver a quality RoadTest.

                                   

                                  - Gough

                                  2 of 3 people found this helpful
                                    • Re: RoadTest Rule Update: Submitting Identical RoadTest Applications Under Different Names
                                      kas.lewis

                                      Interesting how this has gone from a discussion about a something we don't have a clear understanding of to being decided and definitely determined. Sounds like in future the FBI, Scotland Yard, CIA, NSA MI5 among others should be involved in the selection process to be sure al rules, regulations and correct intentions are followed.

                                       

                                      OOPS let's not forget a full complement of lawyers to be sure any interpretation of the rules fits with the best "intentions" of the road test community.

                                       

                                      Kas

                                        • Re: RoadTest Rule Update: Submitting Identical RoadTest Applications Under Different Names
                                          lui_gough

                                          Well, as with everything, it's both clear and unclear at the same time depending on how liberal you want to be in your interpretations. I'm only providing my views as I see it from the T&Cs as published on prior RoadTests.

                                           

                                          I mean, the outcome of this discussion is likely going to be another rule that will probably go into the T&Cs, so if the problem is about the interpretation of the T&Cs, then basically we're back to square one ... It's always a case of definitions?

                                           

                                          That being said, I do agree with the fact that adding more stuff to the T&Cs isn't going to change much on the premise that so many people don't read them anyway. People still probably submit poor applications despite all the "how to apply" tips provided. I guess this was more about warning anyone else thinking about doing the same?

                                           

                                          - Gough

                                          2 of 2 people found this helpful
                                        • Re: RoadTest Rule Update: Submitting Identical RoadTest Applications Under Different Names
                                          jomoenginer

                                          One can create an environment that will prevent from being detected so there is not guarantee with that.  However, thanks for showing the attitude that has led me to never participate in another RoadTest on this site again.

                                            • Re: RoadTest Rule Update: Submitting Identical RoadTest Applications Under Different Names
                                              lui_gough

                                              Of course, there will always be ways to evade it, but that doesn't mean we should just turn a blind eye to it either, especially if there isn't a legitimately good, fair and just reason for it happening.

                                               

                                              Each to their own, but you don't see me participating in design challenges here either for a particularly good reason as well ...

                                               

                                              - Gough

                                                • Re: RoadTest Rule Update: Submitting Identical RoadTest Applications Under Different Names
                                                  kas.lewis

                                                  lui_gough

                                                  Each to their own, but you don't see me participating in design challenges here either for a particularly good reason as well ...

                                                  Nope you just apply to the high end gear RoadTests, I guess it's bang for your buck.

                                                   

                                                  It's interesting how you complain bitterly about multiple entrants (seems poorly written ones) but the thought of sharing opportunities for good gear does not enter the equation. Your complaining sounds like more of its not fair that you might get a slight disadvantage from someone else than real concern about fair players in the RoadTest Environment. I guess each to his moral self.

                                                    • Re: RoadTest Rule Update: Submitting Identical RoadTest Applications Under Different Names
                                                      lui_gough

                                                      It all boils down to what I'm interested in - I apply for RoadTests I'm interested in and I put in a commensurate effort in my applications which aligns with the value of the item. If the conditions are not something I feel I can meet, I'm not going to bother applying. If people aren't selective about their applications or how they apply their effort - this is not my fault. The fact I do end up with high-end gear is not relevant at all - nor is the thread about sharing gear, so I have no idea why you're mentioning it. I've never been against sharing gear in any way (where did I say I was?), but being in Australia doesn't provide many opportunities as I'm not going to be spending large amounts and risking gear to just ship it over to someone for an experiment.

                                                       

                                                      If you feel this is complaining bitterly - it's not. I merely spent the time to basically explain that this doesn't seem like a new rule at all - it is already covered by existing rules so I think what is being done is kosher, but perhaps insufficient. I also acknowledged that "rules are rules" and people will be free to ignore and not read them. But I also pointed out there are ways to better detect/weed out such duplicate submissions and identify bad actors. The fact this is made explicit by Randall is probably to try and deter such things happening in the future

                                                       

                                                      Between all of this, somehow you feel that this is somehow affecting fair players? That is what I find a bit curious. A fair player isn't likely one that has two accounts and would accidentally post a verbatim submission. Likewise the chances of two unrelated people posting verbatim submissions is very improbable. How am I hurting fair players by mentioning that the existing policy already covers this? I'm basically saying that this doesn't change anything because the existing rules already cover it ... but perhaps you don't agree with my suggested list of punishments - that's fine ... but I don't think advocating harsher penalties for something fair players wouldn't likely do is something that would hurt fair players at all - merely improve their collective chances.

                                                       

                                                      Logically - if you penalise the duplicate submissions - reduces pool number to fair non-duplicate submissions -> increases chances for fair players. Of course, i have an interest in this because I don't submit my applications in duplicate and spend some time to research and write them. But also I don't want to see Randall have to weed through them manually - it's not easy to spot if there are lots of applications and if only some duplicates are spotted and others are missed, this could be unfair. That was why I mentioned potentially better ways to use metadata to help flag such possibilities. I don't see how this is not concern for the fair players - if I truly had no concern, I would advocate leaving the duplicates in or recommending them because they were somehow "more interested".

                                                       

                                                      In fact, it sounds like you're implying that somehow I'm using a strategy which is "unfair" and that I would do "anything" to win more RoadTests. But no, the whole point of posting was because I really couldn't understand why Randall even needed to mention it at all, as I always thought it to be the rule.

                                                       

                                                      - Gough

                                          • Re: RoadTest Rule Update: Submitting Identical RoadTest Applications Under Different Names
                                            cghaba

                                            Some sweepstakes or free giveaways are encouraging multiple application from the same person as each access to that page increases some page access number, resulting in more numbers of advertising displays. It is not the case for applying and winning a RoadTest review. This must be clear for all participants and, this large discussion will make easier to find the right answer for those who will search on element14 site about "Submitting Identical RoadTest Applications" AND/OR "Under Different Names" 

                                            • Re: RoadTest Rule Update: Submitting Identical RoadTest Applications Under Different Names
                                              colporteur

                                              What salary does a RoadTest biblical Solomon garner today?

                                               

                                              I chuckle at the self awareness or lack of each post provides. You are either the sheep or lemmings herder of the technology myopic and heads in the sky. I do not envy your role but, do wish you the best of luck

                                               

                                              Please continue to share your experiences as the RoadTest overseer. These opportunities provide us mere mortals, insight into the program. When I first joined the site  You provided the chance to explore and provide feedback on the RoadTest application process. This exercise enabled me to recognized the potential of the RoadTest venture for Element14 and why it must be well managed.

                                               

                                              Keep up the good work! Your focus and investment will continue to improved the program. Please accept this post as it's intended. A shameless, overt plug to gain favour in the eyes of the RoadTest judge, so I get a larger half of the next FREE child.

                                               

                                               

                                              Sean

                                              • Re: RoadTest Rule Update: Submitting Identical RoadTest Applications Under Different Names
                                                rscasny

                                                To all commenters,

                                                 

                                                I appreciate everyone's input. It's made me think about our RoadTest Rules. Here's an update.

                                                 

                                                Yesterday, we researched the two accounts (our developer team and community manager) that had identical applications. They may be coming from the same person, or two different people working together. We aren't 100% sure. If you are going to work with a team, I would recommend submitting one application with both names so we know, but this team effort is primarily for school teams.

                                                 

                                                Every applicant needs to submit his own application. No one can submit an application for someone else. I recall recommending a few times a guy who does not write great. But he did his best and his reviews were informative.

                                                 

                                                Beyond that, I found some old correspondence from this person's email; he said he didn't receive a roadtest product that he was sent. We checked the tracking number and it was already delivered. He was unaware of it. It seemed odd to us at the time although I guess this situation is possible, to give him the benefit of the doubt.

                                                 

                                                Whether these two applications are from one person or two doesn't matter since one of the accounts has at least one outstanding roadtest review that I am waiting on from him to submit. I've been told by another one of our program managers that he was sent product for another project that he has yet to complete.

                                                 

                                                When I see someone is completing their reviews by the deadline, the person's credibility rises in my eyes.

                                                 

                                                Randall Scasny

                                                RoadTest Program Manager

                                                5 of 5 people found this helpful