I have been trying for some time to ascertain the statistics showing the accuracy of the MARS test. At various times the MARS Company has published accuracy rates ranging from 84% on the low side to 95% on the high side. Currently, they are stating that the MARS test has a 90%. I am not sure why they reduced the published accuracy.
(Click here for reference) (http://www.wisdompanel.com/faqs/default.aspx?show=true&acc=true#question8 - broken link) (
Click here for a report of the 84% accuracy number) (http://www.symbioscience.com/news_item9.asp - broken link)
During the
K9 College Cruise, I had the pleasure of taking a course with
Dr. Danika L. Bannasch, DVM, Ph.D. (Click on her name for more). She is a geneticist at University of California Davis. Her work centers on dogs and horses. I asked her about the accuracy of the MARS test. She stated, “It is completely bogus.” I was unable to get her to define, “bogus” in more statistically relevant terms, but she was insistent that the tests did not provide a level of accuracy.
So, I continued to search for information regarding accuracy. I found several stories,
such as this one (http://www.symbioscience.com/news_item9.asp - broken link), which relates a story of a single dog which when tested via the MARS program resulted in different answers for each test. Well, one story is interesting, but it is hardly a statistically valid statement. So, I went further.
And yet I was unable to find one single study, conducted by someone who had no interest in the MARS Veterinary Company, who tested the accuracy of the MARS test. Simply, all the data available has been published by the MARS Company. This does not mean that the MARS Company lied. However, it does make validation of their claims of accuracy difficult. When I was attending
SuperZoo last year, I saw that MARS Veterinary had a booth. I approached the booth and engaged the MARS representative in a discussion about a specific breed. He stated, “You know Dr. X? We’ve been working with Dr. X for sometime.” Since I know that X's education is limited to an AA degree in computers, or some such subject, I was surprised that he referred to her as “Doctor.” I immediately became suspicious of anything the representative would say
Never-the-less, I attempted to discuss with the representative the statistical foundation for the company’s claims about the Wisdom Panel. I asked him about Type I errors, Type II errors, and the frequency of false positives and false negatives. He could not answer even one of my questions. Frustrated, I advised him to take a statistics class and then I left the area where the MARS Vet people had set up their booth
After all of this I came to the conclusion that with the scant information that I have to go on, I had to assume that the MARS Wisdom Panel test was, as the MARS company states, 90% accurate. That means it would have 5% false positives and 5% false negatives. It is highly unlikely that any scientific process would have the same percentage of false positives as false negatives, but to continue my analysis, I felt that I had no choice but to make that assumption
OK…so all of that brings us to this point: If the MARS test says that a dog is a specific breed, how inaccurate is that? If someone believes a dog is a specific breed, has the pedigree, and sends it in, what is the probability that there will be a false positive or a false negative? In other words, what is the probability that a perfectly healthy dog will be culled from someone’s breeding program in error?
So, just to keep things interesting (for me, perhaps not for you), I included a test question on one of the exams that I give my introductory statistics students at the university. They correctly concluded that out of 1,000 dogs of a specific breed thusly tested: 50 would be false positives and 50 would be false negatives. Since most breeders, in my opinion, are most interested in the accuracy from the standpoint of removing dogs from the breeding pool, the false negatives are more important than the false positives.
In other words, the question was: “Given the information above, what is the probability that a dog identified as NOT being a specific breed actually being that breed?”
We know that out of the 1,000 dogs tested, 900 would be correctly identified. We also know of those not correctly identified, 5% would be false negatives—in other words, the test results would indicate that they are not that specific breed when, in fact, they are. So what is the probability that if a dog of a specific breed is tested, and the results show the dog is not that breed, that the test is wrong? The answer is not merely the 5% accuracy number. The correct answer is relies on the study of probabilities, and is not as intuitive as one would think at first.
The correct answer is if a dog is tested, and is identified by the test as being something other than the breed shown on the pedigree, there is a 10% chance that the test is wrong. That is, if we believe the statistics from the MARS corporation.
Of course, if you use the statistics from MARS original studies, you end up with a 16% error rate.
Whatever the real number is, this much I know: If a dog is tested and found to be not-a-biewer, then there is between a 10% and 16% chance that the test has incorrectly identified the dog’s genetics.
Comments? Did I make a mistake here?
Please register to participate in our discussions with 2 million other members - it's free and quick! Some forums can only be seen by registered members. After you create your account, you'll be able to customize options and access all our 15,000 new posts/day with fewer ads.
I think I comprehended enough of your OP to understand the accuracy of the canine DNA tests are not that great... at least the test is certainly not worth my hard-earned money - not for what could end up being a less-than conclusive result. 
