Lies and Unbiased Product Testing
September 22, 2008
The third party product testing space performs an important mission within the industry – they keep the vendors honest. That mission is why I was so fired up when I took a job at Neohapsis in 2007 running their product testing lab. And, while I left Neohapsis a few months ago, I’m still fired up about product testing and the important role that a truly independent third party can bring to this industry.
The flip-side of that emotional intensity is that I get rather upset whenever I hear rumors about someone abusing their stance as an unbiased third party. And I recently heard something that made me ill.
I had a source approach me about a product testing firm who he suggested I blog about. This “independent testing firm” apparently does the most blatantly unethical thing I’ve ever heard in that industry:
They write the results that their testing will discover IN THE CONTRACT with the vendor who is requesting the testing.
It’s one thing to have a “wink-wink, nudge-nudge” sort of relationship with a vendor. It’s another to tilt the test criteria slightly, or even to accept the vendor’s claims as solid assumptions during your testing. Both of those annoy me, but they’re the kinds of things that go on in most organizations (though they did not happen under my watch at Neo… I’m too much of an idealist and a pain in the ass).
But when you have the outcome written in to the contract? In my mind, that’s fraud if you’re going around pretending to be an unbiased third party.
I couldn’t believe that anyone would truly stoop to this level. And, since my source had left the company where he had worked with the testing lab, he couldn’t get me a copy of the contract.
So, before I went on-record and wrote this one out, I wanted a bit more proof. I asked my amazing team of Indian VAs to check it out for me and provide me confirmation – not the information, just confirmation. Just get one of the contracts and tell me if the results are in there. Search, investigate, and interview customers of the lab to attempt to prove it one way or another.
The response to that investigation?
A cease and desist letter to my VAs from the test lab. (Which is why I’m not posting the lab’s name… if I’m too busy to blog, I’m way too busy to bother with lawyers).
So, I’ll say it this way – if you’re reading a report about a product from a third-party lab that claims to be unbiased, take it with a grain of salt. Especially if the report came to you from the product vendor’s sales/marketing team.
Or you can just ask to see the vendor’s contract with the testing lab, and see if you get handed a cease and desist letter.
![[del.icio.us]](http://images.del.icio.us/static/img/delicious.small.gif){kind=small}
I’m not an economist, but…
September 19, 2008
I just read the info on the new US mortgage bailout.
I’m bothered.
I can’t figure out how this works. I mean, I get the idea – the federal government purchases (and later attempts to sell) “hundreds of billions of dollars” of bad paper.
But, if the paper is no good, it means there’s no resale value.
So, that hundreds of billions of dollars gets piled on top of the federal debt.
It seems to me that, in the medium term, that extra debt exerts further downward pressure on the US dollar against other international currencies. (What’s interesting is that, up to now, most of the US borrowing has been for international and discretionary purposes like war and trade – now we’re borrowing large-scale for domestic purposes. It seems to me that it’s like the difference between borrowing on a credit card to eat at restaurants and borrowing to buy groceries…)
As the dollar declines further, the US has a harder and harder time remaining solvent and inflation increases. At that point, more bad paper will emerge (i.e. more mortgage defaults as gas hits $10/gal and a loaf of bread costs $5), making this all get a whole lot worse.
This doesn’t seem to be the right way out.
USA Today made a fantastic point today – the USA is not following its own counsel. From the article:
Throughout more than a decade of recurrent crises in nations such as Mexico, Russia and Thailand, the United States offered the same advice: Let the market solve the problem and get the government out of the way……
…. In the 1990s, officials of the U.S. Treasury and the U.S.-backed International Monetary Fund urged the leaders of crisis-hit countries to embrace market-oriented policies designed to put their economies on sounder, long-term footing. But the recommendations — to slash government spending and privatize bloated state companies — meant genuine pain for millions and thus real political costs for leaders.
It seems to me that we’re taking massive short term action to avoid the long term consequences of our actions. It’s like someone who is writing bad checks: you write one, then you write another to cover that one (plus a little more), then another, and another, until, eventually, you can’t write a $1M check to cover everything you’ve done.
Unfortunately, as anyone who has piled lie on top of lie to avoid getting caught knows, if you come clean and pay the piper early, the pain isn’t so bad. It’s only by putting it off over and over again that we create a situation that ends up as a disaster.
It seems to me that this is just another way of putting off the inevitable. Anyone who has read the story of Japan’s collapse in the 90s knows what eventually happens – you eventually can’t cut the interest rate any further, and can’t borrow any more.
At that point, everything comes back in to line with a snap. And putting it off another six months only makes it hurt that much worse.
Human Exploitation 101
September 11, 2008
So, my first column is up over at Ethical Hacker.net – this one’s an overview of the skills required to be good at exploiting vulnerabilities in humans.
Give it a read and let me know what you think.