Avoid Losses: 5 Ways Investment Researchers Lie To You

Things are seldom as they appear…

Nowhere is this more true than with investment research. Every day another trading system or sure-fire way to riches gets published.

It always looks good on the surface – impressive statistics, testimonials, lots of social proof. You will find them in best-selling books, or published by an authority figure (like a professor), or endorsed by a trade journal.

None of that matters. It can’t be trusted.

There is still a good chance it can be total hogwash so you must do your own due diligence. You can’t blindly invest your savings based on third party research no matter how legitimate it appears on the surface… unless you are willing to lose your money.

How Investment Research Creates Fiction From Fact

Any time humans interact with data there is opportunity for error. Below are the 5 most common ways problems occur…

Data Errors: Investment research requires data sampling to keep the project from becoming unwieldy. Unfortunately, sampling introduce bias – by definition. Do you exclude or include companies that have gone out of business (survivorship bias)? What time periods do you test (cyclical bias)? Big cap, small cap, or micro-cap? End of day data, intra-day, weekly, or monthly? Every decision will affect the results – some quite dramatically. Just because an investment strategy appears to work on one time period does not imply it will work on another time period… or even in the future (which is the only time period that matters).
Undisclosed Assumptions: How are transaction costs accounted for? Did the system assume reasonable fills during the 2008-09 decline or the 1987 crash? What are the assumptions for the bid/ask spread? Commissions? Depending on trading frequency these small details can completely invalidate an otherwise stellar investment strategy.
Judgment vs. Mathematical Rigor: Beware of any “research” that allows room for interpretation, vagary, or judgment. The human mind can and will deceive (both in love and investment research). If the study isn’t mathematically rigorous then skip it. It’s either quantified or it’s fiction.
Interpretive Bias: Marketers use statistics like a drunkard uses a light pole – for support instead of illumination. You can’t understand a person’s opinion until you know the shoes they stand in. What are they selling? Did they build a business around this research? Who paid for the research? Do they have an axe to grind? All these things can pervert the science of statistics into supporting biased opinion and business expediency. Always analyze the source of the research and their financial motivations.
Algorithmic Errors: Some researchers simply screw up. They publish results that include blatant algorithmic errors or data errors. Seriously, it happens with surprising frequency. Worse yet, the flawed conclusions then get repeated throughout the blogosphere creating social proof and authority making the work appear indisputable.

The Devil Is In The Details

These concerns may sound like a bunch of financial geek details reserved for propeller-heads with thick glasses and pens in their pockets… but it’s not.

These concepts can make or break your financial future. They are critically important.

Think about it – you must base your investment decisions on something. If the logic and research behind those decisions is flawed then the outcome will likely be expensive. Knowing the difference between valid and fictitious investment strategy is critically important to your financial security.

The issues discussed above are not reserved for data freaks: they are relevant to every investor… including you.

The essence of investing is to only put your assets at risk for a positive mathematical expectation. Anything less is gambling. The only way you can know if your investment strategy provides a positive mathematical expectation is solid research. Period.

And if I’m sounding like a curmudgeon please understand this is based on decades of actual research myself and digesting other professional’s research. The process is far more tenuous than anyone who has not personally dug deep into this subject can be expected to understand.

I spent a decade of my life verifying and testing published trading systems for a hedge fund. Surprisingly, almost none of them worked as published. Seriously! The models were riddled with flawed assumptions and data issues that made them unusable for real-time investing.

Granted, I did this research a decade ago so you could argue the market is more sophisticated now… but I doubt it. See this post by the Empirical Finance Blog on The Magic Formula for current evidence. If you think this is an isolated circumstance then remember the infamous “Beardstown Ladies” fraud and how their published track record was complete nonsense.

These are best-selling books by major publishing houses offering questionable research conclusions.

I know it shouldn’t be this way and you should be able to trust what you see in writing, but it’s not the way the game works. Sorry.

It all boils down to a simple equation – small errors in investment research compound into HUGE errors in results. It is the nature of compound returns. The equation is multiplicative – both for growing wealth and compounding errors in research.

The devil is in the details.

2 Ways To Solve The Problem

So what’s an investor to do? How can you determine what investment strategies are sufficiently trustworthy to justify putting capital at risk?

Verify the research yourself: Yes, it is a lot of work and probably not a viable solution for most, but it is an acceptable alternative if you have a passion for this subject.
Find independent, third-party verification: If you won’t verify it yourself then you will have to find someone else who did. Fortunately, with the growth of the internet this is frequently a viable alternative. Third party bloggers and competing authors often corroborate claims.

That’s it! Those are really the only two ways to resolve the issue. Either verify the research yourself or find someone else who did. Either way, it must be verified and deeply understood before putting capital at risk. Anything less is gambling.

In addition, I like to see two more pieces of evidence before putting capital at risk on investment research…

Real-time track record: I prefer the track record be independently audited and to include data periods that represent adverse conditions for the investment strategy. For example, if it is a trend-following strategy I like to see how the strategy managed risk during sideways markets. This is critically important.
Market logic: I avoid any investment approach that is rooted in data-mining instead of market logic. In other words, the math should quantify an exploitable inefficiency in market price behavior and not just be a statistical aberration of the data.

I understand this is all a painful inconvenience. Investing should be easier and you should be able to trust published research. Unfortunately, historical fact disagrees.

The reality is money is hard to save and easy to lose. It may take extra effort to complete proper due diligence, but I guarantee it will be far less effort than having to replace investment losses from earned income.

So what do you think? Maybe you can cite some more examples of bogus research in the comments below? What ideas did I miss and what thoughts would you like to add? Please tell me your thoughts in the comments below…