If you’ve ever built a motorcycle from scratch, you can appreciate how the process is nothing like an IKEA instruction manual.  Getting the final product just the way you like it is a journey that takes time and often requires seeking out new information and components at various steps along the way.  But the reward of squeezing out those extra few horsepower or getting the ride just the way you like it is very high, and well worth the extra effort over simply buying the easy option off the dealer lot.

While mechanical engineering is my background, these days I spend a lot more time playing with financial data than nuts and bolts.  But the fundamental thought process is still the same.  I like to think of the site and all of the underlying calculations as my own intellectual garage where I can try out new ideas, and I’m always looking for opportunities for improvement.  

In that spirit, I’ve been doing a lot of research lately not simply in portfolios or comparison calculations but more deeply in the underlying data that powers it all.  In the process, I’ve found a way to improve my stock index calculations and am happy to share what I’ve learned.

The short story is that I discovered that I missed an important factor in how indices are divided between value and growth stocks.  While CRSP divides the market in two when deciding how to initially classify a new company to the index, it does not immediately sell a company when its valuation changes and falls into the other half of the market.  Instead, it sets overlapping bands that accommodate for normal market fluctuations while minimizing unnecessary  turnover that would drive up trading fees and reduce the return to the investor.  If the total stock market is a square sorted by valuation on the vertical axis, the bands look like this:

I’ll spare you a detailed breakdown of the methodology and skip to the payoff.  By tweaking the model to accommodate index fund banding strategy rather than simply splitting the market down the middle, the final growth and value calculations are more accurate than before.  Cool!  And while I was improving the calculations I also took the opportunity to add total stock market numbers to the spreadsheet.  So now there’s even more data to play with.

For lots more information on how this all works, I’ve updated the Stock Index Calculator page with a full explanation.  You can also download the accompanying spreadsheet to study the numbers for yourself.  I’m holding off on updating all of the site data for now while I work on some other projects, but in the meantime feel free to check out the new methodology and returns.  As always, feedback is highly encouraged.

Now if you will excuse me, it’s time to head back to the garage.  Enjoy the new data!