Someone else may want to delve into this futher because I only know about a few spokes on this wheel but here goes. Two main reasons.
1. Fees, Fees, Fees, Fees, Fees. Even the most efficient ETF's have some kind of fee involved. SPY is probably one of the most efficient.
2. An ETF is a sample of the Index it attempts to represent. For example, the S&P 500 contains 500 companies but the SPY contains probably only about 100. The manager attempts to equally weight the ETF holdings to correlate with movements in the actual index. This is not always possible and may result in discrepancies with the actual index.
Don't get too excited about trying to capitalize on a discrepency in the actual NAV of an ETF vs it's current bid/ask. There are hundreds of Sharks in the Hedge Fund industry that make their money arbitraging ETF's in the same way that they arbitrage the Futures/Spot price/Index markets. In an ETF like SPY, the discrepancy between NAV and latest trading price might be ~.01% if you're lucky. When you're playing with millions of $$$ though, nickel dime trades over time do accumulate. If a shark sees that the SPY is trading below what they perceive to be it's NAV based on the current prices of it's holdings, they flood the system with buy orders. When it's trading above it's NAV, they flood it with sell orders. A constant tug-o-war amongst the big boys. Impossible for us Plebes to attempt as this is what's called Program Trading.
Example of how difficult is to arbitrage a highly liquid ETF: On 4/9/08 SPY had a NAV of 135.36 according to Yahoo. The closing price of the SPY that day was 135.83. The next day, 4/10/08, the SPY opened at 135.35 (very close to prior day's close NAV). SPY is a traders playground.
In the world we live in today where anyone has access to a trading account, opportunities for actual arbitrage are extremely rare. Burton Malkiel made big bucks loading up on Closed End Funds in 2002 as they traded at a discount of 10-20% below their NAV. (Hence the argument against the 'Efficient Market Theory'.) As more and more Hedge Funds rush into the game, this practice has become increasingly obsolete.
In summary, fees are the real killer. The S&P 500 is a mere mathematical equation that doesn't take into account the Bid/Ask Spread or Commissions when rebalancing it's portfolio.