The S&P 500 is a list of the largest and most profitable public companies. It's hard to do better than the most successful businesses. Most other indices and hedge funds don't outperform the S&P 500. Most private equity shops don't. Most real estate investors don't. So it shouldn't come as a surprise that venture capital doesn't.
Most startups don't get big. How many startups founded in the past decade have become hugely profitable? It's not that many. A handful out of the 500k or so funded startups. Meanwhile the S&P keeps chugging along at 8% annually.
It's hard to do better than the most successful businesses is not a statement that makes sense from the investor's perspective.
The price of an investment is based on the expected profitability of a company, an investment in a barely profitable company, if priced correctly, should yield returns at least equal to good companies like Apple, Google, and Microsoft, as the investment would be discounted to compensate for the poor expected future earnings of the company you are investing in.
That's getting into perfectly-spherical-cow territory, though. Investors aren't logical and neither are founders, and there's a real chance that an investor-fair deal isn't going to get any bites. The BATNA for most founders who don't secure funding is "go get a job that pays a lot of money"; while most VC investment is lopsided in the investor's favor through other means (equity preferences etc.), not landing deals makes your fund's LPs ask why they're letting you hold their bag of cash.
Most startups don't get big. How many startups founded in the past decade have become hugely profitable? It's not that many. A handful out of the 500k or so funded startups. Meanwhile the S&P keeps chugging along at 8% annually.