The supplemental index is where Google puts websites and pages it doesn't trust. The regular index features the search results that you usually see. Results from the supplemental index will show when there are not enough documents that match the query in the main index.
If your website or pages are in the supplemental index, it means that Google doesn't trust your website. There are several ways to get into the supplemental index:
Too many low quality links
Not enough links
Too many low quality outbound links
Duplicate content
Too many pages for your pagerank (low PR, many pages)
Your site is new and has a low link profile. To fix this, get more authoritative links.
There's no way of knowing if your website is in the supplemental index apart from a good guess. If you have a new website, Google may place you in the supplemental index by default, until the site ages and gains some authority. If you're in the supplemental index, Google is not likely to crawl all your URLs. Also, be sure that you do not have broken links and 404s, since those can get you into the supplemental index.
Spam Detection
As a quality website owner, you will get plenty of spam links. There's no way to offset it, but accept the fact and focus your efforts on something more productive.
As you get spam links, Google will compare inbound and outbound links from your site. If you link to bad neighborhoods, you be will considered part of the spam network, so be careful to whom you link. Assuming you get spam inbound links by default, and don't link to spam from your site, you can offset inbound spam link effects by getting more quality and authoritative links. Aaron Wall states that every single quality link is equal to 40 - 60 spam links, thus if you have two quality links and 60 spam links, in Google's eyes you will only have one quality link (this is a very rough example).
Human Reviewers and Behavioral Data
Some SEOs claim Google has up to 10,000 human reviewers who check search results for integrity and review websites flagged by algorithms. In his column Search 4.0, Danny claims that search engines are putting humans back into the task and taking behavioral aspects into account in algorithms.
Google Optimization 
