Google Crawls The Deep Web
Google has announces that they are experimenting with technologies to index the deep web, the sites hidden behind forms, in order to be
the gateway to large volumes of data beyond the normal scope of search engines. Directions like
nofollowand
noindexare still respected, so sites can still be excluded from this type of search