# wildcard tells all robots to crawl site User-agent: * # excludes the following pages from the crawl Disallow: /images/ Disallow: /members/ Disallow: /crow-gallery1/ Disallow: /crow-gallery2/ Disallow: /crow-gallery3/ Disallow: /crow-gallery4/ Disallow: /crow-gallery5/ Disallow: /crow-gallery6/ Disallow: /crow-gallery7/ Disallow: /crow-gallery8/ Disallow: /crow-gallery9/ Disallow: /crow-gallery10/ Disallow: /crow-gallery11/ Disallow: /crow-gallery12/ Disallow: /FTP Control File/ Disallow: /WinGrab Settings for Thumbnail Pages/