User-agent: * # match all bots Disallow: /forum/ Disallow: /images/ Disallow: /pages/ Disallow: /pdf/ Allow: index.htm