WebsiteDays.com create your web site free, Hosting, Coding, E-commerce, Free Website Builder. Download free website builder to create your website! Personal, and ECommerce website builders...

WebsiteDays.com for new-zealand dunedin

Click on your region below to contact a Websitedays.com designer, or Websitedays.com sales person for info about Online Website builder system.

You can also send an email to

and we can forward your enquiry to respective designers in your area.

Akaroa Akitio Albany
Alexandra Amberley Antares
Ardmore Aro Valley Arrowtown
Ashburton Auckland Balclutha
Blenheim Cambridge Carterton
Christchurch Clyde Coalgate
Collingwood Coromandel Cromwell
Dannevirke Darfield Dargaville
Devonport Dunedin East Tamaki
Eastbourne Edgecumbe Ellerslie
Eltham Endeavour Inlet Epsom
Fairlie Featherston Feilding
Flag Swamp Foxton Beach Franz Josef
Geraldine Gisborne Glen Eden
Glen Innes Glendowie Glenfield
Glenorchy Gore Great Barrier Island
Greenlane Greymouth Greytown
Hahei Hamilton Hanmer Springs
Hastings Havelock North Hawarden
Hawera Helensville Henderson
Hobsonville Hokitika Howick
Hunterville Hutt Valley Inglewood
Invercargill Island Bay Johnsonville
Kaiapoi Kaikohe Kaikoura
Kaitaia Kaiwaka Kawerau
Kerikeri Khandallah Kilbirnie
Lake Tekapo Leeston Levin
Lincoln Loburn Lower Hutt
Lumsden Lyttelton Maketu
Mangaweka Mangawhai Mangonui
Manukau Manurewa Maraetai
Martinborough Marton Massey
Masterton Matakohe Matamata
Matauri Bay Methven Milford
Morrinsville Mossburn Motueka
Mount Albert Mount Cook Mount Eden
Mount Maunganui Mount Wellington Napier
Nelson New Lynn New Plymouth
Newmarket Newton Normanby
North Shore City Northcote Oakura
Oamaru Ohakune Ohaupo
Onehunga Opotiki Opua
Orakei Orewa Otahuhu
Otaki Otane Otorohanga
Owaka Oxford Paekakariki
Paengaroa Paeroa Pahiatua
Paihia Pakuranga Palmerston North
Panmure Papakura Papamoa
Papatoetoe Paraparaumu Parnell
Pauanui Pauatahanui Penrose
Petone Picton Piopio
Pleasant Point Plimmerton Point Wells
Ponsonby Porangahau Porirua
Port Chalmers Pukekohe Queenstown
Raetihi Raglan Rangiora
Rarotonga Raumati Beach Raurimu
Remuera Renwick Riccarton
Richmond Riverton Rotorua
Royal Oak Russell Silverdale
Springfield St Arnaud Stoke
Stokes Valley Stratford Tairua
Takapau Takapuna Tapanui
Taradale Taumarunui Taupo
Tauranga Tawa Te Anau
Te Aro Te Aroha Te Awamutu
Te Kuiti Te Puke Te Rapa
Thames Tihoi Timaru
Titirangi Tokomaru Bay Tokoroa
Tolaga Bay Tuakau Tuatapere
Turangi Tutukaka Twizel
Upper Hutt Waiheke Island Waihi
Waikanae Waimate Wainuiomata
Waipara Waipawa Waipu
Waipukurau Waitakere Waitomo Caves
Waiuku Waiwera Wakefield Marlborough
Wakefield Nelson Wanaka Wanganui
Ward Warkworth Wellington
Wellsford Westport Whakapapa
Whakatane Whangamata Whangaparaoa
Whangarei Whitianga Woodville

»   The Proper Way To Use The robot.txt File

Article submitted in category: Web Design And Development
Tags: | disallow | file | user-agent | index | robottxt file | robottxt | spiders | * disallow | user-ag

The Proper Way To Use The robot.txt File by: Jimmy Whisenhunt

When optimizing your web site most webmasters don’t consider using the robot.txt file. This is a very important file for your site. It let the spiders and crawlers know what they can and can not index. This is helpful in keeping them out of folders that you do not want index like the admin or stats folder.

Here is a list of variables that you can include in a robot.txt file and there meaning:

User-agent: In this field you can specify a specific robot to describe access policy for or a “*” for all robots more explained in example.

Disallow: In the field you specify the files and folders not to include in the crawl.

The # is to represent comments

Here are some examples of a robot.txt file

User-agent: *

Disallow:

The above would let all spiders index all content.

Here another

User-agent: *

Disallow: /cgi-bin/

The above would block all spiders from indexing the cgi-bin directory.

User-agent: googlebot

Disallow:

User-agent: *

Disallow: /admin.php

Disallow: /cgi-bin/

Disallow: /admin/

Disallow: /stats/

In the above example googlebot can index everything while all other spiders can not index admin.php, cgi-bin, admin, and stats directory. Notice that you can block single files like admin.php.

About The Author

Jimmy Whisenhunt is the webmaster at VIP Enterprises http://www.vipenterprises.org

vipenter@vipenterprises.org

This article was posted on February 06, 2005