Daily Pakistani Latest Tutorials, TV Serials, Movies, Trailers, Songs and Softwares

Create robots.txt File for Your Website

Create robots.txt File for Your Website

Create robots.txt File for Your Website

How to create a robot file? Why it’s important for your website? How you can create a simple robot text file through which you can tell search engine what they should index. Through robots.txt Website owner gave instruction to Web Robots.

In Robots.txt file we write the code in this way:

User-agent: *

Disallow: /

The “User-agent: *” means this section will apply to all robots. The “Disallow: /” tells the robots that it should not visit any pages on the site.

You can also prevent directories to index from search engine. Robots will not visit any of your website page from which you will disallow them.

To exclude all robots from the entire server:

User-agent: *

Disallow: /

To allow all robots complete access:

User-agent: *

Disallow:

Or just create an empty robots.txt file, or do not use one at all.

To exclude all robots from part of the server:

User-agent: *

Disallow: /tmp/

Disallow: /junk/

Disallow: /inbox/

To exclude a single robot:

User-agent: BadBot

Disallow: /

To allow a single robot:

User-agent: Google

Disallow:

User-agent: *

Disallow: /

To exclude all files except one:

User-agent: *

Disallow: /~zee/stuff/

Instead you can overtly prohibit all disallowed pages:

User-agent: *

Disallow: /~zee/inbox.html

Disallow: /~zee/text.html

Disallow: /~zee/bar.html

This was all about robots file. Another way you can tell robots about your sitemap:

User-agent: *

Allow: /

Sitemap: http://www.yourdomain.com/sitemap.xml

This is the way through which you can tell search engines to about your sitemap. In my further advance tutorial about SEO, I will teach you that how you can make your website rank better. You should apply all those techniques which one I am teaching to you. If you have any problem while creating robots.txt file you can tell us. We will help you.

Hasb e Haal 12 May 2017 TV Serials

Hasb e Haal 12 May 2017[..]

Watch Online Pakistani Talk Show Hasb e Haal 12 May 2017 Azizi as Ranjha. Watch [...]

Kal Tak with Javed Ch 10 May 2017 TV Serials

Kal Tak with Javed Ch 10 May 2017[..]

Watch Online Pakistani Talk Show Kal Tak with Javed Ch 10 May 2017 Mustafa Kamal[...]

Khabardar with Aftab Iqbal 5 May 2017 TV Serials

Khabardar with Aftab Iqbal 5 May 201[..]

Watch Online Pakistani Talk Show Khabardar with Aftab Iqbal 5 May 2017 russian j[...]

Chai Toast aur Host with Anoushay Ashraf 3 May 2017 TV Serials

Chai Toast aur Host with Anoushay As[..]

Watch daily pakistani show Chai Toast aur Host with Anoushay Ashraf 3 May 2017. [...]