Membership is FREE, giving all registered users unlimited access to every DNForum feature, resource, and tool! Optional membership upgrades unlock exclusive benefits like profile signatures with links, banner placements, appearances in the weekly newsletter, and much more - customized to your membership level!

Can someone help me with a 404 Document Not Found error ?

Status
Not open for further replies.

BobDiGiTaL

Domain Lover
Legacy Exclusive Member
Joined
Aug 13, 2005
Messages
1,556
Reaction score
19
I noticed this section of cpanel that lets me see errors within my website.

404 Document Not Found

Required but not found URLs (HTTP code 404)
URL (1) Error Hits - Referers - /robots.txt - 32

Do I need to create a page " /robot.txt" so this error will not occur?
Or is it no big deal?
 

David G

Internet Entrepreneur
Legacy Exclusive Member
Joined
Apr 15, 2002
Messages
5,755
Reaction score
63
No, do not see a connection with robots.txt.

You could put 404 error code forwarding inside an htaccess file in the same folder where the index page is located and forward the 404's to it.
 

dcristo

Level 9
Legacy Exclusive Member
Joined
Feb 25, 2005
Messages
3,709
Reaction score
151
You can create a custom 404 error page, which you define in your .htaccess file.
 

MrDude

Level 8
Legacy Platinum Member
Joined
Jun 10, 2005
Messages
1,219
Reaction score
1
You should create a blank robots.txt file, it aint accessed by users generally so wont make a difference setting up a 404 page, its search engines checking for crawling rules.
 

dcristo

Level 9
Legacy Exclusive Member
Joined
Feb 25, 2005
Messages
3,709
Reaction score
151
You should create a blank robots.txt file, it aint accessed by users generally so wont make a difference setting up a 404 page, its search engines checking for crawling rules.

robots.txt is to prevent certain pages/directories not being crawled. Not for 404 errors.
 

lilian

Level 6
Legacy Platinum Member
Joined
Jun 28, 2004
Messages
586
Reaction score
0
You can create a custom 404 error page and redirect it.
 

BobDiGiTaL

Domain Lover
Legacy Exclusive Member
Joined
Aug 13, 2005
Messages
1,556
Reaction score
19
what does it mean?
Why is someone looking for a /robots.txt file ? Should I make a /robots.txt file so it can be found. What should be in it?
 

VirtualT

Level 8
Legacy Exclusive Member
Joined
Aug 11, 2006
Messages
2,228
Reaction score
19
you don't NEED a robots.txt file, it will probably be something crawling your site.
Either ignore the error or create a robots file to stop the error, it doesn't really matter
 
Status
Not open for further replies.

Who has viewed this thread (Total: 1) View details

Who has watched this thread (Total: 1) View details

Top Bottom