Enjoy unlimited access to all forum features for FREE! Optional upgrade available for extra perks.
Daily Diamond

Can someone help me with a 404 Document Not Found error ?

Status
Not open for further replies.

BobDiGiTaL

Domain Lover
Legacy Exclusive Member
Joined
Aug 13, 2005
Messages
1,556
Reaction score
19
I noticed this section of cpanel that lets me see errors within my website.

404 Document Not Found

Required but not found URLs (HTTP code 404)
URL (1) Error Hits - Referers - /robots.txt - 32

Do I need to create a page " /robot.txt" so this error will not occur?
Or is it no big deal?
 
Domain Days 2024

MrDude

Level 8
Legacy Platinum Member
Joined
Jun 10, 2005
Messages
1,219
Reaction score
1
You should create a blank robots.txt file, it aint accessed by users generally so wont make a difference setting up a 404 page, its search engines checking for crawling rules.
 

dcristo

Level 9
Legacy Exclusive Member
Joined
Feb 25, 2005
Messages
3,709
Reaction score
151
You should create a blank robots.txt file, it aint accessed by users generally so wont make a difference setting up a 404 page, its search engines checking for crawling rules.

robots.txt is to prevent certain pages/directories not being crawled. Not for 404 errors.
 

BobDiGiTaL

Domain Lover
Legacy Exclusive Member
Joined
Aug 13, 2005
Messages
1,556
Reaction score
19
what does it mean?
Why is someone looking for a /robots.txt file ? Should I make a /robots.txt file so it can be found. What should be in it?
 

VirtualT

Level 8
Legacy Exclusive Member
Joined
Aug 11, 2006
Messages
2,228
Reaction score
19
you don't NEED a robots.txt file, it will probably be something crawling your site.
Either ignore the error or create a robots file to stop the error, it doesn't really matter
 
Status
Not open for further replies.

The Rule #1

Do not insult any other member. Be polite and do business. Thank you!

Members Online

Sedo - it.com Premiums

IT.com

Premium Members

Premium Members

MariaBuy

Our Mods' Businesses

UrlPick.com

*the exceptional businesses of our esteemed moderators

Top Bottom