robots.txt

[Dudeking]

Hay, think this is the right place to post this.

I just used some random site to create this

# Robots.txt file created by http://www.webtoolcentral.com
# For domain: http://hexzero.co.uk

# All robots will spider the domain
User-agent: *
Disallow:

# Disallow directory /Dudeking/
User-agent: *
Disallow: /Dudeking/

# Disallow directory /cgi-bin/
User-agent: *
Disallow: /cgi-bin/


I don't want to get it wrong because Google will stop indexing me.

Can I safely remove all the comments?
Also what is the user-agent thing?
Is it needed?
Can I remove it?

Thanks :peace:

[Dudeking]

NVM found this:-

http://www.searchtools.com/robots/robots-txt.html