> For the complete documentation index, see [llms.txt](https://gitbook-88.gitbook.io/ctf-writeup/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://gitbook-88.gitbook.io/ctf-writeup/ba-chang/gong-fang-shi-jie/robots.md).

# robots

**Robots协议：** 网站通过Robots协议告诉搜索引擎哪些页面可以抓取，哪些页面不能抓取。 告诉在服务器上什么文件是可以被查看的。