This does not provide any new information and may just triple the number of crawled pages
This commit is contained in:
부모
0db269f3dc
커밋
c4f2433300
|
@ -2,3 +2,4 @@
|
|||
|
||||
User-agent: *
|
||||
Disallow: /media_proxy/
|
||||
Disallow: /interact/
|
||||
|
|
불러오는 중...
Reference in New Issue