robots.txt 규칙 분석기
Fetch robots.txt and test whether specific URLs are allowed or blocked for any user-agent, including wildcard and path rules. Use it to debug crawler access and verify disallow rules.
도구 결과는 영어로 생성됩니다.
Fetch robots.txt and test whether specific URLs are allowed or blocked for any user-agent, including wildcard and path rules. Use it to debug crawler access and verify disallow rules.
도구 결과는 영어로 생성됩니다.