# How do I modify robots.txt?

You can modify the `robots.txt` file by extending the `\Aero\Store\Pipelines\RobotsTxt` pipeline.

For example, you can prevent search engines from crawling the admin login page:

```php
\Aero\Store\Pipelines\RobotsTxt::extend(function (\Aero\Store\Collections\RobotsTxtCollection $content, \Closure $next) {
    $adminSlug = config('aero.admin.slug');

    $content->agent('*')->disallow("/{$adminSlug}");

    return $next($content);
});
```

---

### Parameters

<table border="1" id="bkmrk-parameter-type-descr" style="border-collapse: collapse; width: 100%;"><colgroup><col style="width: 33.3731%;"></col><col style="width: 33.3731%;"></col><col style="width: 33.3731%;"></col></colgroup><tbody><tr><td>**Parameter**</td><td>**Type**</td><td>**Description**</td></tr><tr><td>`$content`</td><td>`\Aero\Store\Collections\RobotsTxtCollection`</td><td>The collection of rules for the robots.txt file.</td></tr><tr><td>`$next`</td><td>`\Closure`</td><td>The next step in the pipeline.</td></tr></tbody></table>

### Methods

<table border="1" id="bkmrk-method-description-i" style="border-collapse: collapse; width: 100%;"><colgroup><col style="width: 50.0596%;"></col><col style="width: 50.0596%;"></col></colgroup><tbody><tr><td>**Method**</td><td>**Description**</td></tr><tr><td>`agent(string $agent)`</td><td>Gets or creates a new rule collection for the specified user agent.</td></tr><tr><td>`removeAgent(string $agent)`</td><td>Removes a user agent entirely (including all of its defined rules).</td></tr><tr><td>`allow(string $value)`  
</td><td>Adds an `Allow: $value` rule.</td></tr><tr><td>`removeAllow(string $value)`  
</td><td>Removes an `Allow: $value` rule.</td></tr><tr><td>`disallow(string $value)`  
</td><td>Adds an `Disallow: $value` rule.</td></tr><tr><td>`removeDisallow(string $value)`  
</td><td>Removes an `Disallow: $value` rule.</td></tr><tr><td>`sitemap(string $value)`  
</td><td>Adds an `Sitemap: $value` rule.</td></tr><tr><td>`removeSitemap(string $value)`  
</td><td>Removes an `Sitemap: $value` rule.</td></tr><tr><td>`crawlDelay(string $value)`  
</td><td>Adds an `Crawl-delay: $value` rule.</td></tr><tr><td>`removeCrawlDelay(string $value)`  
</td><td>Removes an `Crawl-delay: $value` rule.</td></tr><tr><td>`requestRate(string $value)`  
</td><td>Adds an `Request-rate: $value` rule.</td></tr><tr><td>`removeRequestRate(string $value)`  
</td><td>Removes an `Request-rate: $value` rule.</td></tr></tbody></table>