1. @nuxtjs/robots
A Nuxt.js module that injects a middleware to generate a robots.txt file
@nuxtjs/robots
Package: @nuxtjs/robots
Created by: nuxt-community
Last modified: Fri, 18 Nov 2022 20:19:54 GMT
Version: 3.0.0
License: MIT
Downloads: 246,319
Repository: https://github.com/nuxt-community/robots-module

Install

npm install @nuxtjs/robots
yarn add @nuxtjs/robots

@nuxtjs/robots

npm version
npm downloads
Github Actions CI
Codecov
License

A Nuxt.js module that injects a middleware to generate a robots.txt file

Features

  • Nuxt 3 and Nuxt Bridge support
  • Generate robots.txt for static mode
  • Add middleware for robots.txt

Setup

  1. Add @nuxtjs/robots dependency to your project
 yarn add @nuxtjs/robots # or npm install @nuxtjs/robots
  1. Add @nuxtjs/robots to the modules section of nuxt.config.js
 export default {
  modules: [
    // Simple usage
    '@nuxtjs/robots',

    // With options
    ['@nuxtjs/robots', { /* module options */ }]
  ]
}

Using top level options

 export default {
  modules: [
    '@nuxtjs/robots'
  ],
  robots: {
    /* module options */
  }
}

Options

configPath

  • Type: String
  • Default: robots.config

rules

  • Type: Object|Array
  • Default:
 {
  UserAgent: '*',
  Disallow: ''
}

Robots config

If you need to use function in any rule, you need to create a config file through the configPath option

 export default [
  { UserAgent: '*' },
  { Disallow: '/' },
  { BlankLine: true },
  { Comment: 'Comment here' },
      
  // Be aware that this will NOT work on target: 'static' mode
  { Sitemap: (req) => `https://${req.headers.host}/sitemap.xml` }
]

output:

 User-agent: *
Disallow: /

# Comment here
Sitemap: https://robots.nuxtjs.org/sitemap.xml

The keys and values available:

  • UserAgent = User-agent
  • CrawlDelay = Crawl-delay
  • Disallow = Disallow
  • Allow = Allow
  • Host = Host
  • Sitemap = Sitemap
  • CleanParam = Clean-param
  • Comment = # Comment
  • BlankLine = Add blank line

Note: Don't worry, keys are parsed with case insensitivity and special characters.

Contributing

You can contribute to this module online with CodeSandBox:

Edit @nuxtjs/robots

Or locally:

  1. Clone this repository
  2. Install dependencies using pnpm install
  3. Prepare development server using pnpm dev:prepare
  4. Build module using pnpm build
  5. Launch playground using pnpm dev

License

MIT License

Copyright (c) - Nuxt Community

RELATED POST

Enhancing Vue.js Development: Harnessing the Potential of Vue-Loader

Enhancing Vue.js Development: Harnessing the Potential of Vue-Loader

Simplify Data Validation in Vue.js: A Step-by-Step Guide to Using Regex

Simplify Data Validation in Vue.js: A Step-by-Step Guide to Using Regex

Troubleshooting Made Easy: Common Issues and Solutions with vue-loader Without vue-cli

Troubleshooting Made Easy: Common Issues and Solutions with vue-loader Without vue-cli

Optimizing Webpack 4 with Vue CLI 3: Disabling the Cache-Loader

Optimizing Webpack 4 with Vue CLI 3: Disabling the Cache-Loader

Step-by-Step Guide: How to Add a Function to Your Vuex Plugin

Step-by-Step Guide: How to Add a Function to Your Vuex Plugin