MetaBrainz Foundation
Create and integrate robots.txt file for CritiqueBrainz
CritiqueBrainz website is missing the robots.txt file. It should disallow access to the API and other pages that we don't want robots to touch. You should talk with one of the mentors and other developers in the community to decide the exact list of parts to exclude.
An example implementation can be found in the MetaBrainz website source code.
Task tags
Students who completed this task
Anshuman73
Task type
- code Code
Level
Beginner