Crawlers robots

Crawler Robot is user-agent use to crawl the web, it is a generic term for any program (such as a robot or spider) used to automatically discover and scan websites. It has 101 robots to popular Web Search Engine.

Guide

1. Download visual studio code is you do not have it
2. Type F1 to open the visual studio codes command window
3. Type "install extenstion"
4. Search for "Crawler"

Features

To use Crawlers robots you can be open it using:

1. Type the Keyboard button F1 to open the visual studio codes command window
2. Insert using Keyboard Crawler: robots and Click on it using the Mouse
3. Choose Web Search Engine you need

Or

1. Click on Shift + Alt + R
2. Choose Web Search Engine you need

Or

1. Using direct Snippet commandline crawler on Workspace
2. Choose Web Search Engine you need
3. Change The Label in content attribute with
   all || none, index || noindex, follow || nofollow, notranslate, noydir, noodp, noarchive, snippet || nosnippet, noimageindex, nocache

Tip :
This Extension require HTML Language.

Requirements

1. vscode : >= v1.0.0

2. npm : >= v4.0.5

3. node : >= v5.10.0

Release Notes

0.1.0

Initial release of Crawlers robots