Release date: January 2019
Document version: 1.0
- Introduction
- Installation
- Using the Autodiscovery Tool
- Autodiscovering Devices in CloudShell
- Creating Connections on Discovered Devices
- Input Data Files
The Autodiscovery tool enables CloudShell admins to discover a large number of devices at once “into” CloudShell, instead of having to manually create them one by one in CloudShell Portal.
Since the tool performs a bulk discovery of devices, the relevant shells must be imported into CloudShell prior to running the tool.
The tool supports devices that are modeled in CloudShell, but can also be customized to discover devices that are not modeled in CloudShell, see Autodiscovering devices not modeled in CloudShell.
The basic flow for autodiscovering resources is as follows:
- Create a list of the devices you want to discover, by providing specific IPs and/or IP ranges.
- Make sure you have an appropriate shell for each device you wish to discover.
- We recommend to first search for the shells in the Quali Community Integrations page. Note that you can also extend existing shells to match the device’s specifications or create new shells from scratch - see the CloudShell Dev guide for more information.
- Import the shells into CloudShell - see Importing Shells in the CloudShell online help.
- Run the Autodiscovery tool. When the process completes, the discovered device resources are included in the Inventory dashboard in CloudShell Portal.
-
Windows, Linux, or Mac OS versions that support Python v2
-
Network access to the devices you want to discover
-
SNMP enabled on all devices
-
CloudShell version 8.0 and above
-
Python 2.7.x
Important: If you have another version of Python installed on the autodiscovery machine, this may cause unexpected behavior.
-
pip
There are two ways to install the Autodiscovery tool:
-
Install via pip (public PyPi repository):
pip install cloudshell-autodiscovery -
Install from the tool’s GitHub repository source:
git clone git@github.com:QualiSystems/cloudshell-autodiscovery.git cd cloudshell-autodiscovery pip install -r requirements.txt python setup.py installIf git is not installed, replace the first line with the following:
git clone https://github.com/QualiSystems/cloudshell-autodiscovery.git
The tool is installed in the machine’s default Python27 folder. For example: c:\Python27\Scripts folder.
To use the tool, navigate to this folder in command-line, unless you have set these folders as env variables which will allow you to run the tool from anywhere on the machine. To do so, add the python/pip installation folder paths to your local environment variables. For example C:\Python27 and C:\Python27\Scripts.
You can perform these procedures in either offline or online mode. In online mode, you generate the autodiscovered device configurations and create them as is, while in offline mode, you can optionally modify the autodiscovered device configurations before creating and discovering them into CloudShell.
To list the commands available in this tool, run the following command-line:
autodiscovery --help
To learn what version of the Autodiscovery tool you are using, run the following command-line:
autodiscovery version --help
To view specific options associated with any autodiscovery command, type the command and add --help. For example:
autodiscovery <echo-input-template> --help
This chapter explains how to discover devices in CloudShell using the Autodiscovery tool. The tool supports devices that are modeled by Quali-published shells and devices for which you have a new or extended shell.
To autodiscover devices modeled in CloudShell:
-
Open command line and navigate to the autodiscovery tool’s installation folder.
-
To generate the input file, run the following command-line:
autodiscovery echo-input-template --save-to-file input.ymlThe input file is created in the folder where you ran the command. If you want the file to be created in a different location, specify the full path to this location.
-
To generate the file in json format, change “yml” to “json”. For reference, see sample input files: Input file in YAML format or Input file in JSON format.
-
To rename the input file, use:
autodiscovery echo-input-template --save-to-file <input filename>.[yml|json]
-
-
Open the input file in your preferred editor and update the device info and CloudShell server credentials.
-
Update the fields to include the details of the devices you want to discover as follows:
Field Description IP of devices to discover devices-ips: Add a single device ip or a range of device ips and the domain in which to create them.
• range: If you want to add a string of devices, it must follow this format: xxx.xxx.xx.100-110. You can have a range within the IP address in any segment of the address, for example xxx.xxx.9.1-10.xxx.
• domain: Specify the CloudShell domain. If you want the devices to be created in the Global domain (default), omit this line.IP and credentials for the CloudShell API cloudshell:
• ip: Address of the CloudShell API
• user: Admin user on CloudShell
• password: Admin user's passwordPossible SNMP community strings community-strings: Add possible SNMP read community strings for the devices, such as public, public2 etc. Additional settings per Vendor vendor-settings:
Default:
• cli-credentials: (mandatory) Add default values to be used if all devices have the same CLI credentials, including user, password, enable password etc.
• folder-path: Add the folder path as it appears in CloudShell Resource Manager Client.
Cisco/Juniper: (vendor specific information) Add other device credentials as required for specific vendors.
• cli-credentials: Add user and password
• folder-path: Add the folder path as it appears in CloudShell Resource Manager Client. -
Save your changes. Make sure you do not save the file to a different folder.
-
-
At this point, you can choose to immediately discover the devices as resources into CloudShell or make modifications to the device settings before discovering them. Perform the appropriate procedure:
To immediately discover the devices in CloudShell:
-
Run the following command-line from the folder containing the input file:
autodiscovery run --input-file input.ymlIf you changed the file name, make sure you replace “input.yml” with the new name.
The devices are discovered in CloudShell. A discovery_report.xlsx file is produced in the input file’s containing folder, providing the following information about the discovered devices:
Field Description IP Device IP Vendor Device vendor, for example: Cisco sysObjectID OID unique name for each device Description Device description SNMP Read Community SNMP Read Community string, for example: Public Model Type Device model type, for example: router, switch, etc. Device Name Resource’s display name in CloudShell Domain Global (default) or other domain assigned in the input file in CloudShell Folder Containing folder of the resource, as displayed in CloudShell Portal’s Inventory dashboard and the Resource catalog in the blueprint and sandbox diagrams. Attributes Resource attributes, for example: Enable SNMP=False, SNMP Read Community=Public. User can add any attribute defined in the shell model. Added to CloudShell Indicates the devices that were added to CloudShell as resources. Possible values are: Skipped, Failed, and Success. Comment Any issues related to the processing of a specific device -
To generate a log file, add the tag:
--log-file <log filename> -
To run this command-line without discovering the resources on CloudShell, i.e. only create the resources in CloudShell but do not discover them, add the tag:
--no-autoload.
Note that if you use--no-autoload, after the tool creates the resources in CloudShell, you will have to manually discover each individual resource in CloudShell.
-
To edit device details before discovery:
-
To generate an Excel report, run the following command-line from the folder containing the input file:
autodiscovery run --input-file input.yml --report-file <report filename> --offlineBy default, the report file is saved to the current user’s C: drive folder. However, you can save the file in a different existing folder, for example:
autodiscovery run --input-file input.yml --report-file C:\Users\Administrator\temp\<report filename> --offline -
Update the report file as appropriate.
-
To discover the devices into CloudShell from the report, run the following command-line:
autodiscovery run-from-report --input-file input.yml --report-file <report filename>.xlsxYou must run this command from the same folder where the report file is saved. By default, the file is saved to the location where you ran the command.
-
To generate the report in console format instead of .xlsx (default), add the tag:
--report-type console -
To generate a log file, add the tag:
--log-file <log filename>
-
-
When autodiscovering unmodeled devices, you must ensure that:
-
You have created or extended shells for these devices and loaded them into CloudShell.
-
You created a vendor configuration input file called extended_vendors.json. The extended_vendors.json file provides the device details (device name, model etc.), see Offline Mode.
Note: For assistance using the extended_vendors.json file, contact customer support.
The autodiscovery tool attempts to recognize the devices, using the extended_vendors.json, and associate each one with the relevant loaded shell.
The additional vendor configuration input file does not override the input file but is added to it. This process, therefore, requires two configuration files: an “input” file and an “extended vendors configuration” file.
You can autodiscover devices, for which you created or extended your own shell(s), in two modes:
• Offline mode – should be used when you want to verify the information before creating and discovering the devices in CloudShell. This is the recommended mode when you want to autodiscover devices for which you created or extended your own shell (s) or when running the tool for the first time.
• Online mode – should be used when you do not need to verify the information before creating and discovering the devices in CloudShell.
In offline mode, the Autodiscovery tool gives you an opportunity to verify the information before creating the devices in CloudShell.
To autodiscover devices not modeled in CloudShell in offline mode:
-
To generate the input file, run the following command-line:
autodiscovery echo-input-template --save-to-file input.ymlThe input file is created in the folder where you ran the command. If you want the file to be created in a different location, specify the full path to this location.
-
To generate the file in json format, change “yml” to “json”. For reference, see sample input files: Input file in YAML format or Input file in JSON format.
-
To rename the input file, use
autodiscovery echo-input-template <input filename>.[yml|json]
-
-
Open the input file in your preferred editor and update the device info and CloudShell server credentials as explained in Autodiscovering devices modeled in CloudShell.
-
Create and update the extended vendors configuration file.
-
Run the following command-line:
autodiscovery echo-vendors-configuration-template --save-to-file extended_vendors.jsonThe extended_vendors.json file is created and saved to the folder where you ran the command.
This data file is generated only in JSON format. In the future, you will be able to generate the file in YAML format as well. For reference, see a sample input file: Additional vendors configuration file in JSON format.
-
If you want the file to be created in a different location, specify the full path to this location.
-
To rename the extended_vendors.json file, use:
autodiscovery echo-vendors-configuration-template --save-to-file <extended_vendors filename>.json
-
-
Edit the extended_vendors.json file with additional vendor information. See the Additional vendors configuration file editable parameters table for details.
-
-
Generate the discovery_report.xlsx Excel file that combines the information from the input file with the information in the additional vendors configuration file - extended_vendors.json. This file is used to discover the devices in CloudShell.
-
Run the following command-line:
autodiscovery run --input-file input.yml --config-file extended_vendors.json --offlineIf you changed the file names, you need to replace “input.yml” and/or “extended_vendors.json” with the new name(s) here.
You must run this command from the same folder where the input file and the extended_vendors.json files are saved.
A discovery_report.xlsx Excel file is saved to the folder where you ran the command.
-
Review the discovery_report.xlsx file and update the configurations accordingly.
-
Save your changes.
-
Create CloudShell resources for the devices by running the following command-line:
autodiscovery run-from-report --input-file input.yml --report-file discovery_report.xlsxYou must run this command from the folder containing the input file and the discovery_report.xlsx files.
CloudShell discovers the devices and generates a discovery_report.xlsx file, containing the autodiscovery details, in the folder where you ran the command. Use this file to troubleshoot any issues.
-
To generate the report in console format instead of .xlsx (default), add the tag:
--report-type console -
To generate a log file, add the tag:
--log-file <log filename>
-
-
In online mode, the Autodiscovery tool immediately attempts to create and discover the resources in CloudShell.
-
To generate the input file, run the following command-line:
autodiscovery echo-input-template --save-to-file input.ymlThe input file is created in the folder where you ran the command. If you want the file to be created in a different location, specify the full path to this location.
-
To generate the file in json format, change “yml” to “json”. For reference, see sample input files: Input file in YAML format or Input file in JSON format.
-
To rename the input file, use:
autodiscovery echo-input-template <input filename>.[yml|json]
-
-
Open the input file in your preferred editor and update the device info and CloudShell server credentials, as explained in Autodiscovering devices modeled in CloudShell.
-
Generate the vendor configurations data file.
-
Run the following command-line:
autodiscovery echo-vendors-configuration-template --save-to-file extended_vendors.json-
The extended_vendors.json file is saved in the folder where you ran the command. If you want the file to be created in a different location, specify the full path to this location. For reference, see a sample input file: Additional vendors configuration file in JSON format.
-
To rename the extended_vendors.json file, use:
autodiscovery echo-vendors-configuration-template --save-to-file <extended_vendors filename>.json
-
-
Edit the generated extended_vendors.json file with additional vendor information. See the Additional vendors configuration file editable parameters table for details.
-
-
Generate the input file that combines the information from the input file with the information in the additional vendors configuration file (extended_vendors.json).
-
Run the following command-line:
autodiscovery run --input-file input.yml --config-file extended_vendors.jsonIf you changed the file names, you need to replace “input.yml” and/or “extended_vendors.json” with the new name(s) here.
You must run this command from the same folder where the input file and the extended_vendors.json files are saved.
CloudShell discovers the devices and generates an Excel file discovery_report.xlsx in the folder where you ran the command, containing the autodiscovery details. Use this file to troubleshoot any issues.
-
To generate a log file, add the tag:
--log-file <log filename> -
To run this command-line without discovering the resources on CloudShell, i.e. only creating the resources in CloudShell without discovering them, add the tag:
--no-autoload
Note that if you use--no-autoload, after the tool creates the resources in CloudShell, you will have to manually discover each individual resource in CloudShell.
-
-
| Field | Description |
|---|---|
| name | Name of the vendor |
| aliases | Regex string which is an alias for the vendor name. You can include a single alias or a list of aliases. |
| type | Device type Currently, the tool only supports “networking” devices. A “networking” device is any device whose device statistics are accessed via SNMP. |
| default_os | (Optional) If the OS on the device cannot be identified, this OS is used. |
| default_prompt | Regexp string for the default prompt |
| enable_prompt | Regexp string for the enable prompt |
| operation_systems | • name: Name of the operating system • aliases: Regex string which is an alias for the OS name. You can include a single alias or a list of aliases. • default_model: Model type of the device (switch, router, etc.) • models_map: Add the aliases that will be used to refer to “switch” or “router”. If Autodiscovery cannot identify the model, the tool will use the default. • families: Resource family names for the device on CloudShell. o Switch first_gen • family_name • model_name • driver_name second_gen • family_name • model_name • driver_name o router first_gen • family_name • model_name • driver_name second_gen • family_name • model_name • driver_name |
This section describes how to connect the discovered resources to your physical network.
There are two ways to create these connections: automatic and manual.
In automatic mode, the Autodiscovery tool discovers all the ports on the specified resources and creates physical network connections based on their "Adjacent" attributes.
-
To discover and create resource connections, run the following command-line from the folder containing the input file:
autodiscovery connect-ports --<input filename>.[yml|json] --resources-names <resources name> --domain <domain>- Replace
<resources name>with the comma-separated names of the resources you want to discover and connect - Replace
<domain>with the CloudShell domain of the resources - To generate a log file, add the tag:
--log-file <log filename> - To generate the report in console format instead of .xlsx (default), add the tag:
--connections-report-type console - To only generate the report without creating any connections in CloudShell, add the tag:
--offline. This report can be used later with theautodiscovery connect-ports-from-reportcommand.
The tool generates a connect_ports_report.xlsx file containing the discovered connections in the folder where you ran the command. Use this file to troubleshoot any issues.
- Replace
Manual mode is a three-step process. First, generate a “device connections” Excel file, then edit the file with the required connections, and finally run it using the Autodiscovery tool. You can also use the connect_ports_report.xlsx report that is generated with the autodiscovery connect-ports --offline command as a “device connections” Excel file.
-
Create the resource connections Excel file.
-
In the following command-line, replace with a name for the file and run this command-line:
autodiscovery echo-excel-connections-report-template --save-to-file <connections filename>An Excel file .xlsx is generated in the folder where you ran the command. If you want the file to be created in a different location, specify the full path to this location.
-
Specify the physical port connections between the devices.
Note: For sub-resources, you must include the full address of each port in CloudShell.
Field Description Resource Name (Optional) Name of the resource where port is located. Source Port Full Name Full path to the resource’s port on CloudShell.
For example: DUT 1/Chassis 1/Module 1/Port 1Adjacent (Optional) "Adjacent" port's attribute value. Target Port Full Name Full path to the resource’s port on CloudShell.
For example: Switch 2/Chassis 1/Module 1/Port 1Domain CloudShell domain of the resources Connection Status Read-only field indicating the status after running the connect-ports command.
• Success - Ports were successfully connected
• Skipped - Connections were discovered but not added to the CloudShell
• Failed - Ports were not successfully connectedComment Read-only field indicating any additional information/error messages returned in case of a connection failure. -
Save your changes. Do not change the file name.
-
To apply the resource connections, run the following command-line from the folder containing the input file and the extended_vendors.json file:
autodiscovery connect-ports-from-report --<input filename>.[yml|json] --connections-report-file <connections filename>- To generate a log file, add the tag:
--log-file <log filename>
- To generate a log file, add the tag:
This section provides the files you will receive when you run the command to produce the input file in YAML and JSON format as well as the additional vendors configuration file in JSON format.
• Additional vendors configuration file in JSON format
# IP of devices to discover (could be a range or single one)
devices-ips:
- range: 192.168.10.3-45
domain: Some Domain
- range: 192.168.8.1-9.10
domain: Some other Domain
- 192.168.42.235
# IP and credentials for the CloudShell API
cloudshell:
ip: 192.168.85.9
user: admin
password: admin
# Possible SNMP community strings
community-strings:
- public
- public2
# Additional settings per Vendor (Possible CLI credentials (user/password), resource folder)
vendor-settings:
default:
cli-credentials:
- user: root
password: Password1
enable password: Password2we
- user: root1
password: Password2
folder-path: autodiscovery
Cisco:
cli-credentials:
- user: cisco
password: Password1
- user: cisco2
password: Password2
folder-path: cisco
Juniper:
cli-credentials:
- user: juniper
password: Password1
- user: juniper2
password: Password2
enable_password: Password2
{
"cloudshell": {
"ip": "192.168.85.9",
"password": "admin",
"user": "admin"
},
"community-strings": [
"public",
"public2"
],
"devices-ips": [
{
"domain": "Some Domain",
"range": "192.168.10.3-45"
},
{
"domain": "Some other Domain",
"range": "192.168.8.1-9.10"
},
"192.168.42.235"
],
"vendor-settings": {
"Cisco": {
"cli-credentials": [
{
"password": "Password1",
"user": "cisco"
},
{
"password": "Password2",
"user": "cisco2"
}
],
"folder-path": "cisco"
},
"Juniper": {
"cli-credentials": [
{
"password": "Password1",
"user": "juniper"
},
{
"enable_password": "Password2",
"password": "Password2",
"user": "juniper2"
}
]
},
"default": {
"cli-credentials": [
{
"enable password": "Password2we",
"password": "Password1",
"user": "root"
},
{
"password": "Password2",
"user": "root1"
}
],
"folder-path": "autodiscovery"
}
}
}
[
{
"name": "Cisco",
"aliases": [
"[Cc]iscoSystems"
],
"type": "networking",
"default_os": "IOS",
"default_prompt": ">\\s*$",
"enable_prompt": "(?:(?!\\)).)#\\s*$",
"operation_systems": [
{
"name": "IOS",
"aliases": [
"CAT[ -]?OS",
"IOS[ -]?X?[E]?"
],
"default_model": "switch",
"models_map": [
{
"model": "switch",
"aliases": [
"[Cc]atalyst",
"C2950"
]
},
{
"model": "router",
"aliases": [
"IOS[ -]?X?[E]?"
]
}
],
"families": {
"switch": {
"first_gen": {
"family_name": "Switch",
"model_name": "Cisco IOS Switch",
"driver_name": "Generic Cisco IOS Driver Version3"
},
"second_gen": {
"family_name": "CS_Switch",
"model_name": "Cisco IOS Switch 2G",
"driver_name": "Cisco IOS Switch 2G"
}
},
"router": {
"first_gen": {
"family_name": "Router",
"model_name": "Cisco IOS Router",
"driver_name": "Generic Cisco IOS Driver Version3"
},
"second_gen": {
"family_name": "CS_Router",
"model_name": "Cisco IOS Router 2G",
"driver_name": "Cisco IOS Router 2G"
}
}
}
}
]
}
]