Plugins: Test harness, test fixture, docs, and local-type example (#356)

* Rename train-gem-fixture to train-test-fixture
* Update test fixture plugin to support platform forcing and basic connection setup
* First draft of docs and examples
* train-test-fixture feature complete
* Top-level files for train-local-rot13
* Unit tests pass
* Add fixture files
* Another bonkers plugin helper.
* Passing functional tests
* Enable using Train project rubocop config
* Linting
* PR feedback
* Correct unit test expected value

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

.rubocop.yml

@@ -4,14 +4,14 @@ AllCops:
   - Gemfile
   - Rakefile
   - 'test/**/*'
-  - 'examples/**/*'
+  - 'examples/plugins/train-*/test/**/*'
   - 'vendor/**/*'
 Documentation:
   Enabled: false
 AlignParameters:
   Enabled: true
 Encoding:
-  Enabled: true
+  Enabled: false
 HashSyntax:
   Enabled: true
 LineLength:

Gemfile

@@ -25,7 +25,7 @@ group :test do
   # (Gem::Specification.find_by_path('train-gem-fixture') will return nil)
   # but it's close enough to show the gempath handler can find a plugin
   # See test/unit/
-  gem 'train-gem-fixture', path: 'test/fixtures/gempath/gems'
+  gem 'train-test-fixture', path: 'test/fixtures/plugins/train-test-fixture'
 end
 
 group :integration do

docs/plugins.md

@@ -0,0 +1,149 @@
+# Train Plugins
+
+## Introducing Plugins
+
+Train plugins are a way to add new transports and platform detection to Train.
+
+If you are familiar with InSpec plugins, be forewarned; the two plugin systems are not similar.
+
+### Why Plugins?
+
+#### Benefits of plugins
+
+Plugins address two main needs that the Chef InSpec Engineering team (which maintains Train) encountered in 2017-2018:
+
+* Passionate contributor parties can develop and release new Train transports at their own pace, without gating by Chef engineers.
+* Reduction of dependency bloat within Train.  For example, since only the AWS transport needs the aws-sdk, we can move the gem dependency into the plugin gem, and out of train itself.
+
+#### Future of existing Transports
+
+The Chef InSpec Engineering team currently (October 2018) plans to migrate most existing Train transports into plugins.  For example, AWS, Azure, and GCP are all excellent candidates for migration to plugin status.  The team commits to keeping SSH, WinRM, and Local transports in Train core.  All other transports may be migrated.
+
+In the near-term, InSpec will carry a gemspec dependency on the migrated plugins.  This will continue a smooth experience for users relying on (for example) Azure.
+
+## Managing Plugins
+
+### Installing and Managing Train Plugins as an InSpec User
+
+InSpec has a command-line plugin management interface, which is used for managing both InSpec plugins and Train plugins.  For example, to install `train-aws`, simply run:
+
+```bash
+$ inspec plugin install train-aws
+```
+
+The management facility can install, update, and remove plugins, including their dependencies.
+
+### Installing Train Plugins outside of InSpec
+
+If you need a train plugin installed, and `inspec plugin` is not available to you, you can install a train plugin like any other gem.  Just be sure to use the `gem` binary that comes with the application you wish to extend.  For example, to add a Train Plugin to a ChefDK installation, use:
+
+```bash
+$ chef exec gem install train-something
+```
+
+### Finding Train plugins
+
+Train plugins can be found by running:
+
+```bash
+$ inspec plugin search train-
+```
+
+If you are not an InSpec user, you may also perform a RubyGems search:
+
+```bash
+$ gem search train-
+```
+
+## Developing Train Plugins for the Train Plugin API v1
+
+Train plugins are gems.  Their names must start with 'train-'.
+
+You can use the example plugin at [the Train github project](https://github.com/inspec/train/tree/master/examples/train-local-rot13) as a starting point.
+
+### The Entry Point
+
+As with any Gem library, you should create a file with the name of your plugin, which loads the remaining files you need.  Some plugins place them in 1 file, but it is cleaner to place them in 4: a version file, then transport, connection and platform files.
+
+### The Transport File
+
+In this file, you should define a class that inherits from `Train.plugin(1)`.  The class returned will be `Train::Plugins::Transport` or a descendant.  This superclass provides DSL methods, abstract methods, instance variables, and accessors for you to configure your plugin.
+
+Feedback about providing a clearer Plugin API for a future Plugin V2 API is welcome.
+
+#### `name` DSL method
+
+Required. Use the `name` call to register your plugin.  Pass a String, which should have the 'train-' portion removed.
+
+#### `option` DSL method
+
+The option method is used to register new information into your transport options hash. This hash contains all the information your transport will need for its connection and runtime support. These options calls are a good place to pull in defaults or information from environment variables.
+
+#### @options Instance Variable
+
+This variable includes any options you passed in from the DSL method when defining a transport. It will also merge in any options passed from the URL definition for your transport (schema, host, etc).
+
+#### `connection` abstract method
+
+Required to be implemented. Called with a single arg which is usually ignored.  You must return an instance of a class that is a descendant of `Train::Plugins::Transports::BaseConnection`.  Typically you will call the constructor with the `@options`.
+
+### Connection File
+
+The your Connection class must inherit from `Train::Plugins::Transports::BaseConnection`.  Abstract methods it should implement include:
+
+#### initialize
+
+Not required but is a good place to set option defaults for options that were passed with the transport URL. Example:
+
+```Ruby
+def initialize(options)
+  # Override for cli region from url host
+  # aws://region/my-profile
+  options[:region] = options[:host] if options.key?(:host)
+  super(options)
+end
+```
+
+#### run_command_via_connection
+
+If your transport is OS based and has the option to read a file you can set this method. It is expected to return a `Train::File::Remote::*` class here to be used upstream in InSpec. Currently the file resource is restricted to Unix and Windows platforms. Caching is enabled by default for this method.
+
+#### file_via_connection
+
+If your transport is OS based and has the option to run a command you can set this method. It is expected to return a `CommandResult` class here to be used upstream in InSpec. Currently the command resource is restricted to Unix and Windows platforms. Caching is enabled by default for this method.
+
+#### API Access Methods
+
+When working with API's it's often helpful to create methods to return client information or API objects. These are then accessed upstream in InSpec. Here is an example of a API method you may have:
+
+```Ruby
+def aws_client(klass)
+  return klass.new unless cache_enabled?(:api_call)
+  @cache[:api_call][klass.to_s.to_sym] ||= klass.new
+end
+```
+
+This will return a class and cache the client object accordingly if caching is enabled. You can call this from a inspec resource by calling `inspec.backend.aws_client(AWS::TEST::CLASS)`.
+
+#### local?
+
+This flag helps Train decide what detection to use for OS based platforms. This should be set to `true` if your transport target resides in the same instance you are running train from. This setting is not needed for API transports or transports that do not use platform detection.
+
+#### platform
+
+`platform` is called when InSpec is trying to detect the platform (OS family, etc).  We recommend that you implement platform in a separate Module, and include it.
+
+### Platform Detection
+
+Platform detection is used if you do not specify a platform method for your transport. Currently it is only used for OS (Unix, Windows) platforms. The detection system will run a series of commands on your target to try and determine what platform it is. This information can be found here [OS Specifications](https://github.com/inspec/train/blob/master/lib/train/platforms/detect/specifications/os.rb).
+
+When using an API or a fixed platform for your transport it's suggested you skip the detection process and specify a direct platform. Here is an example:
+
+```Ruby
+def platform
+ Train::Platforms.name('Aws').in_family('cloud')
+ force_platform!('Aws',
+   release: '1.2',
+ )
+end
+```

examples/plugins/train-local-rot13/Gemfile

@@ -0,0 +1,20 @@
+# encoding: utf-8
+source 'https://rubygems.org'
+
+# This is Gemfile, which is used by bundler
+# to ensure a coherent set of gems is installed.
+# This file lists dependencies needed when outside
+# of a gem (the gemspec lists deps for gem deployment)
+
+# Bundler should refer to the gemspec for any dependencies.
+gemspec
+
+# Remaining group is only used for development.
+group :development do
+  gem 'bundler'
+  gem 'byebug'
+  gem 'inspec', '>= 2.2.112' # We need InSpec for the test harness while developing.
+  gem 'minitest'
+  gem 'rake'
+  gem 'rubocop', '= 0.49.1' # Need to keep in sync with main InSpec project, so config files will work
+end

examples/plugins/train-local-rot13/LICENSE

@@ -0,0 +1,13 @@
+Copyright (c) 2018 Chef Software Inc.
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

examples/plugins/train-local-rot13/README.md

@@ -0,0 +1,72 @@
+# Example Train Plugin - train-local-rot13
+
+This plugin is provided as a teaching example for building a Train plugin.  Train plugins allow you to connect to remote systems or APIs, so that other tools such as InSpec or Chef Workstation can talk over the connection.
+
+train-local-rot13's functionality is simple: it acts as a local transport (targeting the local machine), but it applies the [rot13](https://en.wikipedia.org/wiki/ROT13) trivial cypher transformation on the contents of each file it reads, and on the stdout of every command it executes.
+
+Please note that ROT13 is an incredibly weak cypher, and can be broken by most elementary school students.  Do not use this plugin for security purposes.
+
+## Relationship between InSpec and Train
+
+Train itself has no CLI, nor a sophisticated test harness.  InSpec does have such facilities, so installing Train plugins will require an InSpec installation.  You do not need to use or understand InSpec.
+
+Train plugins may be developed without an InSpec installation.
+
+## To Install this as a User
+
+You will need InSpec v2.3 or later.
+
+If you just want to use this (not learn how to write a plugin), you can so by simply running:
+
+```
+$ inspec plugin install train-local-rot13
+```
+
+You can then run:
+
+```
+$ inspec detect -t local-rot13://
+== Platform Details
+
+Name:      local-rot13
+Families:  unix, os, windows, os
+Release:   0.1.0
+Arch:      example
+
+$ inspec shell -t local-rot13:// -c 'command("echo hello")'
+uryyb
+```
+
+## Features of This Example Kit
+
+This example plugin is a full-fledged plugin example, with everything a real-world, industrial grade plugin would have, including:
+
+* an implementation of a Train plugin, using the Train Plugin V1 API, including
+  * a Transport
+  * a Connection
+  * Platform configuration
+* documentation (you are reading it now)
+* tests, at the unit and functional level
+* a .gemspec, for packaging and publishing it as a gem
+* a Gemfile, for managing its dependencies
+* a Rakefile, for running development tasks
+* Rubocop linting support for using the base Train project rubocop.yml (See Rakefile)
+
+You are encouraged to use this plugin as a starting point for real plugins.
+
+## Development of a Plugin
+
+[Plugin Development](https://github.com/inspec/train/blob/master/docs/dev/plugins.md) is documented on the `train` project on GitHub.  Additionally, this example
+plugin has extensive comments explaining what is happening, and why.
+
+### A Tour of the Plugin
+
+One nice circuit of the plugin might be:
+ * look at the gemspec, to see what the plugin thinks it does
+ * look at the functional tests, to see the plugin proving it does what it says
+ * look at the unit tests, to see how the plugin claims it is internally structured
+ * look at the Rakefile, to see how to interact with the project
+ * look at lib/train-local-rot13.rb, the entry point which InSpec will always load if the plugin is installed
+ * look at lib/train-local-rot13/transport.rb, the plugin "backbone"
+ * look at lib/train-local-rot13/connection.rb, the plugin implementation
+ * look at lib/train-local-rot13/platform.rb, OS platform support declaration

examples/plugins/train-local-rot13/Rakefile

@@ -0,0 +1,41 @@
+# A Rakefile defines tasks to help maintain your project.
+# Rake provides several task templates that are useful.
+
+#------------------------------------------------------------------#
+#                    Test Runner Tasks
+#------------------------------------------------------------------#
+
+# This task template will make a task named 'test', and run
+# the tests that it finds.
+require 'rake/testtask'
+
+Rake::TestTask.new do |t|
+  t.libs.push 'lib'
+  t.test_files = FileList[
+    'test/unit/*_test.rb',
+    'test/integration/*_test.rb',
+    'test/function/*_test.rb',
+  ]
+  t.verbose = true
+  # Ideally, we'd run tests with warnings enabled,
+  # but the dependent gems have many warnings. As this
+  # is an example, let's disable them so the testing
+  # experience is cleaner.
+  t.warning = false
+end
+
+#------------------------------------------------------------------#
+#                    Code Style Tasks
+#------------------------------------------------------------------#
+require 'rubocop/rake_task'
+
+RuboCop::RakeTask.new(:lint) do |t|
+  # Choices of rubocop rules to enforce are deeply personal.
+  # Here, we set things up so that your plugin will use the Bundler-installed
+  # train gem's copy of the Train project's rubocop.yml file (which
+  # is indeed packaged with the train gem).
+  require 'train/globals'
+  train_rubocop_yml = File.join(Train.src_root, '.rubocop.yml')
+
+  t.options = ['--display-cop-names', '--config', train_rubocop_yml]
+end

examples/plugins/train-local-rot13/lib/train-local-rot13.rb

@@ -0,0 +1,21 @@
+# This file is known as the "entry point."
+# This is the file Train will try to load if it
+# thinks your plugin is needed.
+
+# The *only* thing this file should do is setup the
+# load path, then load plugin files.
+
+# Next two lines simply add the path of the gem to the load path.
+# This is not needed when being loaded as a gem; but when doing
+# plugin development, you may need it.  Either way, it's harmless.
+libdir = File.dirname(__FILE__)
+$LOAD_PATH.unshift(libdir) unless $LOAD_PATH.include?(libdir)
+
+# It's traditonal to keep your gem version in a separate file, so CI can find it easier.
+require 'train-local-rot13/version'
+
+# A train plugin has three components: Transport, Connection, and Platform.
+# Transport acts as the glue.
+require 'train-local-rot13/transport'
+require 'train-local-rot13/platform'
+require 'train-local-rot13/connection'