In this chapter, we'll go over the pre-requisites to get started on using the Workato Gem in a few sections:
This guide below showcases how you can do the following things:
- Build, install and run the Workato Gem
- Setting up your connector project
- Running CLI commands
- Building connectors
- Write and run RSpec tests for your connector
- Enable CI/CD on your Github with your connector structure
- Install RVM ("Ruby Version Manager") or a Ruby manager of your choice. You can find more at here
- Choose between Ruby versions
2.7.X,3.0.X,3.1.X. Our preferred version is2.7.6. - Verify you're running a valid ruby version. Do this by running either
ruby -vor the commands within your version manager. i.e.,rvm currentif you have installed RVM. - For Windows you need tzinfo-data gem installed as well.
gem install tzinfo-data - SDK depends on
charlock_holmesgem. Check gem's documentation if you have troubles when install this dependency. Additional details for Windows
ruby -v
Output:
ruby 2.7.XInstallation is also done through bash which then looks for our latest version of the SDK gem in rubygems.org.
gem install workato-connector-sdkVerify that your gem is correctly installed by typing in the command workato in terminal. You should see the following output:
Commands:
workato edit <PATH> # Edit encrypted file, e.g. settings.yaml.enc
workato exec <PATH> # Execute connector defined block
workato generate <SUBCOMMAND> # Generates code from template
workato help [COMMAND] # Describe available commands or one specific command
workato new <CONNECTOR_PATH> # Inits new connector folder
workato oauth2 # Implements OAuth Authorization Code flow
workato push # Upload and release connector's code
Options:
[--verbose], [--no-verbose] Quick Tip: Typing the
workatocommand allows you to know what commands are possible during development. Find out more about individual keys usingworkato help editetc.
You may also know the exact location of the Workato Gem using gem which
gem which workato-connector-sdkNow that you're familiar with some CLI tools let's get down to actual connector making! We're going to go over some of the basics you'll need that will make your life easier.
Here is a basic summary of how a self-sufficient connector project should look like: (It looks similar to a normal ruby project)
. # Your connector's root folder
├── .github # Folder which stores information about your Github workflows if you're using github
├── .gitignore # Stores the information which should not be pushed via git
├── Gemfile # Store the dependencies of your project
├── Gemfile.lock # This will be automatically created.
├── README.md # This will become your Connector's description
├── logo.png # The logo of your connector. Used when you push your connector to Workato.
├── connector.rb # Your actual connector code
├── master.key # Your master key if you're encrypting your files
├── settings.yaml.enc # Where you store your credentials for your connector
├── fixtures # Folder where your input and output JSONs are stored
├── tape_library # Folder where your VCR recordings are stored
├── .rspec
└── spec # Where you store your RSpec tests
├── connector_spec.rb
├── spec_helper.rb
└── vcr_cassettesThis folder structure isn't something you need to abide by, but we're going to use this structure for the rest of the documentation!
Quick Tip: You can use the CLI command
workato new <PATH>which will generated some of the folder above. Some have been omitted to give you freedom to design your connector project as you see fit.
So let's take a look at what each of the files should contain:
This folder stores information about your github action workflows. This is what we will use later on for running unit tests on Github.
This stores files that should not be pushed via git. Since you have a master.key which is used to encrypt your settings.yaml files and potentially your VCR recordings, this should be added to your .gitignore.
/.bundle/
/.yardoc
/_yardoc/
/coverage/
/doc/
/pkg/
/spec/reports/
/tmp/
master.key
# rspec failure tracking
.rspec_status
This file declares all the gems (dependencies) that your ruby project will need. All of them are needed for you to run rspec.
# frozen_string_literal: true
source 'https://rubygems.org'
gem 'rspec'
gem 'vcr'
gem 'workato-connector-sdk'
gem 'webmock'
gem 'timecop'
gem 'byebug'
gem 'rubocop' # Only if you want to use rubocop. Not added by default.You don't need to create this file. It'll be created later on. This file just holds a store of all the dependencies you have, their versions, and also the dependencies your dependencies might have.
This file shows up on your Github project (or other Git software you use). Use it to document what your connector does! Not created via workato new <PATH> commands.
When you use the workato push command to sync your connector with your Workato workspace, this is the default file for your connector's description.
The logo of your connector. Not created via workato new <PATH> commands.
When you use the workato push command to sync your connector with your Workato workspace, this is the default file for your connector's logo.
Well, this is your actual connector code. This file should be a replica of your connector code in Workato.
The encryption key is used to encrypt your files. This key is not only for encrypting your connection credentials BUT other things that might be sensitive, like account properties. Not created via workato new <PATH> command if you didn't select secure for your project settings.
Depending on whether you're encrypting your files or not (you really should!), it'll show as a .yaml.enc file or a simple .yaml file, respectively. Not created via workato new <PATH> commands.
Your settings file should have the following structure if you have multiple credentials.
[Your connection name]:
api_key: valid_key
domain: valid_domain
[Other connections]:
api_key: invalid_key
domain: invalid_domainIf you only require one set of credentials, you may have the credentials defined at the root level.
api_key: valid_key
domain: valid_domainTo create a settings.yaml.enc file, you'll need to run the following command - EDITOR="nano" workato edit settings.yaml.enc where you can replace nano with your preferred editor. You can either add your credentials in then or just save. Both your settings.yaml.enc and master.key file will be created. Your master.key file may have been created if you selected secure whilst using workato new <PATH>. When you choose to create new encrypted files, the same master.key will be used.
IMPORTANT: Be sure to add your master.key to your .gitignore or similar files if you're committing your project. This key will allow anyone to decrypt your files.
The folder where you may store your input and output JSONs for use in RSpec or CLI. Sample structure:
├── fixtures
│ ├── actions
│ │ └── search_customers
│ │ ├── input.json
│ │ └── output.json
│ ├── methods
│ │ └── sample_method
│ │ ├── input.json
│ │ └── output.json
│ ├── pick_lists
│ │ └── dependent
│ │ └── input.json
│ └── triggers
│ └── new_updated_object
│ ├── customer_config.json
│ ├── customer_input_poll.json
│ ├── customer_input_poll_page.json
│ ├── customer_output_fields.json
│ ├── customer_output_poll.json
│ └── customer_output_poll_page.jsonHolds standard options which will be passed to RSpec whenever it is run.
Sample .rspec contents:
--format documentation
--color
--require spec_helper
Where RSpec will store your VCR cassettes of API requests recorded. These requests are essential for stable unit tests. Not created via workato new <PATH> commands BUT created when RSpec is run.
This is the folder that will contain all your RSpec tests. RSpec is a ruby testing tool that can be used in conjunction with our Workato Gem to allow you to define, write and run unit tests for your connector!
This is your connector's main rspec file which holds all unit tests for your connector. You could split up your connector spec file into multiple folders if it helps organize your spec tests. All of your spec tests can be run in a single command using bundle exec rspec.
This file contains all the commands that should be setup before each rspec run. You may copy the files directly below. See section 5 for more details of what the spec_heper.rb file should look like.
So you've set up your project, and you're finally ready to get started on improving your connector.
workato help
Commands:
workato edit <PATH> # Edit encrypted file, e.g. settings.yaml.enc
workato exec <PATH> # Execute connector defined block
workato generate <SUBCOMMAND> # Generates code from template
workato help [COMMAND] # Describe available commands or one specific command
workato new <CONNECTOR_PATH> # Inits new connector folder
workato oauth2 # Implements OAuth Authorization Code flow
workato push # Upload and release connector's code
Options:
[--verbose], [--no-verbose] You may also gain more info about a specific command via workato help [command]
workato help [command]
[ Help for specific Workato gem command ]
workato help edit
Usage:
workato edit <PATH>
Options:
-k, [--key=KEY] # Path to file with encrypt/decrypt key.
# NOTE: key from WORKATO_CONNECTOR_MASTER_KEY has higher priority
[--verbose], [--no-verbose]
Edit encrypted file, e.g. settings.yaml.enc
NOTE: You will need to use this method to create any encrypted file. When you run this command for the first time, the
.encfile will be created and themaster.keywill be created.
NOTE: If no key is specified in your command, the gem will look for the key
master.keyin the root folder of whichever directory you're calling the command from.
workato help exec
Usage:
workato exec <PATH>
Options:
-c, [--connector=CONNECTOR] # Path to connector source code
-s, [--settings=SETTINGS] # Path to plain or encrypted file with connection configs, passwords, tokens, secrets etc
-n, [--connection=CONNECTION] # Connection name if settings file contains multiple settings
-k, [--key=KEY] # Path to file with encrypt/decrypt key.
# NOTE: key from WORKATO_CONNECTOR_MASTER_KEY has higher priority
-i, [--input=INPUT] # Path to file with input JSON
[--closure=CLOSURE] # Path to file with next poll closure JSON
[--continue=CONTINUE] # Path to file with next multistep action continue closure JSON
-a, [--args=ARGS] # Path to file with method arguments JSON
[--extended-input-schema=EXTENDED_INPUT_SCHEMA] # Path to file with extended input schema definition JSON
[--extended-output-schema=EXTENDED_OUTPUT_SCHEMA] # Path to file with extended output schema definition JSON
[--config-fields=CONFIG_FIELDS] # Path to file with config fields JSON
-w, [--webhook-payload=WEBHOOK_PAYLOAD] # Path to file with webhook payload JSON
[--webhook-params=WEBHOOK_PARAMS] # Path to file with webhook params JSON
[--webhook-headers=WEBHOOK_HEADERS] # Path to file with webhook headers JSON
[--webhook-subscribe-output=WEBHOOK_SUBSCRIBE_OUTPUT] # Path to file with webhook subscribe output JSON
[--webhook-url=WEBHOOK_URL] # Webhook URL for automatic webhook subscription
-o, [--output=OUTPUT] # Write output to JSON file
[--oauth2-code=OAUTH2_CODE] # OAuth2 code exchange to tokens pair
[--redirect-url=REDIRECT_URL] # OAuth2 callback url
[--refresh-token=REFRESH_TOKEN] # OAuth2 refresh token
[--debug], [--no-debug]
[--verbose], [--no-verbose]
Description:
The 'workato exec' executes connector's lambda block at <PATH>. Lambda's parameters can be provided if needed, see options part.
Example:
workato exec actions.foo.execute # This executes execute block of foo action
workato exec triggers.bar.poll # This executes poll block of bar action
workato exec methods.bazz --args=input.json # This executes methods with params from input.json
There are a few assumptions we make when you don't declare arguments:
--connectoris assumed to beconnector.rb--settingsis assumed to besettings.yaml.encorsettings.yamlas a fallback--connectionis NOT assumed. But if there is only one set of credentials in the file, we will use that.
Some other things of note:
--verboseallows you to track all incoming and outgoing API requests.--inputallows you to reference a file which is the json input to your execute block--outputallows you to write or overwrite the output of a specific CLI utility.
workato help generate
Commands:
workato generate help [COMMAND] # Describe subcommands or one specific subcommand
workato generate schema # Generate schema by JSON example
workato generate test # Generate empty test for connector
Use command to generate Workato Schema from a sample file. Supported inputs csv, json
workato generate help schema
Usage:
workato generate schema
Options:
[--json=JSON] # Path to JSON sample file
[--csv=CSV] # Path to CSV sample file
[--col-sep=COL_SEP] # Use separator for CSV converter
# Default: comma
# Possible values: comma, space, tab, colon, semicolon, pipe
[--api-token=API_TOKEN] # Token for accessing Workato API or set WORKATO_API_TOKEN environment variable
- Use
workato generate testto generate tests based on your connector.rb file.
workato help new
Usage:
workato new <CONNECTOR_PATH>
Options:
[--verbose], [--no-verbose]
Description:
The 'workato new' command creates a new Workato connector with a default directory structure and configuration at the path you specify.
Example: workato new ~/dev/workato/random
This generates a skeletal custom connector in ~/dev/workato/random.
This helps you to create a sample connector project. You may also use workato new ./[Connector_name] to create it in the current directory you're in. There may be secondary questions which prompt you about HTTP mocking behaviour.
create
create Gemfile
create connector.rb
create .rspec
Please select default HTTP mocking behavior suitable for your project?
1 - secure. Cause an error to be raised for any unknown requests, all request recordings are encrypted.
To record a new cassette you need set VCR_RECORD_MODE environment variable
Example: VCR_RECORD_MODE=once bundle exec rspec spec/actions/test_action_spec.rb
2 - simple. Record new interaction if it is a new request, requests are stored as plain text and expose secret tokens.
-
securemeans all your unit test's HTTP requests will be encrypted. HTTP requests are recorded via VCR to ensure your tests are stable. As such, we also provide you an easy way to encrypt these recordings so your authorization credentials are not stored in plain text. This is recommended. -
simplemeans your HTTP requests will be stored in plain text.
workato help oauth2
Usage:
workato oauth2
Options:
-c, [--connector=CONNECTOR] # Path to connector source code
-s, [--settings=SETTINGS] # Path to plain or encrypted file with connection configs, passwords, tokens, secrets etc
-n, [--connection=CONNECTION] # Connection name if settings file contains multiple settings
-k, [--key=KEY] # Path to file with encrypt/decrypt key.
# NOTE: key from WORKATO_CONNECTOR_MASTER_KEY has higher priority
[--port=PORT] # Listen requests on specific port
# Default: 45555
[--ip=IP] # Listen requests on specific interface
# Default: 127.0.0.1
[--https], [--no-https] # Start HTTPS server using self-signed certificate
[--verbose], [--no-verbose]
Implements OAuth Authorization Code flow
Use this to implement the OAuth2 Authorization code grant flow for applicable connectors. Applicable connectors are ones where the connection hash has type: 'oauth2. For more information, check out our guide on our main docs site.
workato help push
Usage:
workato push
Options:
-t, [--title=TITLE] # Connector title on the Workato Platform
-d, [--description=DESCRIPTION] # Path to connector description: Markdown or plain text
-l, [--logo=LOGO] # Path to connector logo: png or jpeg file
-n, [--notes=NOTES] # Release notes
-c, [--connector=CONNECTOR] # Path to connector source code
[--api-token=API_TOKEN] # Token for accessing Workato API.
# If present overrides value from WORKATO_API_TOKEN environment variable.
[--environment=ENVIRONMENT] # Data center specific URL to push connector code.
# If present overrides value from WORKATO_BASE_URL environment variable.
# Examples: 'https://app.workato.com', 'https://app.eu.workato.com'
[--folder=FOLDER] # Folder ID if you what to push to folder other than Home
[--verbose], [--no-verbose]
Upload and release connector's code
This allows you to push your connector code from your connector project locally to your workspace. This allows you to quickly cycle from testing connector functionality and the UX of your connector.
At this point, we should highlight some of the key differences between building your connector on Workato's Cloud SDK console and using the Workato Gem.
| Workato Cloud SDK Console | Workato Gem |
|---|---|
| Able to test connections, actions and triggers in their entirety | Able to test specific keys of a connector separately. i.e. execute: and output_fields: can be tested separately |
| Able to debug the exact look and feel of input and output fields. i.e. dynamic input and output fields | No UI but able to quickly evaluate resultant Workato schema of input and output fields using CLI Utils |
| No unit tests available | Able to convert CLI commands into unit tests quickly |
| No access to account_properties or lookup tables in debugger console | Able to store account_properties and credentials in encrypted/unencrypted formats. Able to store lookup tables in unencrypted format. |
As we continue to improve on the Workato Gem and its capabilities, more features will soon be added to the Workato Gem. In the meantime, here is an example how the Cloud console and the gem can be used in conjunction with each other.
- Upon creating a new connector on Workato, you will need to first establish connectivity on the cloud console. This is essential for you to do before you bring your connector development locally using the Workato Gem.
- After you have completed successfully creating a connection, you should have a few things on hand
- A set of working credentials
- Your connector code with a working set of credentials
- Now, you're able to start creating your project structure (defined in step 2 of this guide) with this connector code.
- Enter in your working credentials in the format detailed in step 2.8
- Now you're ready to begin development using the SDK gem.
Assuming we have a simple connector that uses API key authentication like this:
{
title: "Chargebee",
connection: {
fields: [
{
name: "api_key",
control_type: "password",
hint: "You can find your API key " \
"under 'Settings'=>'Configure Chargebee'=>'API Keys and Webhooks'" \
" in Chargebee's web console.",
label: "Your API Key"
},
{
name: "domain",
control_type: "subdomain",
url: "chargebee.com"
}
],
authorization: {
type: "basic_auth",
apply: lambda do |connection|
user(connection['api_key'])
end
},
base_uri: lambda do |connect|
"https://#{connect['domain']}.chargebee.com"
end
},
test: lambda do |connection|
get("/api/v2/plans")
end,
# More code below
}and a settings.yaml.enc or settings.yaml file with the following details
My Valid Connection:
api_key: valid_api_key
domain: valid_domain
My Invalid Connection:
api_key: invalid_api_key
domain: invalid_domainYou can now run the following commands to verify that the test: lambda function you have defined is working:
workato exec test --connection='My Valid Connection' #Output of the test: lambda function should be shown
workato exec test --connection='My Invalid Connection' #You should see a `Workato::Connector::Sdk:RequestError` highlighting 401 unauthorizedYou needn't have to declare the settings file. The Workato Gem automatically looks for a settings.yaml.enc or settings.yaml file (by this exact file name) so you don't need to declare it.
Note: The output of this lambda function often isn't important. As with the behaviour of the
test:in the cloud console, Workato only requires that this lambda function (and all HTTP requests within) are invoked successfully.
Alternatively, you may also have your settings.yaml or settings.yaml.enc file with the structure below
api_key: valid_api_key
domain: valid_domainYou can now run the following commands to verify that the test: lambda function you have defined is working:
workato exec test #Output of the test: lambda function should be shownNote: This, of course, removes one less declaration in your call. You may also choose to store your invalid credentials in another file like
invalid_settings.yaml.enc
Note: You may also see a intermediary command from the Gem asking if you'd like to refresh your access tokens. This is done when HTTP requests are made which have a response that triggers the
refresh_onblock. Selecting yes would cause the Gem to update your settings file with the latest auth credentials.
For auth code grant flows, the Workato Gem allows you to simulate the OAuth2 flow using the workato oauth2 command.
{
title: 'TrackVia',
connection: {
fields: [
{
name: 'custom_domain',
control_type: 'subdomain',
label: 'TrackVia subdomain',
hint: 'Enter your TrackVia subdomain. e.g. customdomain.trackvia.com. By default, <b>go.trackvia.com</b> will be used.',
optional: 'true'
},
{
name: 'client_id'
},
{
name: 'client_secret',
control_type: 'password'
}
],
authorization: {
type: 'oauth2',
authorization_url: lambda do |connection|
"https://#{connection['custom_domain'].presence || 'go.trackvia.com'}/oauth/authorize?response_type=code"
end,
acquire: lambda do |connection, auth_code, redirect_uri|
url = "https://#{connection['custom_domain'].presence || 'go.trackvia.com'}"
response = post("#{url}/oauth/token").payload(
redirect_uri: redirect_uri,
grant_type: 'authorization_code',
code: auth_code,
client_id: connection['client_id'],
client_secret: connection['client_secret']
).request_format_www_form_urlencoded
user_key = get("#{url}/3scale/openapiapps").params(access_token: response['access_token']).dig(0, 'userKey')
[
response,
nil,
{
user_key: user_key
}
]
end,
refresh: lambda do |connection, refresh_token|
url = "https://#{connection['custom_domain'].presence || 'go.trackvia.com'}"
post("#{url}/oauth/token").payload(
client_id: connection['client_id'],
client_secret: connection['client_secret'],
grant_type: 'refresh_token',
refresh_token: refresh_token
).request_format_www_form_urlencoded
end,
refresh_on: [401, 403],
apply: lambda { |connection, access_token|
params(user_key: connection['user_key'])
headers(Authorization: "Bearer #{access_token}")
}
},
base_uri: lambda do |connection|
if connection['custom_domain'].presence
"https://#{connection['custom_domain']}/openapi/"
else
"https://go.trackvia.com/openapi/"
end
end
},
test: ->(_connection) { get('views') },
# More code below
}and a settings.yaml.enc or settings.yaml file with the following details
client_id: valid_client_id
client_secret: valid_client_secretYou can now run the following commands to go through the OAuth2 Authorization code flow which includes a browser popup.
workato oauth2 SDK.Gem.OAuth2.Flow.mov
Note:
--verbosecan be used to detail everything, including the HTTP requests.
Now after you've successfully gone through the flow, you may be use the same workato exec test command to verify you're applying your token properly in your requests! Depending on when you received your token, you may also see a intermediary command from the Gem asking if you'd like to refresh your access tokens (if it has expired). This is done when HTTP requests are made which have a response that triggers the refresh_on block. Selecting "Yes" would cause the Gem to update your settings file with the latest auth credentials.
Take note, you may also use workato exec to execute lambdas in your authorization hash like acquire and refresh. That said, we highlight recommend you use workato exec test and workato oauth2 which handle the updating of your settings.yaml file automatically.
Continuing from the previous example, let's take a look at a simple action and invoke the individual lambda functions.
{
title: "Chargebee",
connection: {
# Connection code found above
},
test: lambda do |connection|
get("/api/v2/plans")
end,
actions: {
search_customers: {
title: "Search customers",
subtitle: "Search for customers using name",
description: "Search customer in Chargebee",
input_fields: lambda do |object_definitions|
[
{
name: "name",
label: "Name to query by",
hint: "Provide the name of the customer to query"
},
{
name: "id",
label: "Name to query by",
hint: "Provide the name of the customer to query"
}
]
end,
execute: lambda do |connection, input, input_schema, output_schema, closure|
get("/api/v2/customers",input)
end,
output_fields: lambda do |object_definitions|
[
{
name: "first_name"
},
{
name: "last_name"
},
{
name: "id"
}
]
end
}
},
}And a settings.yaml.enc or settings.yaml that is the same as the example for testing your connection.
You can now run the following commands to execute the execute: lambda function for this action.
workato exec actions.search_customers.execute --connection="My Valid Connection" --input="fixtures/actions/search_customer/search_customer_input.json" #The output of the lambda function should be shown.Let's decompose this command.
-
workato exec actions.search_customers.executedefines that you first want to test an action, followed bysearch_customerswhich is the key for the action you're testing.executetells us that you want to test theexecute:lambda function. -
--connection="My Valid Connection"tells us to use the connection named "My Valid Connection" in our settings file. If your settings file only has one set of connection settings, you don't need to define this. -
--input="fixtures/actions/search_customer/search_customer_input.json"The path to the input json file which mimics theinputargument for theexecute:lambda function.
Now, this command assumes a few things which you can also specify.
-
The file in which the credentials are stored (assumed to be
settings.yamlorsettings.yaml.enc) but you can specify it with--settings=. If theconnectionargument is used in theexecutelambda function, this is used for that. It is also used for any authorization logic required for your action's HTTP requests. -
The connector to reference for the path (
actions.search_customers.execute). Assumed to beconnector.rbbut you can specify it with--connector=
NOTE: Use
workato help execto find more arguments you can use!
So let's talk about testing a method as well. Methods in connectors are essential to reuse code for data pre or post-processing or to store Workato schema.
{
title: "Chargebee",
connection: {
# Connection code found above
},
test: lambda do |connection|
get("/api/v2/plans")
end,
methods: {
sample_method: lambda do |string1, string2|
string1 + string2
end
},
}You can now run the following commands to execute the lambda function for this method.
workato exec methods.sample_method --args='fixtures/methods/sample_method/sample_method_input.json' #The output of the lambda function should be shown.Let's decompose this command.
workato exec methods.sample_methoddefines that you first want to test a method, followed bysample_method, which is the key for the method you're testing.--args='fixtures/methods/sample_method/sample_method_input.json'tells us to execute the method with the arguments found in the "sample_method_input.json".
The sample_method_input.json might look like this:
[
"hello",
"world"
]As you can see, the json is an array where each index belongs to each argument in the method (string1 and string2), respectively. If only one argument is needed, you could give a single index array or provide the value itself!
Quick Tip: Testing methods which need to send HTTP requests Since some methods also send outgoing HTTP requests, you may supply
--settingsto supply authorization credentials. If not, the method will default to yoursettings.yaml.encorsettings.yamlfile.
You may also use byebug in conjunction with your CLI tools to debug efficiently. Do this by first adding require 'byebug' to the top of your connector file. Then you may place the word byebug anywhere in your code to set a breakpoint for additional debugging.
require 'byebug' # added before your connector
{
title: "Chargebee",
# More code here
methods: {
get_customers: lambda do
response = get('/api/v2/customers')
byebug
response
end,
sample_method: lambda do |string1, string2|
byebug
string1 + ' ' + string2
end
}
}where running the sample_method on CLI would allow you to print out variables and control the execution flow of the connector.
workato exec methods.sample_method --args='input/sample_method_input.json'
[777, 786] in connector.rb
777: byebug
778: response
779: end,
780:
781: sample_method: lambda do |string1, string2|
782: byebug
=> 783: string1 + ' ' + string2
784: end
785: }
786: }
(byebug) string1
"hello"
(byebug) string2
"world"
(byebug) c
cin byebug means continue, which allows the execution to continue forward. Read more on how to use byebug here.
So now that you can run CLI commands let's get to the fun stuff! Writing unit tests to prevent regressions with your connector.
Let's begin by revisiting that folder structure we saw in step 2.
.
├── Gemfile
├── Gemfile.lock
├── README.md
├── connector.rb
├── fixtures # Folder to store all your input and output jsons for CLI commands and tests
│ ├── actions
│ │ └── search_customers
│ │ ├── search_customers_input.json
│ │ └── search_customers_output.json
│ └── methods
│ ├── get_all_output_fields
│ │ └── args.json
│ └── make_schema_builder_fields_sticky
│ ├── make_schema_builder_fields_sticky_input.json
│ └── make_schema_builder_fields_sticky_output.json
├── invalid_settings.yaml.enc
├── master.key
├── settings.yaml.enc
├── .rspec
└── spec
├── connector_spec.rb
└── spec_helper.rbYou may see that a few more files have been added.
-
fixturesThese folders here are for you to store your input and output jsons for various parts of your connector you want to test. Keep in mind that you'll use input json files to test your connector's actions, triggers, methods etc. Input jsons should be formed manually (You may be able to design them yourself or also use the debugger console on the cloud SDK to build these jsons). Output jsons can be created from CLI commands by declaring the--outputparameter to save the output of a CLI command to a file. -
.rspecfile Holds the configurations for your RSpec runs. The.rspecfile specifies default flags that get passed to the rspec command when you run your tests. So if you want one of the options you see listed onrspec --helpto apply by default, you can add them here.--format documentationallows your tests to be grouped.--colorenables coloring in the rspec output.--require spec_helpertells your rspec runs to require yourspec_helper.rbbefore every run.
--format documentation
--color
--require spec_helper
- spec folder
Your spec folder will contain various files and folders that will allow you to run rspec - a ruby unit testing framework. Most of your tests will be contained inside the
connector_spec.rbfile but we'll go over the other files in the folder first.
spec_helper.rb- loaded in every rspec run where you can define certain common attributes. You can learn more about what configurations you can give over here.
# frozen_string_literal: true
require "webmock/rspec"
require "timecop"
require "vcr"
require "workato-connector-sdk"
require "workato/testing/vcr_encrypted_cassette_serializer"
require "workato/testing/vcr_multipart_body_matcher"
RSpec.configure do |config|
# Enable flags like --only-failures and --next-failure
config.example_status_persistence_file_path = ".rspec_status"
# Disable RSpec exposing methods globally on `Module` and `main`
config.disable_monkey_patching!
config.expect_with :rspec do |c|
c.syntax = :expect
end
end
VCR.configure do |config|
config.cassette_library_dir = "tape_library"
config.hook_into :webmock
config.cassette_serializers[:encrypted] = Workato::Testing::VCREncryptedCassetteSerializer.new
config.register_request_matcher :headers_without_user_agent do |request1, request2|
request1.headers.except("User-Agent") == request2.headers.except("User-Agent")
end
config.register_request_matcher :multipart_body do |request1, request2|
Workato::Testing::VCRMultipartBodyMatcher.call(request1, request2)
end
config.default_cassette_options = {
record: ENV.fetch('VCR_RECORD_MODE', :none).to_sym,
serialize_with: :encrypted,
match_requests_on: %i[uri headers_without_user_agent body]
}
config.configure_rspec_metadata!
endQuick Tip: This spec_helper.rb is generated for you when you use
workato new <PATH>to generate a new connector. The example above shows a spec_helper.rb which is created when your project issecure. This encrypts all VCR recordings using yourmaster.key. By default our record mode for secure isnonewhich means no new VCR cassettes are recorded. You can change this by setting a new environment variableVCR_RECORD_MODEtoonce.
vcr_cassettes- VCR allows us to record API requests and stub the response after it is recorded. Find out more here.
- Your connector_spec file (also contained in your spec folder)
Here's how your connector_spec file MAY look like. Of course, you're able to configure your spec file to your team's liking. We won't go into too much detail about how to write RSpec because it's ultimately up to you how you want to build your unit tests! Below we have a sample that you might find helpful. I also like this tutorial here.. Below, you can see an example
# frozen_string_literal: true
RSpec.describe 'connector', :vcr do
let(:connector) { Workato::Connector::Sdk::Connector.from_file('connector.rb', settings) }
let(:settings) { Workato::Connector::Sdk::Settings.from_default_file }
it { expect(connector).to be_present }
describe 'test' do
subject(:output) { connector.test(settings) }
context 'given valid credentials' do
it 'establishes valid connection' do
expect(output).to be_truthy
end
it 'returns response that is not excessively large' do
# large Test responses might also cause connections to be evaluated wrongly
expect(output.to_s.length).to be < 5000
end
end
context 'given invalid credentials' do
let(:settings) { Workato::Connector::Sdk::Settings.from_encrypted_file('invalid_settings.yaml.enc'}
it 'establishes invalid connection' do
expect { output }
.to raise_error('500 Internal Server Error')
end
end
end
endQuick Tip: You may also use the command
workato generate testto generate RSpec test stubs for you to begin writing unit tests for your connector. This handles most of the heavy lifting such as instantiating your connector or settings.
To run RSpec, you should have the project structure setup. Running rspec is as easy as running the Workato Gem in CLI. You simply type bundle exec rspec in bash in your project home directory, and rspec should begin running.
note: you may also run
rspecbut usingbundle exec rspecensures that the rspec Gem version you're using to run the tests is the version specified in your Gemfile.
You may also use run only specific tests at a time. If not, rspec will run all spec files in your spec folder.
bundle exec rspec ./spec/connector_spec.rb:16 #Runs the test or group of tests at line 16 of your spec file. First, your spec file should have included at least require 'bundler/setup' and require 'workato-connector-sdk'. Include require 'json' to read JSON files as well!
To instantiate your connector, you can use this:
let(:connector) { Workato::Connector::Sdk::Connector.from_file('connector.rb', settings) }To instantiate your settings, you can use from_default_file which defers to your settings.yaml.enc or settings.yaml file.
let(:settings) { Workato::Connector::Sdk::Settings.from_default_file }To instantiate your settings from an alternative setting file, you can use from_encrypted_file or from_file.
let(:settings) { Workato::Connector::Sdk::Settings.from_encrypted_file('invalid_settings.yaml.enc') }subject(:output) { connector.test(settings) } # executes the test: key when the subject `output` is used.This method accepts 1 argument. settings in the first position similar to the test: lambda.
Example:
context 'given valid credentials' do
it 'establishes valid connection' do
expect(output).to be_truthy # Since the test lambda gives a response, it is truthy.
end
it 'returns response that is not excessively large' do
# large Test responses might also cause connections to be evaluated wrongly
expect(output.to_s.length).to be < 5000
end
end let(:action) { connector.actions.copy_asset }This method accepts four arguments. settings in the first position. input in the second position. extended_input_schema and extended_output_schema in the third and fourth position, respectively. In our example below, we have omitted extended_input_schema and extended_output_schema as they were not used in the action.
Example:
describe 'execute' do
subject(:output) { action.execute(settings, input) }
context 'given asset'
let(:input) { JSON.parse(File.read('fixtures/actions/copy_asset/copy_asset_input.json')) }
let(:expected_output) { JSON.parse(File.read('fixtures/actions/copy_asset/copy_asset_output.json')) }
it 'uploads asset'
expect(output).to eq(expected_output)
end
end
endconnector.methods.[method_name](*input) # Where the arguments depend on your method definition. There are two ways to define your method in your tests. You can pass the arguments directly or from a file. For example, given a method:
sample_method: lambda do |string1, string2|
string1 + ' ' + string2
endIn RSpec, you may pass the string1 and string2 arguments directly.
connector.methods.sample_method("hello", "world")Or use the same JSON file sample_method_input.json we had in step 4.3
input = JSON.parse(File.read('input/sample_method_input.json'))
output = connector.methods.sample_method(*input)Here's another example, given a method:
make_schema_builder_fields_sticky: lambda do |input|
input.map do |field|
if field[:properties].present?
field[:properties] = call("make_schema_builder_fields_sticky",
field[:properties])
elsif field["properties"].present?
field["properties"] = call("make_schema_builder_fields_sticky",
field["properties"])
end
field[:sticky] = true
field
end
end,In RSpec, your tests for it may look something like this:
RSpec.describe 'methods/make_schema_builder_fields_sticky', :vcr do
let(:connector) { Workato::Connector::Sdk::Connector.from_file('connector.rb') }
let(:settings) { Workato::Connector::Sdk::Settings.from_default_file }
subject(:result) { connector.methods.make_schema_builder_fields_sticky(arg_1) }
context 'given non-sticky schema' do
let(:arg_1) { JSON.parse(File.read('fixtures/methods/make_schema_builder_fields_sticky/make_schema_builder_fields_sticky_input.json')) }
let(:expected_output) { JSON.parse(File.read('fixtures/methods/make_schema_builder_fields_sticky/make_schema_builder_fields_sticky_output.json')) }
it 'makes all fields sticky' do
expect(result).to eq(expected_output)
end
end
endFor methods that send HTTP requests, the credentials used are the ones you instantiate your connector with. i.e.
Workato::Connector::Sdk::Connector.from_file('connector.rb', settings)
This process should remain similar to other CI/CD tools. First, you want to set up dependencies in your CI/CD environment. Depending on whether you're reading this during the Workato Gem's Beta phase or not, you might need to include the Workato Gem's original gemspec and dependencies to build the Workato Gem. When the gem is publicly available, we will release the Workato Gem to rubygems.org, so bundle install should suffice.
Afterward, you will need to create a GitHub workflows file. Under the .github/workflows folder, create a ruby.yml file.
Note: If you are using encrypted settings (
settings.yaml.enc), be sure to add your master.key to your.gitignoreand set your environment variables in your Github repository. Find out more here. Your variable should be your master.key's contents. We named itWORKATO_CONNECTOR_MASTER_KEYfor the purpose of the example below. This environment variable will be used to run the rspec instead of yourmaster.keyfile which shouldn't be present in your Github repository.
name: Connector Unit Test
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
ruby-version: ['2.7', '3.0', '3.1']
steps:
- uses: actions/checkout@v2
- name: Set up Ruby
uses: ruby/setup-ruby@v1
with:
ruby-version: ${{ matrix.ruby-version }}
bundler-cache: true
- name: Run tests
env: # Only needed if using encrypted files.
WORKATO_CONNECTOR_MASTER_KEY: ${{ secrets.WORKATO_CONNECTOR_MASTER_KEY }}
run: bundle exec rspec
# - name: Push to DEV workspace # Use this to push to DEV. This can be enabled when a PR is merged.
# env:
# WORKATO_API_TOKEN: ${{ secrets.WORKATO_DEV_ENVIRONMENT_API_TOKEN}}
# run: workato pushYou may also add more Github actions for rubocop to automate this.
