Running Tests with Cycle-CLI Command Line Interface

Running Tests with Cycle-CLI Command Line Interface

Problem

Running tests using the full Cycle client is not always practical, especially when combining Cycle Tests with a Continuous Integration or Continuous Testing process.

Solution

Cycle has the ability to run from the command line using Cycle-CLI.

(NOTE: Starting with Cycle 2.14, you will need to secure an access token from the Cycle application and include that access token in your Cycle-CLI command line parameters when running Cycle-CLI on a test agent in your testing pipeline. Instructions for securing an access token can be found here in the Cycle User Manual.

Starting with Cycle 2.18, we also support generating OAuth2 client credentials that provide a longer time to expiration than an access token. Instructions on creating and using client credentials can be found at this link in the Cycle User Manual.)

To execute Cycle from the command line:

1. Add the path to the Cycle executable to your PATH system environment variable (typically C:\Program Files (x86)\CycleLabs\Cycle)
-OR- alternatively you will need to specify the full path to your Cycle executable when running Cycle-CLI (Ex: C:\Program Files (x86)\CycleLabs\Cycle\cycle-cli). 

2. Navigate to your Cycle project directory and run Cycle-CLI followed by any optional parameters, with the last parameter being the name of the Feature FilePlaylist, or Group Test to be executed.
  • Note that if you are running cycle-cli from a machine where you do not also run the Cycle UI (such as in a CI/CD pipeline), you will need to use the --token argument to provide a valid token for authentication. It is recommended that this is injected as an environment variable for security. Please see the user manual linked here for additional details.
  • By default, cycle-cli will look for a .cycuser file that matches the user that you logged into Cycle 2 with when you obtained the token. E.g., if you logged in as j.doe@cyclelabs.io, it will look for a j.doe.cycuser file. If you use a more generic .cycuser file in your pipeline to store settings required for your pipeline tests to run successfully, for example, jenkins.cycuser, you can use the -u (--user-profile) argument to specify the .cycuser file to use.
Please Note: Your test agent running Cycle-CLI will need to be able to communicate with our Azure B2C instance in order to authenticate your token. Please refer to the article Addressing Cycle 2 Authentication Issues Caused by Network Configuration to get a list of URLs that need to be whitelisted for network traffic as well as other potential network security issues that may need to be resolved (such as disabling Cycle traffic SSL inspection in Zscaler and/or making Cycle proxy-aware).

Example

Example (with Cycle added to your PATH environment variable):

cycle-cli --token $Env:Token -u CycleUserName -e ALL_TESTS –p Example.cycproj C:\Cycle\Example.feature

Example (using complete executable path):

"C:\Program Files (x86)\CycleLabs\Cycle\cycle-cli" --token $Env:Token -u CycleUserName -e ALL_TESTS –p Example.cycproj C:\Cycle\Example.feature

Note: If your Feature includes steps that prompt the user, these steps will be skipped.

 

Required parameters

 

Parameter
Description
Example
Feature, Playlist, or Group Test File
Path to Feature File, Playlist, or Group Test File
C:\path\to\feature\feature_name.feature 
C:\path\to\playlist\playlist_name.cycplay
C:\path\to\playlist\group_test.cvt

Optional parameters


Parameter
Description
Example
--echo-settings
Echo the current settings to the console and exit
cycle-cli --echo-settings
-e, --error-handling [arg]
Error Handling Strategy (examples below)
cycle-cli --error-handling ALL_TESTS [files to run]
-l, --log-level [arg]
Logging level
cycle-cli --log-level verbose [files to run]
--no-tick
Turn off time-elapsed update for steps
cycle-cli --no-tick [files to run]
-o, --output-directory [arg]
Report output directory
cycle-cli --output-directory C:\Cycle\Output [files to run]
-p, --project-file [arg]
Project file (.cycproj file) or project directory. 
cycle-cli --project-file myproject.cycproj [files to run]
--settings
additional settings as key value pairs
cycle-cli --settings ProduceLocalReport=true [files to run]
-s, --settings-file [arg]
Settings file location to override settings used in Cycle client
cycle-cli --settings-file C:\Cycle\settingsoverride.json [files to run]
-t, --tags [arg]
Tags to be executed
cycle-cli --tags picking [files to run]
--token [arg]
Token for authentication (available since 2.14)
cycle-cli --token $TOKEN [files to run]
-u, --user-profile [arg]
An alternate .cycuser file to use for settings when executing tests
cycle-cli -u jenkins.cycuser [files to run]
-x, --shutdown-timeout [int]
Forcibly terminate Cycle upon test completion (in seconds) (default -1)
cycle-cli -x 300 [files to run]
-h, --help
Show help message
cycle-cli --help
--version
Show Cycle version
cycle-cli --version

Example running a playlist, supplying a token set as an environment variable, specifying the output directory, and specifying the project directory. 

"C:\Program Files (x86)\CycleLabs\Cycle\cycle-cli" --token $Env:Token -o C:\Cycle\Output -e ALL_TESTS –p Example.cycproj C:\Cycle\Example.cycplay

Error Handling Strategies


Cycle Test Type
Error Handling Strategy
Example
Features
FEATURE = End Feature Execution
SCENARIO = Skip to next Scenario
cycle-cli --error-handling FEATURE palletpick.feature
Playlists
NONE = Skip to next Scenario
SINGLE_TEST = Skip to next feature
ALL_TESTS = End Playlist Execution
cycle-cli --error-handling SINGLE_TEST listpick.cycplay
Group Tests
NONE = Stop Failed Terminal
SINGLE_TEST = Stop Failed Group
ALL_TESTS = End Group Test Execution
cycle-cli --error-handling ALL_TESTS volumetest.cvt


Command Line Help

Cycle-CLI includes a help option that will display the available command line options to you.

At the prompt enter: 

cycle-cli –-help

Example:

C:\Program Files (x86)\CycleLabs\Cycle>cycle-cli –help

Help Output: 

Usage:
  ./cycle-cli.exe [flags] [test files]

Flags:
      --echo-settings             Echo the current settings to the console and exit
  -e, --error-handling string     Error Handling Strategy - see user manual for valid options
  -h, --help                      help for ./cycle-cli.exe
  -l, --log-level string          logging level
      --no-tick                   turn off time elapsed update for steps
  -o, --output-directory string   Report output directory
  -p, --project-file string       Project file
      --settings strings          additional settings as key value pairs
  -s, --settings-file string      Settings file location
  -x, --shutdown-timeout int      Forcibly terminate Cycle upon test completion (in seconds) (default -1)
  -t, --tags strings              tags
      --token string              B2C Refresh token to use for authentication
  -u, --user-profile string       The .cycuser file to use when running cycle-cli, if you are not using <cycle username>.cycuser.
  -v, --version                   version

Project File Parameter

This can be either the Project directory or the .cycproj file.

For example, a user has a Project in c:\users\MyUser\myproj either c:\users\MyUser\myproj or c:\users\MyUser\myproj\myproj.cycproj would be acceptable arguments to follow the --Project-file or -p parameter. This can be an absolute path or relative path.

If this argument is not used, Cycle-CLI will look for a .cycproj file the current directory. So in command line, if the user changes directory to c:\users\MyUser\myproj , they would not have to use the --project-file argument. 

Cycle-CLI results

When Cycle-CLI executes, the Step results are output to the command line. Steps that pass are output with a Pass and the number of ms they took to complete. Steps that failed are output with [Fail]. Cycle-CLI completes with the line 

Feature: Feature Name [Fail]/[Pass] - {number} ms.

Any of the reporting formats available to the main Cycle client can be generated if the respective settings are applied.

Replicating Settings

In many cases, Cycle-CLI is run on a different computer than Cycle itself. It is important for teams to carefully control what settings are in use when running Cycle-CLI. It is possible to replicate the needed settings by specifying a Project file for your Cycle-CLI execution. All of the pertinent execution settings will be included in the .cycproj file. Execution settings can be stored in generic Cycle User files (.cycuser) to specify execution settings such as reporting settings and web driver locations.


    • Related Articles

    • How to Server Tune Cycle-CLI

      The Cycle application provides the ability to input additional JVM parameters to tune how Cycle runs. This ability to tune Java parameters allows you to do things such as set the maximum memory heap size allocated to the Cycle Java process or specify ...
    • Are There Prerequisites to Deploy the Cycle Appliance?

      The Cycle Appliance provides a platform that facilitates continuous testing and continuous integration through cloud-based infrastructure running Jenkins in Azure. The main objective of the Cycle Appliance is to make getting started with these ...
    • How to manage Multi-Branch Pipeline Testing with the Cycle Appliance

      The Cycle Appliance provides a platform that facilitates continuous testing and continuous integration through cloud-based infrastructure running Jenkins in Azure. A key aspect of continuous testing and continuous integration is giving your ...
    • How To Update Jira Statuses With Jenkins Pipeline In Cycle Appliance

      The Cycle Appliance provides a platform that facilitates continuous testing and continuous integration through cloud-based infrastructure running Jenkins in Azure. A key aspect of continuous testing and continuous integration is giving your ...
    • How To Post Jenkins Pipeline Results from the Cycle Appliance to Slack

      The Cycle Appliance provides a platform that facilitates continuous testing and continuous integration through cloud-based infrastructure running Jenkins in Azure. A key aspect of continuous testing and continuous integration is giving your ...