21.3.0

What's Changed

Android

• Added annotation filter to cli-tool. Now you can filter tests by annotation names when you launch tests run via cli
• Fixed dynamic emulator creation when SDK 27 was selected

21.2.1

What's Changed

Android

• Hotfix. Fixed dynamic emulator screen params for cli-tool. Now it is possible to setup screen resolution and density for dynamic emulator via cli-tool

21.2.0

What's Changed

Android

• Added new feature - dynamic emulators. Now you can configure and reuse emulators with desired screen resolution and density. For example, now it is possible to use emulator with full hd resolution and 440 dpi density.
  Read more here
• Fixed bug that occurred when test execution was interrupted by a timeout, resulting in the screen recording not being downloaded
• Fixed bug with "device offline" message that sometimes happened while device booting
• The process of archiving tests artifacts has been optimized
• Improved documentation
• Added Android SDK 33 to worker and removed 24

21.1.0

What's Changed

iOS

• Xcode Test Plans are now supported in emceeplan. Learn more
• we parse more errors from XCResult and enrich reports with them

Android

• added new feature with ability to download artifacts from device custom directory on each run
• added clients network calls retry on bad responses
• made worker more stable and robust - now most unexpected errors converts to tests errors and not to workers crashes. It will give faster feedback loop for the users
• updated internal dependencies, such as Kotlin, Coroutines, etc.

21.0.0

What's Changed

This release we focused on rethinking how users run tests on-premise on our main platforms. For some time, it's been implemented differently on iOS and Android. Most iOS users tended to use simplified approach conveying tests arguments in command line rather than using a file-based approach with distinct queue configuration and test-arg-file. It led to messy scripts in CI and extra complexity for new users. On the other hand, Android users employed a declarative approach using emceeplan file. We reworked emceeplan config to match needs of all our focus platforms, made it platform agnostic(almost) with simplified configuration ready to work from scratch.

Meet new major Emcee release:

• emceeplan file format reworked to support both iOS and Android platforms similar way, a lot of test parameters are shared between platforms
• documentation and guides are reworked to match these changes, find more
• install Emcee using Homebrew, learn how

as well as changes in core Emcee technology:

Crossplatform

• queue now uses Vapor web server that brings stability and performance
• queue sends time consumption analytics for requests on all endpoint

iOS

• to prevent system overwhelming every test exception reason is now limited to 10kb

Android

• added new worker metrics. See Android worker metrics doc for more info
• added new failed tests retry strategy retryOnWorker - now it is possible to retry failed test on the same worker
• added ability to configure tests split strategy in emceeplan. Default strategy is timeWeighted
• Android emulator updated to version 33.1.24
• there is Mac OS demo mode. Now it is possible to run whole Emcee system for Android tests on the Mac (for demonstration only)
• added JUnit report generation
• updated Gradle version to 8.5 and Kotlin version to 1.9.20 in clients

20.2.0

What's Changed

Crossplatform

• Meet test history service! It tracks test results and time consumption for individual tests and allows other services to use this information to optimally schedule new test run. See more in doc (iOS, Android)
• There is a new timeWeighted test splitter that works hand in hand with test history service and provides the shortest possible time for entire test run. See more in doc (iOS, Android)

iOS

• Minimal supported target is macOS 12.0
• Get chrome tracing for the entire test run by using --trace option. See more in emceeFree runTests --help

Android

• Added readable Artifactory uploading error description
• Fixed possible crash in clients because of redundant internal checks
• Fixed possible zero test time bug
• Added check for desired sdk version before test run start
• Added fix for endless CLI process after test run
• Updated internal emulators version to 33.0.2. Other internal tools were also updated
• Added timeout for emulator boot complete
• Fixed wrong test result when test ended instantly

20.1.0

What's Changed

iOS

• By default buckets divide strategy is equally divided
• By default tests restart strategy is retry on worker
• By default queue is watching for workers and restart them if needed. See silentWorkerAutomaticRestartFeatureConfig
• Simulator setup before tests run is optimized. Now it requires less time
• Emcee now writes less logs by default. The log level can be configured by environment
• Improved plugins errors handling in workers. Memory consumption for plugin support is reduced
• Added Emcee metrics Grafana dashboard example

Android

• Improved test run result info output for clients (Gradle plugin and CLI)
• Added ability to set total tests execution timeout for clients (Gradle plugin and CLI)
• Worker now starts on 41001 port (was 80)
• Added Android metrics Grafana dashboard example
• Documentation was improved
• Android documentation now on the separate site
• Added ability to configure Queue analytics and metrics for containerized Queue
• Fixed bug with Allure report downloading on sdk level 29 and lower
• Fixed test start time event

20.0.0

What's Changed

Common:

• Added support for skipped tests on all platforms

iOS:

• There is new tests discoverer - runtimeXCTest. Use it for discovering dynamically added tests with frameworks like Quick, Kiwi, etc
• scheduleTests command dispatches tests to previously deployed queue. It's useful in systems where Emcee gets deployed with external platform operator
• default file verbosity has been changed to debug (was trace before change)
• better device management and improved cleaning routines that lead to reduced disk and memory consumption

Android:

• Another option to run tests - command line utility. Give it a try!
• Get video recorded during test run by using videoRecordStrategy
• Documentation refined

Cloud:

• Integrate Emcee test runner to your GitHub pipeline with Emcee.cloud github action
• Now Emcee offers alpha support for Appium cloud web server platform! Try it out - https://docs.emcee.cloud/cloud/appium/

Minor fixes:

• retryOnWorker tests retry mode does not cause excessive retries on queue

Announcement

Since version 20.0.0 Emcee we will change a minor version for ongoing regular releases. Major versions will be increased with significant changes or broad refactoring.

Emcee 19 — Android support, UX updates, and more

We are excited to announce that Emcee now supports the Android platform. Run Emcee queue and workers from container images in Docker or your Kubernetes cluster to shorten testing time. Keep calm, we continue to develop iOS part as well.

Android

Android workers, as well as their iOS counterparts, receive test jobs from queue. It means you get smart bucket slicing strategies, test retries, and better hardware utilization out of the box.

• Workers ship Android runtimes on board, on demand start and manage necessary devices with required sdk version. As for now we support sdk levels 24, 27, 29, 31 and will add more on a regular basis.
• It's possible to configure Android worker to send rich analytics. Use these insights when making decisions on equipment purchase or problem investigation.

Try out Android workers now!

iOS

• Xcode test plans are now supported. Build your project with test plan support xcodebuild build-for-testing -testPlan <name> and then feed the resulting xctestrun file to Emcee via --xctestrun. See more in Emcee runTests --help.
• Set http/https proxies through command line invocation or json config.
• Test discoverer now finds tests with swift concurrency.
• Minor improvements: fixed problems when Emcee might lose some analytics events, worker does better dynamic libraries lookup.

Emcee 18 — Resurrecting the Clones and Pluggables

18th release is concentrated on fixing bugs, speeding up things, and separating the functionality between features which become available after purchasing a licence and always-free version.

Emcee now has paid version

Starting from version 17, Emcee is distributed as-is as a closed source project and available in a form of prebuilt binaries. Version 18 goes beyond this and introduces two distinct products: free Emcee and paid Emcee.

Free version (available as a download on Github) allows you to test the product with up to 3 workers, however, you'll need to buy licenses for some richer functionality like:

• Allowing to use more than 3 workers
• For a technical support
• For EmceeAdmin GUI app
• For getting things done that you want to see in Emcee sooner than later.

Main functionality remains available in free version:

• For using plugins in your environment
• For using analytics, such as grafana, statsd, and kibana
• For a bit richer test discovery flows, such as discovering via runtime

Some notes about the current Github repository

The current source code has been frozen from April 2022, which was version 16. We will keep it available as is, and we're not planning on removing it. All made forks will still be present obviously as well, as we can't control them.

Version 17 is closed source, yet fully functional. You can use it without any limitations as well.

We haven't developed any plans on upstreaming changes from v17 into open source yet, however, we are considering it.

Automatic Job Deletion

This is huge one and helpful. Emcee now tracks if job is needed at all, and automatically deletes it if it is not needed to run anymore. This helps on CI: when you cancel the test run, job gets deleted automatically.
Tech note: previously Emcee client process issued a /jobDelete request which sometimes could get lost; now queue itself tracks all jobs.

Worker Resurrection

Emcee now attempts to restart its workers in case if they die. So if you decided to reboot your machine, or if it kernel panics, it will become a worker automatically.

Simulator Cloning

Emcee worker can now clone simulators instead of booting a new one. This speeds up creation quite a bit, and saves disk space. Emcee will clone any previously shut down simulator it booted. If Emcee never booted suitable simulator, it will create one instead. We advise to use smaller values of automaticSimulatorShutdown and automaticSimulatorDelete settings, like 120 seconds, to have better effect of simulator cloning.
As an example, booting a new simulator can take 1-2 minutes easily, but cloning and booting takes around 40 seconds.

Plugins are back!

You can use the new, a little simplified plugin SDK which is available here:
https://github.com/avito-tech/EmceePluginSupport

It's easier to wire up everything, as there are only two exposed targets now: EmceePlugin and EmceePluginModels.

Boilerplate code has been reduced just to:

let plugin = try Plugin { (event: PluginAppleTestEvent) in
    // process an event
}
plugin.join()

Bugs and small features and changes

• Emcee now uses socket API to talk to graphite and statsd instead of state of the art OutputStream APIs. Users complained about crashes related to graphite, and this change should help.
• You can query global queue state via /globalQueueState. Maybe this will be helpful for you to debug how queue is being formed when you submit your jobs.
• Worker ID is exposed as capability now, allowing you to pin your tests to a specific machine. Capability name is emcee.workerid and the value is actually worker ID.
• --device and --runtime arguments now accept only fully qualified Core Simulator IDs, e.g. com.apple.CoreSimulator.SimRuntime.iOS-16-4 and com.apple.CoreSimulator.SimDeviceType.iPhone-14. Please refer to xcrun simctl list for more examples.
• If you want to set environment variables for a test or app while using runTests command, set them in the calling environment with a EMCEE_ prefix.
• CPU architecture is now exposed as worker capability, allowing you to pin your tests to the machine with specific CPU arch. The key is emcee.cpu.architecture, the value is CPU arch returned by uname -a, e.g. arm64.
• testFinished event is now correctly delivered to plugins when tests times out.
• If plugin fails to start, error message will contain its standard output and error for debugging purposes.
• Emcee now deletes its temporary xcresult storage after test run finishes.
• You can now specify range of ports that workers can utilize. Queue server config has a new field workerPortRange which controls exactly that. You can also set queue port range via queuePortRange.
• New command discoverAppleTests lets you run test discovery much easier than before, without composing a test arg file
• Speaking of test arg files, they are gone from a free version of Emcee. We decided to keep them for paid release only, as they provide more flexibility.
• We are happy to say Version 18 supports macOS Ventura. Version 17 does not support Ventura.

Join our Telegram channels!

A decent place to discuss knowledge about Emcee and obtain a licence!

• Russian: https://t.me/emcee_ios

• English: https://t.me/emcee_ios_en