Try “Taurus” for narrative, readable and maintainable Load\Performance tests

One of the best aspects of BDD is that they are human friendly, plain english, narrative, easy for a human (developer) to translate his\her thought process into actions, represents end-user behavior and a living documentation. By reading the output of a BDD test execution, anyone can understand the quality at that moment and the coverage provided by those tests. Above and beyond it invites better collaboration between PO and developers to verify the acceptance criteria in an automated fashion. It helps the functional verification part of the lifecycle.

When we move beyond functional checks, Is there anything similar to convert load and performance acceptance criteria? Right now most of the tools are engineer\tester focused tools, record and playback based and GUI driven.

If not BDD, Taurus seems to solve some critical problems that exist in this space. Except Locust, python based load testing tool, most of the popular tools such as JMeter, Gatling, Tsung, Visual Studio load test are all GUI driven and very hard to get them working for distributed Agile teams where more than one person would want to contribute, bug fix, maintain. Not very easy to code review when another colleague changes the test and accept the pull request and much more troubles.

Taurus is pretty cool,

  • write load tests in plain english (yaml), although its not narrative compared to BDD, Taurus is definitely more readable.
  • source control and easy for code reviews
  • unified DSL to author the tests and more than once tool of choice to run the test (JMeter, Tsung, Gatling etc)
  • readable real time reporting
  • Integration with Blazemeter reporting services, user will see cool charts @ no cost..
  • platform independent.. has Python and Java dependency and can run on any platform where Python and Java can run

Here is a quick sneak peek

This is all you need to run a get request on bing.com

---
  scenarios:
    demo:
      requests:
        - url: "http://www.bing.com"
          method: GET

Here is how you can add load profile to it

  execution:
    concurrency: 100
    iterations: 100
    hold-for: 10m
    scenario: demo

Put this in one .yml file and you are all set to run a load test for 10 mins with 100 concurrent users 100 iterations..couldn’t get easier than this..

So, how to get started?

Visit this page for detailed instructions – http://gettaurus.org/docs/Installation/

Once you install BZT..lets write a simple first load test.

My intention is to create a load test to find 90% response time when 100 concurrent users hit this page http://finance.yahoo.com/q?s=aapl continuously for 5 mins

First, we’ll create the scenario. It needs scenarios top level node and requests underneath. Demo is the name of the scenario.

  scenarios:
    demo:
      store-cache: false
      requests:
        - url: "http://finance.yahoo.com/q?s=aapl"
          method: GET

Second, we’ll setup the load profile. “execution” is the root node that holds the load profile properties and scenario to run (demo).

  execution:
    concurrency: 100
    iterations: 100
    hold-for: 5m
    scenario: demo

Remaining steps below are all optional.
Taurus uses JMeter by default, however, you may instruct to choose something else as well. If that tool of choice doesn’t exist, BZT will download, install it for you and run the test.

  settings:
    default-executor: jmeter

Putting it together

---
  settings:
    default-executor: jmeter
  execution:
    concurrency: 2
    iterations: 2
    hold-for: 30s
    scenario: demo
  scenarios:
    demo:
      store-cache: false
      requests:
        - url: "http://finance.yahoo.com/q?s=aapl"
          method: GET

When you run the test, you should see something similar to below result (note: I reduced the load profile while running from my laptop)

Taurus 1

Depending on the platform you run the test, you might also see this cool report console as well.. its out of box

Taurus 2

Now the icing on the cake is its super easy to integrate this with Blazemeter reporting and distribute some pretty reports around..

  modules:
    blazemeter:
      token: e504c64.......
  reporting:
    - module: blazemeter
      report-name: Test run
      test: 25 user load 25 iterations for 5 mins
      project: demo

In the above code, you will have to key in your blazemeter API key.
Amend this to your existing test and run the test again..you will likely see some cool reports like below

Screen Shot 2015-10-20 at 8.06.08 PM

Screen Shot 2015-10-20 at 8.05.14 PM

You can download the logs from blazemeter report as well

Screen Shot 2015-10-20 at 8.07.44 PM

In addition to all this, your console output should leave you with a link to this .html report for distribution..

This whole process can get better but Taurus is awesome and made traditional JMeter load testing much easier.. I love it.

Advertisements

3 thoughts on “Try “Taurus” for narrative, readable and maintainable Load\Performance tests

    1. We keep a folder (vars) with separate env variables (dev.txt,test.txt,prod.txt) and supply that to bzt command along with .yaml files..if the list is longer, use a shell script for the same purpose.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s