Behaviour-Driven-Development

Earlier today we had a discussion about some confusing information on the EGI website. This led us to discuss what should be there, what shouldn’t be there, what content or links were broken, and what we could do to improve the experience of using our website.

In short, it felt like a typical user experience (UX) interview. There were a few areas for improvement, and a few “just plain errors”, which was all-in-all very useful feedback. In order for that not to go to waste, it would have been nice to capture that user story with some features and scenarios, so that we could validate and fix things.

It just so happens that I’ve been working with Cucumber as a test runner and Watir for browser automation in the context of other things, so I was happy to have something concrete to put these to the test.

Behaviour-driven development

I’ve worked a bit on test-driven development (TDD), where specific tests of states or functionality are written in order to make assertions on a thing - a service, a piece of software, etc. My favourite tools here so far have been Molecule as a test runner for Ansible and Inspec for it’s simple human-readable syntax for writing compliance statements.

This was the first time I’d come across Behaviour-Driven Development, where the way in which people interact with the thing you’re developing is investigated. The powerful thing here was the language that was used to construe the scenarios in which people might interact with services.

Case in point

Sy Holsinger brought up some incongruous behaviour on our website - although the Authentication and Authorisation Infrastructure for EGI (known as CheckIn) had been operational for years, and was being used by many users to authenticate to services, it was marked as “Beta” in the EGI Service Catalogue.

How could this be ?

• Sy

It turns out that Sy was looking at the public-facing catalogue : the things that members of other communities could request of EGI to provide to them as a service. The same product (CheckIn) that has been safely and reliably securing our federation and internal services - and which has been in production for several months already - is also available to external customers, in a beta phase¹. Although still not the best impression one might want to make on a client, this is a very different situation than if the service itself was in Beta! In this case, we could deal with the client with confidence and outline a proposed SLA for the service based in their needs, and they could have good confidence that the product works as expected, since it has been in operation in a production environment for quite a while already. The difference, although subtle, was crucial to engagement, and all lay in how these services were distinguished, between internal and external customers.

So, how do we express this subtle difference? CheckIn has the same name and icon on the internal and external services page, it’s easy for a casual observer to mistake the context.

• Perhaps a subtle visual cue could be added to the page, to remind the visitor where they were?
• Perhaps some text could be added to the external-customer facing page which led to the marketplace where order can be placed? Such a link wouldn’t make any sense on the internal services page.

The fact is that we now understood that there was a problem and had some ideas of what users expected and how to address it. Before hacking away at it though, we need to agree on a story about what kind of user experience would be suitable.

Agreeing on features and scenarios

Now, this may sound like an easy thing to fix, but many things can get lost in translation. With Gherkin syntax though, we can construct a story which is almost guaranteed to be understood in the same way by both the user or customer and the developer. The Feature that I ended up writing for this goes as follows:

Feature: Does the EGI Website correctly describe the state of CheckIn?
  Everybody expects CheckIn to be in production, but it is only production for internal users.
  External users are able to enquire about it, even though it's marked as beta

  Scenario: CheckIn is in Beta for the world, but production internally
    Given I am on www.egi.eu
      When I go to Services
      Then I see a link to Internal Services
      And  I see a link to CheckIn
      And  The page shows that CheckIn for external users is in Beta
    Given I am on the services page
      When I go to internal services
      Then it has a list of services
      And  one of them is Security
      And  The security item lists CheckIn
      But  the service is not marked in Beta

Those might sound like “plain English” descriptions (in fact, I hope they maintain a high degree of fidelity to the actual discussion we had in Slack), but they differ in two important respects:

1. The words Feature, Scenario, Given, When, etc form part of a precise vocabulary of terms which mean specific things - and these things allow us to execute the tests
2. The terms are internationalisable, i.e. this could have been written in French or Zulu and the tests would have still execute the very same tests

Using Watir² to automate a browser, we can connect that scenario to code which is the thing that makes the test executable. If we didn’t have executable tests, we would have to get a person to go clicking through the pages and checking our specification. Granted, now that we have this language, a human could execute this test reliably - and that might be used as a sanity test against our executable implementation - but they would not be able to do it tirelessly, on demand and with perfect reporting every few seconds, or on every change that we make to the story.

Automating UX testing, converging to user happiness

Now we have a ubiquitous language that can be used to express, execute and test user experience, we can get a continuous integration service to do that for us on specified triggers or at specified intervals³.

There are a few other things which Sy reported during his conversation with the customer, and it would be great to be able to collaborate easily on these. Indeed, it it would be nice to be able to collect user stories from all of the UCST team and perhaps even directly from the users and communities themselves. This could be done via the appropriate channels - project JIRAs like EGI’s or EOSC Hub’s, pull requests to the service repository, discussions on the forum - but keeping a common vocabulary.

However, just being able to tell a story, express a feature, construct a scenario with a common vocabulary doesn’t guarantee that you will agree what those features and scenarios will be - far less the implementation of the tests. One still needs to discuss. Doing this via topics on the forum is a great way to get feedback and input. When it comes to the details though, I’ve been using Cucumber Pro. You can see the current project on Cucumber Pro, which makes it far easier to collaborate around building user stories and features.

So now, all we need to do is listen to our users, write features and tests and slowly, but surely iterate towards maximal happiness for all. Rock on

References and footnotes