Testing apps that depend on external services or APIs

A lot of web programming these days doesn’t live in isolation. We consume services over the internet via APIs, and we send information to other services, such as Google Analytics, to track what users are doing.

I’m currently back at the BBC, where we have a lot of teams working on components that cooperate together: programme scheduling and publishing, video encoding, media streaming, DRM verification, content delivery networks. Testing becomes a great challenge when dealing with so many moving parts, across multiple environments (dev, int, stage and live) .. and even when it works, integration tests can get slow.

What do we really mean by integration testing?

I think automated integration testing should test the integration points .. not whether they actually do all line up and integrate correctly. There is a place for both, of course. Sometimes you need full end-to-end integration tests. But they are slow and brittle. I think we can add a little bit more control here.

Example of a dependency on an external API

Suppose I want to test an app that relies on an external service. In this case it’s a weather app.

You can visit this now if you like: aimeerivers.com/weather/Bristol

Change the city in the URL and you’ll find out the weather for that city.

The way it does this is by parsing an API provided by OpenWeatherMap.

http://api.openweathermap.org/data/2.5/weather?q=Bristol

It gives us something like this:

{
  "coord": {
    "lon": -2.59665,
    "lat": 51.45523
  },
  "sys": {
    "country": "GB",
    "sunrise": 1368850387,
    "sunset": 1368907233
  },
  "weather": [
    {
      "id": 800,
      "main": "Clear",
      "description": "Sky is Clear",
      "icon": "01d"
    }
  ],
  "base": "global stations",
  "main": {
    "temp": 286.78,
    "pressure": 1008,
    "humidity": 81,
    "temp_min": 284.15,
    "temp_max": 289.35
  },
  "wind": {
    "speed": 2.6,
    "deg": 230,
    "var_beg": 200,
    "var_end": 270
  },
  "rain": {
    "3h": 0
  },
  "clouds": {
    "all": 0
  },
  "dt": 1368907010,
  "id": 2654675,
  "name": "Bristol",
  "cod": 200
}

If we didn’t know anything about the API dependency, we might write the following feature file:

Feature: Weather report
  As a visitor to Bristol
  I want a page that tells me what the weather will be
  So that I can decide what clothes to wear

  Scenario: Report says it's going to be clear
    When I look at the weather report for "Bristol"
    Then I should see the weather is "Sky is Clear"

We could use Capybara to test, and step definitions would look like this:

When /^I look at the weather report for "(.*?)"$/ do |city|
  visit "/weather/#{city}"
end

Then /^I should see the weather is "(.*?)"$/ do |expected_weather|
  page.should have_css '.description', text: expected_weather
end

I’m sure the problem here should be immediately obvious: the weather changes! So how do we allow for changing weather?

One way is to make the step definition more vague: “Then I should see the correct weather” .. the step definition would have to make a call to the same API, parse the JSON, and make the expectation based on the response it finds. If it comes up with the same answer as the app, we can assume the app is behaving as expected.

It’s is a viable solution, but i don’t think it’s the best.

Stub out the API

Ideally, i should be able to run the app locally. There are a number of good reasons for that anyway: i can debug the code, i am able to trace log files and get more meaningful error messages, it’s faster to run, and i can run it offline (once i have stubbed out the API dependencies!)

If i can’t run the app locally, it’s okay, but i have to be able to configure the app somehow to fetch data from a different source, one that is under my control. We did this a lot in the Olympics, using REST-assured to return responses to API requests. It was a good way to gain control over the ever-changing data.

But if i can run the app locally, i can reroute api.openweathermap.org to localhost, and then it’s mine to play with as i wish.

sudo vim /etc/hosts

Add a line:

127.0.0.1 api.openweathermap.org

Now my machine is going to reroute requests for api.openweathermap.org to localhost. Hurrah! Successfully intercepted!

And of course, my app, running locally, encounters the same thing.

Now i can put the data that i want there. Oh, there’s a catch: we have to run on port 80. But that’s okay, we can do that. Considering it means the app will be completely unaware of the change, i think that’s a small price to pay.

Introducing RackDoubles

RackDoubles allows you to create, change and remove endpoints at runtime.

Start off with a very simple config.ru file to run a Rack app that uses RackDoubles. You have to define a run method but it’s as simple as it could possibly be.

require 'rack_doubles'
use RackDoubles::Middleware

run lambda { |e|
  [404, {'Content-Type' => 'text/plain'}, ['Not found']]
}

Because we need it on port 80, we have to use sudo to run it, or in my case rvmsudo because i use and love Ruby Version Manager.

rvmsudo rackup -p 80

And just leave it running. The magic is about to happen.

Now, watch this. Here’s what we’re going for:

  Scenario: Report says it's going to be clear
    Given the weather API is stubbed to return "Bristol.json"
    When I look at the weather report for "Bristol"
    Then I should see the weather is "Sky is Clear"

I have saved the JSON i want into a file features/support/stubs/Bristol.json

So let’s use RackDoubles to set up the stub response:

def client
  RackDoubles::Client.new('http://127.0.0.1')
end

Given /^the weather API is stubbed to return "(.*?)"$/ do |filename|
  response = File.read(File.join(File.dirname(__FILE__),
                       '..', 'support', 'stubs', filename))
  client.stub('/data/2.5/weather')
  .to_return(200,
             {'Content-Type' => 'application/json'},
             response)
end

Now the data is entirely under my control. The test passes.

The best news is, now i have the opportunity to simulate bad data from the API. And did you spot the 200 status code there? I can easily write a step definition to stub the API to return a 503 Service Unavailable, to see how the app handles that.

  Scenario: API is unavailable
    Given the weather API is unavailable
    When I look at the weather report for "Bristol"
    Then I should see "Please try again later"

Here’s the step definition:

Given /^the weather API is unavailable$/ do
  client.stub('/data/2.5/weather')
  .to_return(503,
             {'Content-Type' => 'text/plain'},
             'Service unavailable')
end

Now if that test fails i have something to tell the developers about. I can easily demonstrate a scenario in which the API is unavailable. These are things that a manual tester cannot easily do. A developer might simulate them with unit tests, but i am able to test it in a reliable, repeatable way that may not be a full end-to-end test, but i think it’s a pretty good integration test.

Of course, there is still a need for full end-to-end test coverage, but i think that’s best covered by manual exploratory and smoke testing, which are still necessary and highly valuable. As Developers in Test, we have the skills to make life a bit easier for ourselves and our teams, so let’s use them.

Automating Cucumber test scripts on a Mac

I’m really enjoying my work at the BBC on the Olympics team! I’m primarily implementing Cucumber scenarios to verify the work that the developers are doing. I’m also using other tools for stress testing and performance monitoring.

My work here is leading me to use Cucumber in weird and wonderful ways! Here’s an example. Yesterday i was asked to run a script every half an hour that tests the Olympics video player to see if it’s playing successfully, and if not, to email people to say that it’s broken. Not your usual Cucumber scenario, but amazingly enough, it can be used to do that!

Forgive the jargon below. PID is a programme identifier, and IVP is the Interactive Video Player that we are building.

  Scenario: Check LIVE video player
    Given a live PID 
    When I check the IVP 
    Then I should email people if anything goes wrong

I can use a combination of javascript hooks and taking screenshots a few seconds apart and comparing them to tell whether the video is running. I decided to run some step definitions that i’ve already defined and rescue any errors to be emailed. There are a few more steps in here but i’ve stripped it down a bit for simplicity.

When /^I check the IVP$/ do                                                                         
  if @pid
    begin
      step 'I attempt to view the IVP host page with that PID'
      step 'the video should be ready'
      step 'the video should be playing'
    rescue Exception => e
      @error = e.message
    end
  end
end

Notice that there might not actually be a live PID at the moment. It feels very weird, to put conditional logic in Cucumber step definitions like this!

Now comes the email part. I used Pony and sendmail as a simple email sending mechanism.

Then /^I should email people if anything goes wrong$/ do
  if @pid && @error
    subject = "Assurance test failed in #{ENV['ENVIRONMENT']}!"
    body = "#{page.current_url} caused this error: #{@error}"
    recipients = ['send_to@me.co.uk']

    Pony.mail(:to => recipients,
              :subject => subject,
              :body => body,
              :via => :sendmail)
  end
end

So, with that working, i set about putting it on a schedule. I thought i could just call the script from a cron task, but it turns out that wouldn’t work because i need to launch Firefox with Selenium to test the flash player.

I found this cool built-in Mac tool called Automator. You can set up all sorts of tasks into a workflow. I just needed a simple shell script:

And now, we can call that from a cron job every 30 minutes! You might have to hunt around a bit for where it saves the file. For me it’s in my Library/Services.

*/30 * * * * automator /Users/daniea16/Library/Services/IVP\ Test.workflow

Another creative alternative to the cron job is to set up recurring calendar appointments in iCal. You can set an alarm that runs a script. Pretty awesome, hey?! :)

I know Cucumber is meant to be a BDD framework, and was never intended to be used for automated regression testing or assurance testing, but isn’t it cool that with a little bit of imagination you can bend it to fit these alternative uses? :)

aimee’s 2011 retrospective – a year in review

This is going to be a long post, all about me! feel free to skip it! :)

An Android app to send prayers to Jerusalem

I have just finished a two-week project building an Android app called @TheKotel – Prayers to Jerusalem.

Now, anyone who knows me even slightly might think this is a bit weird, and believe me, i had a long conversation with myself about it before deciding to take on the work. These are the reasons that i did.

1. I think it’s a nice idea. Putting thoughts into words and feeling that you’ve offloaded them somewhere is sometimes all people need to feel better.
2. I saw a video of Alon printing the prayers and cutting them up and taking them to the wall. It made a personal connection for me.
3. There’s already a successful iPhone app with over 10,000 downloads, so i saw this is definitely something that people want, which is all good publicity for me.
4. Alon raised money from the people who already use the iPhone app to pay for the development of the Android app.
5. I saw it as an opportunity to learn a bit more about Android development, try out PhoneGap and Sencha Touch (which i dropped when i found i wasn’t happy with the quality).
6. I had saved up enough money in the back that i could offer this more-or-less as a gift, not expecting much money for it but instead doing it for reputation and for another Android app in my portfolio.

The concept is very simple. You type your prayer on a piece of paper (or pray out loud with Android’s speech-to-text feature) and press “Amen” and the prayer gets sent to Alon in Israel.

Combined with all the prayers that get sent via twitter and the iPhone app, Alon prints out thousands at a time, cuts them into little slips, rolls them up tight and takes them to the Western-Wall, or The Kotel, the remains of the Second Temple in Jerusalem.

The app is free, and the service is totally free, but donations are gratefully accepted. There is also a Connect screen giving options to share the app with friends and feel connected to the Western-Wall and Jerusalem. We also included some background information about the Wall and how the service got started, as well as some personal messages from a few supporters who helped to raise funds.

Best of all, there’s the gallery where you can see where the prayers are going. I really like this image of Alon putting the prayers in the Wall.

I am very pleased with the final result. It’s nice and fast, good native Android experience, and i think people are going to love it! I enjoyed coming back to Android development, say what you want about Java, but i like it!

If you can bear to listen to me talk about this anymore, Alon asked me to record a video in my “enchanting” British accent, heheh! :)

Device graphics were generated with Device frame generator and released under CC BY 3.0 license.

Build and launch a web shop in a day

My day yesterday began at Winchester station at 05:19 in the morning, with a big ambition:

I arrived in Kent at 08:25 and was met by Alanna who owns Aphrodite’s Desire. We got straight to business. By 08:45 we were settled in with a cup of tea and signing up to Shopify.

Let me back-track a little. We had already had several email conversations and had met previously to discuss options, and decided that Shopify was the best solution. Alanna needed something that is easy to manage, looks good, and can handle orders and payments reliably.

Our first concern was that Alanna’s debit card might not be accepted as Shopify says it wants credit cards. According to two people i spoke to at Shopify, some debit cards work, but not all. Fortunately, Alanna’s Visa card was accepted.

I began by demonstrating how to add products. Alanna’s immediate reaction was, “Oh, it’s just like Facebook!” Alanna was already uploading product photos before i’d even pointed out the option! Huge props to Shopify! Fortunately, Alanna had done a lot of good preparation work writing up descriptions for the products and getting all the photos ready. This was a major advantage to launching in a day.

In the meantime i worked on installing and customising the theme that we’d previously chosen. I asked Alanna’s opinion on a few fonts and we chose some photos for the banner.

I installed the iPhone extension. Alanna had already downloaded the app, and it was cool to see products appearing on the iPhone. We were both impressed by Shopify’s handling of discounted products, and combinations of different variations, some being available and some out of stock. I was impressed at how easily i could extend the theme and add a message that invites people to contact Alanna to ask about unavailable items.

By lunch time we had a pretty good looking shop! We were admiring it on the iPad over lunch. We went out for a quick walk and fresh air before coming back like, “Right! we have a shop to launch! Let’s get to it!”

Adding the facebook and twitter integration was fun. You can ‘like’ products and add comments and it all gets synchronized to facebook. Alanna can click a button to promote products on twitter and facebook. It’s great! Shopify extension apps are awesome!

We spent a bit of time doing the PayPal integration, and then a lot of time signing up for Sage Pay. A lesson learnt: if you want to accept credit/debit cards directly on day 1, you must apply for your Sage Pay merchant account in advance. But it wasn’t so bad to launch only with PayPal.

We had a dilemma when it came to shipping options. Because Alanna hadn’t weighed any of the products, we were leaving them all as weighing nothing. But this meant we couldn’t calculate the shipping. Alanna wanted to charge £5 for a single product, and £1 extra for every additional product. So we came up with the idea of making every product weigh 1lb just so that we can count them in the cart.

That being done, we just needed to tidy up the additional pages and navigation links. I was keen for Alanna to do this, so as to be able to do it again in the future. Shopify provides a very nice content management system which is simple to use. I did a little bit of graphics work on images for Alanna to use.

I think it was about 17:00 when we launched! We spent about half an hour tweeting and facebooking about it like crazy … and then an order came in! Alanna was just looking at the iPhone app and said something like, “Oh, is something wrong here? It says …” and it was an order from one of my friends!

Fantastic! A working web shop in one day! It is totally possible! The shop had 171 unique visitors on its first day.

My day ended with this from my mentor:

I went to bed feeling very satisfied and utterly amazed at knowing i get paid to have a lot of fun and do what i really love to do. It was a brilliant day!

Please visit aphroditesdesire.com to have a look at what we did. Kisses to shopify.com for making it all possible!

Well, hello!

Hello to everybody who knows me from my two previous blogs: edendevelopment.co.uk/blogs/aimee and aimee.mychores.co.uk. I have decided to merge the two into one, which for convenience i’ve decided to have hosted on wordpress.com now.

Big news

In case anyone is not aware, let me share the news that eden development, my place of employment for nearly 3 years, closed its doors in March. For me this is sad news: eden was a truly remarkable place to work.

Through eden i met many amazing people, and learnt to hone the craft of developing excellent quality software. I have learnt how to listen to clients and understand their needs. I have been privileged to take on despo and Alberto as apprentices, imparting my knowledge and care and seeing them develop. Most importantly, i have grown in self confidence, to the point where i am now able to journey out on my own, taking on contracts and freelance work.

Contracting

My contracting has begun at a consultancy firm in London. I have been contracted as a front-end developer, doing HTML, CSS and a bit of Javascript. It is easy work for me, not really particularly challenging. I am happy with that: at the moment i have big challenges to do with commuting and getting used to being self-employed, keeping control of expenses, invoices and tax. I didn’t want the added pressure of difficult work on top of that.

It turns out that the job is suiting me very well. I get on well with the people i work with, and i found i was able to start providing value for the company from my first day. I’m working hard and i believe i am exceeding their expectations of me. At the same time i’m learning how to use Demandware, a powerful ecommerce platform, and i may well find that the skills i learn here could come in useful for me again at some point in the future.

Looking ahead

My contract takes me until the middle of May, at which point i’d like to spend one week working on a freelance project. After that i’d like to take on another contract, and this time i’m particularly looking for something that uses my Ruby skills, as well as my knowledge of behaviour driven development. There are two possibilities i’m looking at, and i’m fairly confident that one of them will work out.

I am always interested in meeting new people to talk about work that i may be able to help with. I am really enjoying the freedom of contracting, so i’m not looking for permanent employment at the moment, unless it’s a very good offer. I’ve realised that job security is a myth, and painful though it was to leave eden, i think i needed that push to venture out on my own.

Right now

At this very moment i am at Scottish Ruby Conference in Edinburgh. I really enjoyed it last year, and this year seems even better. I am meeting new people and rekindling previous friendships. Today i have been inspired to work on a charitable project, i enjoyed an entertaining discussion on programming etiquette by Jim Weirich and Chris Nelson, and i was excited to see how MacRuby works.

It’s lovely that my mentor Enrique is here at SRC, sharing an apartment with me, and my apprentices are both here too. Edenites Chris and tooky are also around, and it’s great to spend time together again. Our shared experience of being part of eden is something that will remain with us, and we will always be good friends.

Notice that i say “edenites”, not “ex-edenites”. For me, eden was all about the culture and the people. My pride for quality of work, my valuing of client relationships, my commitment to learning, my honesty, my humility, these are eden’s values that have become deeply ingrained in me, that i intend should never leave me.

Thank you

If you read this far, thank you. You can see why this post has been a long time coming. It’s only now that i’m at Scottish Ruby Conference that i can really take the time to relax and reflect. The last few weeks have been like a whirlwind for me. Thank you to all who have supported me and encouraged me. It has been scary and exciting, and i’m relishing these new opportunities!

Here’s to my ongoing journey! :)

My software apprenticeship

I have recently had the privilege of being accepted as an apprentice to software craftsman and philosopher Enrique Comba Riepenhausen. This is part of Eden Development adopting a craftsmanship approach to software creation.

I am very excited at this new development in my career. It’s going to be hard work because i have a lot to learn, but i am pleased to have made the commitment and to know that i have someone who is playing a very active part in my personal and professional development. Funnily enough, the more i learn, the more i realise how little i really know. The first task that i have been set as part of my apprenticeship is really showing me that.

Eden Development has given me a blog to talk about my progress during my apprenticeship. So i have decided that my apprenticeship blog will become the place that i talk about my technical and coding interests, leaving this blog to be more personal. So if you are interested in me from a technical point of view, please follow edendevelopment.co.uk/blogs/aimee.

There are actually now five apprentices at Eden Development. If you are interested in software craftsmanship it is well worth looking at the Eden staff blogs aggregation to read all our thoughts and experiences on our own journeys.

Glimmers of Hope

This beautiful photo was taken by @ecomba at Barton Farm this morning:

Quite appropriate, since the team have a big project to deliver today, which has been months in the making. I have worked on the project but i’ve been busy doing other things more recently.


Today i was safely out of the way pair programming with @edentodd at the Coffee House in Winchester:

We projected the code on the projector downstairs, it worked really well!

Coffee shop coding, ftw! :D

Distressing times for the Rails community

Things like this sadden me: Mike Gunderloy resigns as a Rails activist, no longer wishing to be a public spokesperson for Rails. This is a great loss for the community.

Things like this disappoint me: the attitudes of a few people cause a tainting on the entire community. Even if David Naughton has misunderstood the issue, it is clear that the whole debate has had a negative effect on the Rails community.

Unexpected pornography at a professional conference surprises me, shocks me a little. I wonder whether Matt Aimonetti, at any point during the preparation of that presentation, thought “This is likely to offend some people”, and if so, whether Matt decided not to care.

The refusal of some Rails representatives to even acknowledge that there is a problem angers me. Yes it was edgy. Yes it was creative. It can be those things and still be offensive. “Creative” is not a synonym for “acceptable”. Offending people is a big deal. The unpology “I’m sorry that you happened to be offended” is a world away from apologising for having been the cause of offence.

It is not a person’s fault for being offended. Without wishing to be a Bible-basher, there is a lot to be said for this little nugget of advice:

“If it is possible, as far as it depends on you, live at peace with everyone.”

You can’t please everybody all the time, but there are basic precautions that you can take, and when you have made a mistake, you can take responsibility for it and apologise for it.

I think that is the end of my rant, and i hope i will have nothing more to say about it.

* * *

I would like to say publicly, again, that i appreciate my job so very much. I am extremely lucky to work in a family-friendly, vibrant, fun, laid-back yet totally professional environment, doing a job that i enjoy with my whole heart. I don’t expect expect to be discriminated against, nor do i expect any special treatment. I receive precisely the appropriate amount of respect and admiration that i deserve based on my skill as a programmer. I am extremely grateful for it.

Here’s a picture of me loving my job today, along with some of my colleagues, sat on the picnic bench in the sunshine: yfrog.com/4o8a2j :)

* * *

Sort of related to the rant, i have been wondering tonight where my opinions come from. Different people have different opinions, and i can read and intellectually understand varying stances, so what is it that sways me so strongly in one particular direction? I have an opinion on almost any given topic, but i don’t always know what it is that makes me feel that way.

I find the various out-of-control tram hurtling down a track thought experiments quite interesting in terms of moral dilemmas to which people often have a strong opinion, yet cannot quite explain why. Ethics and morals are curious subjects.

What’s going on in aimee’s life?

I’m back! I have been away from the Internet for a week – a self-enforced leave of absence to try to avoid getting RSI. Happily to say, it seems to have worked – just typing as much as i need to at work, and having a rest in the evenings. I love programming and i love my job, so i really don’t want to injure myself.

I have been trying out another keyboard layout: Colemak. It has similar aims to Dvorak but, unlike Dvorak, they actually used a computer to help generate the layout! Also, unlike Dvorak, Colemak layout is fairly similar to Qwerty, possibly making it easier for Qwerty users to switch. It keeps Z, X, C and V in the same place, useful for the obvious keyboard shortcuts.

colemak.com – the website for the layout.
Colemak on a TypeMatrix EZ-Reach keyboard – made for my own reference, but i imagine others may come looking for it too. If you want it as a PDF, let me know.

Work has been fun lately – we’ve been trying out pair programming and i’m really enjoying it. I took a while to believe that it really is beneficial for two people to program together, but i’m coming round to the idea now. Especially when trouble-shooting – there is almost no break in the flow because the combination of two brains focussed on a problem means that we come up with the solution pretty quickly. Plus it’s also really good fun! :D I think the key is to have a separate pairing station set up with two keyboards and two mice plugged in. That gives a feeling of equality. With a little perseverance it soon starts to feel very natural to program together.

Social life has been fun too. The Spring weather is making me very happy. Church has been great, really enjoying it. In my week-of-no-internet i became totally hooked on Life On Mars – can’t wait to see the second series!

I leave you with something that i saw in a bookshop in Southampton yesterday – made me LOL ;-)