Specflow, Selenium and TeamCity - Screenshots in TeamCity results

Trying to replicate automation test can be a real pain. Especially for those flaky test that fail from time to time.

As a result my team started to investigate how we could see what the test actually did while running.

Solution 1 - BrowserStack Automate
We started looking at BrowserStack Automate. The tool is impressive and integrates using the "RemoteWebDriver". You can then log into BrowserStack and watch videos of the test runs.

This was awesome but had some draw back. Specifically speed was an issue. Running the remote browser even in Parallel added about a 3rd to our run time. We run our automation test in TeamCity having yet another tool to monitor results was going to be tedious.

I recommend setting this up if only for the multiple browser support. https://www.browserstack.com/automate/specflow

Solution 2 - WebDriver Screenshot
We then decided to start taking screenshots of the state throughout the journey of the test. You can easily use the WebDriver to take a screenshots. We then would place this image on the Team City Agent.

This worked fast but still accessing the files and associating them to the test was not easy.

Solution 3 - TeamCity static content
Finally we have found a solution that worked for us. Making some tweaks to the screenshot solution we started to upload our images to S3. This meant we had friendly URLs we could spit out in our output logs.

Then using some hacky code we processed the logs and pulled the image content into the TeamCity stacktrace. Each image from the logs is pulled in order and appended to a list showing the journey as per our interactions. This lets us quickly build a view of what happened and spot obvious mistake.

Below is an example of how the screenshot is added to the page. The image shows on the expanded test failure.

How to : https://gist.github.com/tharris29/8c9b4aca6ae0b2bc2e3ce1b3ac406314

Excuse the hacky code and html. I will tidy it up one day. Fill free to improve it.

Functional Programming and a journey to F#

Hopefully this blog will help anyone considering functional programming understand some routes to skill up. This is in fact a brief history of my year journey to Functional Programming and F#.

My first project

During a company hackathon I wanted to see if we could address customer queries with a set of key responses. The idea was to analyse customer queries and respond appropriately. The key behind this was to do sentiment analysis on the text of the query.

I approached this in a way of categorising words and terms. I was able to do this fairly easily by using Stanford Universtity NLP and a training set created by me. In addition to this I had to correct spelling and learn new slang or alternative words this was possible using NHunspell, The end result was a flow similar to this.

This was a fairly linear process until the validation of sentiment begins. When you begin to look at sentiment you have lots of variations (Sad, Anger, Happy, Fun). Working in imperative code at this point I ended up with lots of interfaces and different implementations of validation. The code started to get messy and complicated I thought there must be a better way?

By the way I won the hackathon ;)

Scala

After some research on my project I found that many developers were using Functional programming to achieve machine learning projects. To give myself an introduction in 2014 I attended a course at City University in London focused on Functional Programming and Scala. Whilst this course was well priced and  and informative for a beginner as a full time developer I found the pace a little slow. That said it was still enough to interest me and continue me research.

Coursera 
To recall the reason for my interest in functional programming was to solve machine learning problems. During this journey I took a free course on Coursera run by Stanford university.

Challenges

Sadly the journey of learning a new language or in this case paradigm isn't always easy. While languages can create interest being able to apply them in your regular career can be a challenge. Working for a .NET house the introduction of Scala would always be hard also from experience introducing new languages can often result in poor code bases for all.

F#

Never one to give up along comes F# a .NET language that lets me build on skills I already have and in addition it interops seamlessly with my C# code. This allows me to start writing code in F# without starting new projects.

I read "Functional Programming for the Real World"  which I highly recommend. The language used in this book makes it easy to understand coming from a C# background.

This book taught me the basics but in the last month I was lucky enough to get the opportunity to take part in Tomas Petricek and Phil Trelford's  Fast track to FSharp. This has really concreted my understand with lots of real world style applications and examples. The course is expensive but if you are a .NET developer and want to introduce functional programming it is worth it.

Journey Continues

My journey continues with F# as I start to attempt to introduce it in Testing and Domain modelling first.

But for some of you the journey might not have ended at F# and you might want to consider other languages for your functional needs. if that's the case I highly recommend investigating Haskell and Erlang.

Developing Universal Windows Platform (UWP) apps

Why now?
//Build/ 2015 peaked my interest in the Windows app platform, something that had never happened before. This all comes down to the Universal Windows Platform. If you are unfamiliar with the Windows 10 UWP I suggest checking out. Below is a link to a brief article which explains some of the benefits. In short one development lifecycle to create an application for all Windows 10 platforms.

Guide to Universal Windows Platform (UWP) apps

I thought I would have a dabble at the developer experience and this is the result. Think Tinder for cats. Using the thecatapi.com API I was able to creating a simple swipe and vote app for random cats.

What made it so easy?

I opted for the C# environment being my usual language. Firstly that brought all my current experience of .NET with me. In addition sadly in the past I did some Silverlight development and become familiar with XAML. With some help from the abundance of sample apps released by Microsoft for the Windows 10 SDK I was up and running in moments.

Down side

Being on the bleeding edge of development means sometimes you get cut. Sadly after a few hours of putting my app together I wanted to share the fun. At this point my good experience ended as I struggled to figure out how to submit my app. I finally read the statement on the guide "The Store is not currently open for the submission of Windows 10 apps." boooo!.

Whats next?

• Wait to be able to publish the app.
• Explore the Device contracts what is possible in each platform.
• Have fun making some more silly apps.
• Create a more complete tutorial of developing UWP apps.

Waltham Forest Tech Meetup

Recently I moved house all the way to sunny Zone 4 in London/Essex (Waltham Forest is still London). While I love the suburbs of London I found it increasingly difficult to juggle getting home and attending technical Meetups. Missing out on maintaining my skills by putting myself in challenging environments with other tech minded people. 

Rather than accepting my fate as a legacy developer in the leafy suburbs,  I have decided that I would start my own Meetup. I cannot be the only one after all.

Granted this all sounds a little dramatic for Zone 4. Especially when I'm talking about areas like Walthamstow, the new hipster location, move over Shoreditch... I cannot be the only techie feeling like this.

With out further adae I present to you Waltham Forest Tech. 

Waltham Forest Tech Meetup

First and foremost this is going to be friendly environment to share some knowledge and hopefully gain some. My Walthamstow utopia is where everyone understands software engineering and with that I promote that every passionate person comes along.

I'm looking to build a core group of people that are driven by learning or mentoring and somewhere you can come ask questions. In addition to this somewhere you can have a good beer :). Later with aspirations of host  hackathon and presentations where people are interested to do so.

So if you are local to Waltham Forest and want to meet likeminded people, develop skills and network please sign up and come along.

Sorry but no recruiters ;) 

Testing WebApi with Owin Hosting

Lately I have been working with the OWIN specification; specifically OWIN hosting, known as Katana, which is Microsoft's implementation of the OWIN specification. During this time I have come across some tools that have helped me create my project. One in particular stood out so I thought I would share.

Brief introduction to Owin hosting

Here is a brief example of OWIN hosting Startup class initializing WebApi, full solution here.

using Owin;
using System.Web.Http;

namespace WebApi
{
    public class Startup
    {
        public void Configuration(IAppBuilder builder)
        {
            HttpConfiguration config = new HttpConfiguration();
            config.Routes.MapHttpRoute("Default", "{controller}/{customerID}", new { controller = "Customer", customerID = RouteParameter.Optional });

            builder.UseWebApi(config);
        }
    }
}

This is the entry point to your API, regardless of being hosted in IIS, Windows Service or even a command window. This is the "first" part of  the code that will be touched in the solution, meaning it's the first place you could break your site unless you have some test covering it.

To learn more about OWIN Katana, see these examples http://www.asp.net/aspnet/samples/owin-katana.

Why is this important?

This pattern of hosting is the future of ASP.net. This is proven with the latest version of vNext using this method.

Testing boundaries

The current trends on testing seem to be in a state of flux. Many people have different views on what the best way of testing is, an argument I'm not trying to take part in here. However, one of these views is to test from the boundaries, not unit testing. In our API's case, the boundaries can be considered as the Startup.cs and likely a database or some external resource. These are the first and last entry points of our solution.

If you want to know more about boundaries, check out Ian Cooper's presentation on Hexagonal Architecture, I saw it at DDD East Anglia and it clarifies a few points of confusion.

Why does the OWIN host make this easy?

One of the NuGets that has come out with the introduction of OWIN host is the Microsoft.Owin.Testing library. With this library we have the ability to call our API from the outside, from the first boundary point of our code.

When using the Mircosoft.Owin.Testing library, the tests are started in an in-memory host. Any calls to the host will now be routed via your "Startup.Configuration" which in our example will pass the calls to the WebApi framework.

This means no setup of a host (IIS, Windows Service...) before you run your test. The tests are completely contained.

Creating a test

1. Create your API using Owin Hosting, as shown earlier
2. Create a Unit Test project,
3. Add a reference to your API project. This allows us to reference the Startup class.
4. Add the "Microsoft.Owin.Testing" NuGet
5. Add the following code in your test. Changing the URL to your API url.

var server = TestServer.Create<Startup>();

var response = server.HttpClient.GetAsync("/hello/world").Result;
var result = response.Content.ReadAsAsync<IEnumerable<string>>().Result;

Assert.AreEqual("helloworld", result.First());

Now you can run your test knowing that all your routing and configuration is correct, without setting up any IIS or Service to host your solution.

For some other example see https://github.com/justeat/openrasta-hosting-owin/tree/master/src/OpenRasta.Owin.Test

Not just WebApi

This doesn't just work with WebApi! Any framework that has an OWIN hosting can run within the Microsoft.Owin.Testing in-memory host, here are a couple of frameworks that now allow this.

• WebApi
• Nancy
• OpenRasta - created by me :)
• FubuMvc - I'm not familiar with this but it exists

Conclusion

Whether you want to do integration testing, boundary testing or feature testing I believe that you can use the OWIN in-memory host to achieve this. Removing any prior configuration of a server before running the test also allows for easier collaboration on different machines. With Microsoft following the same patterns in vNext it seems like a promising road map to be on.

**UPDATE**

Since releasing this post I have now been working with JustFakeIt. This is a nice wrapper around a in process host using similar methods as above. The benefit is that we can use it like many mock frameworks with expected results and ignoring parameters. Check it out https://github.com/justeat/JustFakeIt.

Finally moved to the cloud

I have been with GoDaddy now for a while but have noticed as I progress with my site and my Mini Projects become more complicated(heavy) that GoDaddy was getting harder to manage and struggling with the loads. I cannot really complain I bought this hosting package for peanuts.

Lucky for me I have a MSDN subscription, with that comes £35 of free Azure credit per month... It was time to move the site. I kept putting it off, assuming it would take a while and I would struggle with the GoDaddy DNS, but thanks to YouTube I found a simple walk through and an amazing bit of background music :).

How To Forward a GoDaddy Domain name to Windows Azure Website

In the end it took all of about 20 minutes including some reading to get up and running.

Just watch out when adding you CNAMEs, you will need to add "awverify", "awverify.www" and change "www", it was just a little hard to see in the video due to resolution.

So now when you visit tomharris.net you are in the cloud.

Agile development - Remote/Distributed teams

Applying Agile methodologies at the best of times can be difficult, getting all the business to buy into the concept is a tough sell. I see the key strength of Agile as regular communications via demos, scrums, kanban… all of these are about communicating progress, however what do you do when you teams are spread globally or on customer sites. In this situation the Kanban whiteboard and post-its don’t work.
I currently work in a company where the development team is spread across multiple regions, from Vietnam to the UK and many stops in-between, people work from home on a regular basis, all this makes communication hard. To overcome this we use a couple of techniques to address the gap and make it feel like we are all in one office.

Daily Scrum

This goes without saying but we use Skype/Lync both these tools allow for our whole team to do a stand-up(seat down with headsets). Ideally getting people to have the video camera on where possible, keep the call short and sweet.

Digital Team Board

Dashboards

We are a Microsoft house traditionally so we opted TFS as our project tracking system. The benefit of TFS 2012 is the introduction of the Web access and dashboard. This new web access brings a wealth of tools with it which in most of the case just don’t exists in the Visual Studio version.

Team Capacity

In TFS dashboard you can also configure and view you Team capacity Vs Requirements. This is a really powerful live view of each individuals ability to deliver, allowing for you to take into consideration holidays per sprint/iteration.

Digital Kanban

In TFS you also have several Kanban style dashboards to allow you to share progress at a multiple of levels.

1. Work>Board>Requirements
2. Work>Board>Team
3. Work>Backlog>Board

All are pretty similar as below but obviously depict different information. One great bit of the Backlog board is the ability to customize the columns, this allows for you to add more steps to your process very easily.

The key is to share these boards during you daily calls with the team, and share them outside the team. The point is to replicate this as a white board on the wall.

Live Burn down

Using the information gathered about the requirements and the capacity TFS is also capable to provide a burn down, this is great to share to the non-technical teams the estimates completion date.
A Burndown is rarely a perfectly linear line, in all processes you will have times where development is more productive than others. Be careful when sharing this that people do not get scared that you are behind or add work because you are in front, the line is a prediction and not fact.

Custom Counters

On the home page of TFS 2012 you also have the ability to configure TFS Query counters, ideal for showing bug counts etc. In addition you can also add on this screen the Build status, for continuous integration this is key as it will show the whole team that the builds broke before they go home for the day.

Alternatives to TFS Dashboard

You have plenty of add-ons for TFS which can achieve similar and more, below is a list but I have not tested them all.
http://scrumdashboard.codeplex.com/
http://www.telerik.com/teampulse/tfs
If you do not use TFS you do have alternatives options still with Team-City for example you can use
http://www.sonarqube.org/

But where is my whiteboard

So, all the above is great but how do we make sure people look at it? “When we was not digital we could ensure that the team could see it everyday buy having it on the whiteboard in the dev area” is the question I can hear. The simple answer is you cannot make sure people look at it unless you keep a board, but lets have a digital board.

Display

We opted for a 42inch screen in each location local and remote(obviously we could not enforce this on home works). We also had to have something to run our display so we repurposed an old laptop, most companies have one, it really doesn’t have to be special.

Sharing content

Now we had our screen we needed some content, I wanted a scrolling screen showing everything from burndowns, capacities, company twitter… The main goal is to make this something worth looking at daily.
Things I have on our screen:

• Company Twitter feed
• Each Project Dashboard
• Burndown chart (Note for this you need to access the tfs team chart api, to do this press F12 click on the chart and it will have an image URL, this is your api).
• Team Capacities
• Popular Blogs
• Microsoft Training sessions
• Company Logo
• Team Social Pictures

One of the key things is to pick content that will change naturally, we need to make the screen engaging.

Adding live web pages (TFS Dashboards)

To achieve this I come across a tool called LiveWeb which is a addin for PowerPoint, you can follow the guide to setting up a presentation containing web pages here. The key benefit that LiveWeb brings is the fact your web pages will refresh each time you load a slide, showing only the latest information.

Looping you PowerPoint

In PowerPoint 2013 you have the ability to loop your slide show based on timings, as below.

• Click on “Set up Slide Show”

• Select “Loop Continously until ‘Esc’” and “Using Timings, if present”

• For each slide set the timings, you do this by click on “Start Recording from Current Slide…”, you then need to wait for the time you want for this slide and then press esc. Do this for every slide until all of them have a timing associated, this process is better for adding new slides as you can change the timing for the current slide only and not all.

• Press F5 and away you go.

You also have the option to Broadcast this presentation, with this option you can potentially let other remote location view the exact same presentation, although I never tried this.
More to come as we continue on our journey of multiple location development…..