„Ah, but I may as well try and catch the wind.“ (Donovan)
GUI-based testautomation (hopefully done with Selenium WebDriver) is programming. No, it’s even harder than common programming, because you have to cope with insane effects. Even if your test suite works fine locally and even if it works for a while on the server, it’s no guarantee that it works reliable. That’s because the AUT is a living thing. Everyone trying to testautomate for few weeks knows what I mean. Sometimes the performance of the AUT’s server is bad and fails your beautiful testcases without reason. Sometimes the AUTs developers feel that they have to change the IDs and Bang! Sometimes heaven decides to change something in the layout and your f*** testcase waits for a f*** link that isn’t visible anymore (without scrolling) because the f*** floating menu decided to shift 50 pixels. Sometimes this, sometimes that. Ok, one could live with it: being called a bungler, analyse, fix and getting more and more stable with every new release. But things aren’t that „easy“: you got a ticket from the defect manager that your testcase works incorrectly (proven with a screenshot). You are running the test suite again and again without beeing able to even reproduce the bug. This type of bugs is called a heisenbug and unfortunately that’s not a rare case in testautomation. A bug in your test suite that comes in let’s say 7 of 100 runs and happens non-deterministic. Without being able to reproduce it, of course you can’t verify if any of your fixes works. Dead end!
But there is a way to „reproduce“ the bug: use the bulk! Run your test-suite 100 times and count the number of appearances of that bug (in above case = 7). Try a fix in your code and rerun the test-suite 100 times. If the appearance of that bug is 0 you managed to fix it.
You can implement the bulk in various ways – my favorite is Jenkins:
0.) Given you’re running your nice testsuite in a Jenkins Job yet and called it „RottenTest“
1.) Create 100 jobs and run them in parallel.
Execute in jenkins script console:

def jobName = „RottenTest“
def job = Jenkins.instance.getItem(jobName) //get a reference to the job containing the heisenbug
def i = 1
while (i < 101) { def newJobName = "CatchingTheWind" + jobName + i def newJob = Hudson.instance.copy(job, newJobName) //create new jobs to get the execute in parallel for shortest possible total execution time newJob.scheduleBuild(new hudson.model.Cause.UserIdCause()) //start the new job i++ } [/sourcecode] 2.) Create a listview to directly compare the results of your 100 runs. Filter it with the following Regex: [sourcecode language="text"] CatchingTheWind.* [/sourcecode] 3.) Delete the 100 jobs after successful bugfix: [sourcecode language="text"] def jobName = "RottenTest" def job = Jenkins.instance.getItem(jobName) //get a reference to the job containing the heisenbug def i = 1 while (i < 101) { def newJobName = "CatchingTheWind" + jobName + i newJob = Jenkins.instance.getItem(newJobName) newJob.delete() i++ } [/sourcecode] Thanks to Karen, the unknown hero, for helping me with the quirks of Jenkins API.
BTW:
Run the complete test-suite because you can’t anticipate all dependencies of your buggy testcase.
The night is your friend.

The myth of the efficiency of colocation in IT was created by software engineers from the First World to protect their wage level against programmers from India, China, Brasil, …
To protect the myth against critical reviews, they injected it into the sacrosanct frameworks of Scrum, XP …
Ironically the evangelists of colocation love Selenium, *the* framework for browser automation. But the Selenium project has no location – committers, contributers and supporters are spread all over the world and they colaborate in a chatroom based on antique technologies.
Let’s hope that the wage level in the developing countries rises soon …
… it’s just polemics, don’t ask me for evidence.

Working on a large project (~200 web developers) with Robot Framework (RF) for ~3 month, I consider the notion of „framework“ in Robot Framework a bit misleading. Robot Framework is more than a framework, it’s also a programming language. In this article I don’t want to bash Robot Framework as a testautomation language, even if I don’t feel comfortable with its strange syntax. I want to show that Robot Framework is a programming language and draw some conclusions.
RF has its own syntax
RF has variables (with different scopes)
You create them like this:

${hi} = 	Set Variable 	Hello, world!
${hi} = 	Set Test Variable 	Hello, world!
${hi} = 	Set Suite Variable 	Hello, world!
${hi} = 	Set Global Variable 	Hello, world!

BuiltIn-Library Documentation
RF has functions
In RF they are called keywords.
You create them like this:

Return One Value 	[Arguments] 	${arg}
	Do Something 	${arg}
	${value} = 	Get Some Value
	[Return] 	${value}

User Guide
RF has conditional statements
It looks like this:

${result} = 	Run Keyword If 	${rc} == 0 	Zero return value
... 	ELSE IF 	0 < ${rc} < 42 	Normal return value
... 	ELSE IF 	${rc} < 0 	Negative return value 	${rc} 	arg2
... 	ELSE 	Abnormal return value 	${rc}

BuiltIn-Library Documentation
RF has loops
And they look like that:

Run my hobbies
    :FOR 	${index} 	IN RANGE 	1 	11
    \    Watch TV
    \    Plague your neighbor
    \    Play with your dog

User Guide
RF has its own IDE
It’s called RIDE and it has some bugs.
RF has its own libraries
Look e.g. in the Selenium2Library
You’ll see, that the naming has nothing in common with what you’re used to from the WebDriver API.
… and RF has its own gotchas and bugs
Because the testcase-timeout produces a flaky bug in the reporting engine, I had to implement a timeout on my own … loop … sleep … boilerplate …
Conclusion

• Test automation is programming. Robot Framework is a programming language. And you have to spend time on learning it.
• I haven’t found any advantages over Java/TestNG yet (after 3 month working full-time with it).
• Think of the Java web-devs in your project/scrum team. Do they want to read/maintain test scripts in a foreign programming language?
• The skill-combination of Robot-Framework and browser-automation is seldom (In XING (leeding business network platform in Germany) you find the skill-combination „Selenium AND Robot Framework“ with 12 people. But the skill-combination „Selenium AND Java“ you find at 1000 people.
• So think of stuffing your project: it is easier to find a pro with Java skills than with Robot Framework skills.
• So think of the size of the community for support, advancing the tools, articles, …

If you are an adventurer, try it with Robot Framework.
If you like to install Robot Framework on .NET, find an instruction here
For more information on Robot Framework read: https://it-kosmopolit.de/strategic-link-collection-of-robot-framework/