Sunday, February 14, 2010

Integrating SLIM+Fitnesse with CruiseControl

On the current project we use SLIM framework within FitNesse for Acceptance testing. We rely quite heavily on the Fitnesse tests results to certify our builds. Hence it was important for us to automate running our SLIM tests and to properly publish the result in the cruise page.
I use the ant scripts to execute Fitnesse test runner from a command-line, parse the resulting XML and transform it using style sheet and finally write the results to the xml files in the cruise log directory. Сonsequently the CruiseControl dashboard with the results looks like here


To have such view each suite should have its own xml log in an appropriate format. I do this just running each suite separately. For this purpose I use the names fitnesse command to get a list of the suites and then run them using the suite Fitnesse command, so it is pretty easy to save the results in the different files. If I don't need to run the whole bunch of the suites (for example: only suites of a particular virtual machine) I use the suiteFilter parameter of the suite command.



As you see the 'exec fit suite' target is a procedure that runs fitnesse against the given suite and store it's output into the 'suite_result' property. After that there is a small trick. Unfortunately, the output contains not only test's result in the xml format, but also the fitnesse command output
FitNesse (v20100103) Started...
 port:              9125
 root page:         fitnesse.wiki.FileSystemPage at C:\FitNesse/FitNesseRoot
 logger:            none
 authenticator:     fitnesse.authentication.PromiscuousAuthenticator
...
Exit-Code: 0
So, it is necessary to filter it and here only strings between <testResults> and </testResults> are written to the logs.
As I already have said, the suiteFilter parameter of the suite command is used to run only particular suites. In  this case fitnesse tries to execute each suite but if it does not fit for the given criterion it's tests do not start but we have the xml output as well. Such output does not have relativePageName tag, so in the 'save log as xml ' I just check it and write xml logs only for suites that were really ran.

Pay attention to the 'xslt' ant's task in the 'save log as xml'target. It is used here to write fitnesse test's results in the particular format by processing they via the XSLT.

That's it for today. Feel free to ask me questions.

5 comments:

  1. The XLST list points to a GoogleDocs link, but doesn't display anything

    ReplyDelete
  2. Hi Kenneth! Do you mean that the 'https://docs.google.com/uc?id=0B5mFzeoGZ2CQZDUyOTBkMGMtOGM0Mi00MTJjLTg1NGUtMDMyMTI5Y2Y4YTkx&export=download&hl=en' link is broken? Actually I have gone to the link and downloaded fit.xml document. It seems all right. If you still have a problem write me to andrey.malkov@gmail.com and I will send it for you. Best regards.

    ReplyDelete
  3. Hello Andrey, and thank you.

    Yes, the link:
    https://docs.google.com/uc?id=0B5mFzeoGZ2CQZDUyOTBkMGMtOGM0Mi00MTJjLTg1NGUtMDMyMTI5Y2Y4YTkx&export=download&hl=en
    does not work. I just tried it again. It's just goes to *my* Google Docs files, not yours.

    I suspect that the reason it works for you is because you are logged on to your account. I bet if you logged out of Google and Blogger, and tried the link again, you'll see the same problem.

    I think you need to somehow make that document public or publish it, etc.

    ReplyDelete
  4. Hi Andrey,

    I've been looking at implementing something similar but have been having problems using ant parallel tag. Have you managed to use this tag?

    My problem is when ant comes to the following


























    ---

    For some reason the longer test always has it's socket closed. Giving me the following output

    HTTP/1.1 200 OK
    Connection: close
    Server: FitNesse-v20110104
    Content-Type: text/xml

    java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(Unknown Source)
    at java.net.SocketInputStream.read(Unknown Source)
    at util.StreamReader$2.read(StreamReader.java:169)
    at util.StreamReader.readUntilFinished(StreamReader.java:106)
    at util.StreamReader.preformRead(StreamReader.java:95)
    at util.StreamReader.readBytes(StreamReader.java:53)
    at util.StreamReader.read(StreamReader.java:46)
    at fitnesse.slim.SlimClient.invokeAndGetResponse(SlimClient.java:71)
    at fitnesse.responders.run.slimResponder.SlimTestSystem.processTablesAndGetHtml(SlimTestSystem.java:278)
    at fitnesse.responders.run.slimResponder.SlimTestSystem.processAllTablesOnPage(SlimTestSystem.java:263)
    at fitnesse.responders.run.slimResponder.SlimTestSystem.runTestsAndGenerateHtml(SlimTestSystem.java:207)
    at fitnesse.responders.run.MultipleTestsRunner.executeTestSystemPages(MultipleTestsRunner.java:126)
    at fitnesse.responders.run.MultipleTestsRunner.startTestSystemAndExecutePages(MultipleTestsRunner.java:106)
    at fitnesse.responders.run.MultipleTestsRunner.executePagesInTestSystem(MultipleTestsRunner.java:92)
    at fitnesse.responders.run.MultipleTestsRunner.internalExecuteTestPages(MultipleTestsRunner.java:83)
    at fitnesse.responders.run.MultipleTestsRunner.executeTestPages(MultipleTestsRunner.java:56)
    at fitnesse.responders.run.TestResponder.performExecution(TestResponder.java:141)
    at fitnesse.responders.run.TestResponder.doSending(TestResponder.java:45)
    at fitnesse.responders.ChunkingResponder.startSending(ChunkingResponder.java:67)
    at fitnesse.responders.ChunkingResponder.access$000(ChunkingResponder.java:17)
    at fitnesse.responders.ChunkingResponder$RespondingRunnable.run(ChunkingResponder.java:106)
    at java.lang.Thread.run(Unknown Source)

    -----

    Have you seen this problem before or know a way around it?

    ReplyDelete
  5. reposted .xml with & for brackets

    &parallel&
    &sequential&
    &java jar="fitnesse.jar" failonerror="true" fork="true" output="ApiOnly.CanRegisterUser.xml"&
    &arg value="-c"/&
    &arg value="${PATH}ApiOnly.CanRegisterUser?test&format=xml"/&
    &arg value="-d"/&
    &arg value="$(DIR}"/&
    &arg value="-p"/&
    &arg value="8080"/&
    &/java&
    &/sequential&
    &sequential&
    &sleep seconds="1"/&
    &java jar="fitnesse.jar" failonerror="true" fork="true" output="ApiOnly.CanRegisterUserDuplicateAccounts.xml"&
    &arg value="-c"/&
    &arg value="${PATH}ApiOnly.CanRegisterUserDuplicateAccounts?test&format=xml"/&
    &arg value="-d"/&
    &arg value="$(DIR}"/&
    &arg value="-p"/&
    &arg value="9090"/&
    &/java&
    &/sequential&
    &/parallel&

    ReplyDelete