Follow-up to Building a Test Script Environment with C# and IronRuby, pt 1, wherein we change emphasis from coding tests in Ruby to writing them in English (Gherkin) with the help of SpecFlow.

After that October post, I managed to construct a full system/regression test suite for a key data-management application, using the combination of Ruby and C# as described. My team has been able to go through several cycles of…

Oops, there is something amiss with that: the new test file should have been added to the system automation test suite at the outset, not at the end. That is, "test-first" should have applied at the system level, in addition to the unit level. But that's not the point of this article. The point is this: at the end, I did not have a product that a business user could easily review and correct.

Arguably, a non-programmer could read the code more easily, thanks to the use of Domain Specific Language concepts. Nevertheless, there was a lot of setup code that obscured intent, and the tests frankly still looked daunting to a non-programmer. Compounding the difficulty is the fact that the non-programmer users are still trying to understand the fundamentals of software testing, in plain English, and without the benefit of meaningful time to study. In other words, asking someone else to verify and maintain these tests would be throwing them into the deep end with inadequate support. The inspect-and-adapt mentality suggests modifying the system testing approach.

Gherkin to the rescue! The Gherkin language provides a very simple format for writing tests in plain English. A programmer then writes code that executes the intended actions, in the context of an automation system. Initially I continued on the Ruby path by trying to install RSpec as the interpreter, but I could not get it to install properly in IronRuby. Undismayed, I revised my goal: no longer would I seek to convert a tester into a Ruby programmer. Instead, I would encourage that tester to write and review English-language test scenarios in Gherkin, which the programming team would then support with code in C# using the SpecFlow framework.

At that point, we were working on a line-of-business application. Whereas learning the rudiments of Ruby, and setting up the code environment, had taken a good deal of effort, I was up-and-running with meaningful tests in SpecFlow within a day. More importantly, the other members of my team were immediately able to utilize and even correct small mistakes in these test scenarios. The learning curve was negligible when presented with a working example. And at long last, we were able to present a file containing test descriptions to a business analyst and get direct feedback.

The IronRuby experiment was not a failure: it delivered a meaningful result. Empirically, however, it was a less-than-satisfying result. We evaluated, re-prioritized, and moved-on to the next iteration of our experimentation: SpecFlow. Now we feel that we have the foundation for real organizational growth around software testing.

1. There’s never enough time partly because testing takes too much time.
2. The repetitive nature of manual testing makes it (a) boring, and (b) easy to mess up.
3. Because of both of these factors, I am loathe to delegate much testing to others. On the one hand I feel bad about pushing things on others just because I don’t want to do them; on the other hand, I have trouble trusting results that I did not generate myself.
4. There is an active interest from others to be more involved in testing; bringing them in will free me up for more interesting projects, and, objectively speaking, provide a good check-and-balance.

Automation can help with all of these factors, and more. Having come to that conclusion, what are the goals for this project?

• Time is still hard-to-come by, so as with any agile project, build only what is needed right now and then start using it. This argues against a fancy UI for now, and in favor of simple test scripts.
• Build components for auto-loading configuration and business data, as needed, into the database. These components should themselves be well-tested, and do not need to be created in the language used for the test scripts.
• Provide a simple and familiar mechanism for saving that configuration data, e.g. csv via Excel.
• These are "business-facing" tests, and they should be readable, with minimal training, by business users and non-programming testers. This argues for creation of a domain-specific language, use of a scripting language that that does not stand on ceremony, and possibly introduction of Behavior Driven Development (BDD) tools.
• Maintain all scripts in source control.

These goals rule out the normal .Net languages. A dynamic scripting language is more appropriate. From the many out there, Ruby seems to be the most obvious choice. With the help of IronRuby, the data-access components can be written (and fully unit-tested) in C#, while scripts are written in Ruby. RSpec might eventually be used for BDD. Follow-up posts will dig into some of the implementation details, as well as the lessons learned along the way.

Solution: careful use of jQuery. There are probably many different solutions already out there, but I wanted to learn how to write one for myself. Key functions: closest, detach, append, prev, next, before, and after.

Table Source Code

A couple of very basic tables, with "action" links in the third column. Judicious use of id and class attributes will turn out to be very helpful…

<table id="table1" style="background-color: Lime" class="displayTable">
      <caption>Table 1</caption>
      <thead>
          <tr>
              <th>One</th>
              <th>Two</th>
              <th>Three</th>
          </tr>
      </thead>
      <tbody>
          <tr id="row1">
              <td>R1 C1</td>
              <td>R1 C2</td>
              <td>
                  <a href="#" id="row1Link" class="rowLink">Move Me</a> |
                  <a href="#" id="row1Up" class="rowUp">Up</a> | 
                  <a href="#" id="row1Down" class="rowDown">Down</a>
              </td>
          </tr>
          <tr id="row2">
              <td>R2 C1</td>
              <td>R2 C2</td>
              <td>
                  <a href="#" id="row2Link" class="rowLink">Move Me</a> |
                  <a href="#" id="row2Up" class="rowUp">Up</a> | 
                  <a href="#" id="row2Down" class="rowDown">Down</a>
              </td>
          </tr>
          <tr id="row3">
              <td>R3 C1</td>
              <td>R3 C2</td>
              <td>
                  <a href="#" id="row3Link" class="rowLink">Move Me</a> |
                  <a href="#" id="row3Up" class="rowUp">Up</a> | 
                  <a href="#" id="row3Down" class="rowDown">Down</a>
              </td>
          </tr>
      </tbody>
  </table>
  <table id="table2" style="background-color: Yellow; margin-top: 30px;" class="displayTable">
      <caption>Table 2</caption>
      <thead>
          <tr>
              <th>One</th>
              <th>Two</th>
              <th>Three</th>
          </tr>
      </thead>
      <tbody>
      </tbody>
  </table>

Between Tables

One principle is that I don't want to hard-code all of the ID values. But that didn't work out entirely. Write a function and apply it to the click event of all of the elements with class "rowLink". In that function, find the parent row. Then find that row's parent table. Remove the row from the display – but use detach instead of remove so that the HTML still exists. See if that table is table1. There's the hard coding. If so, append the row into table2. Then it seems nice to help the reader notice that the row has moved.

// Setup the "Move Me" links
$(".rowLink").click(function () {
    // get the row containing this link 
    var row = $(this).closest("tr");

    // find out in which table it resides
    var table = $(this).closest("table");

    // move it
    row.detach();

    if (table.is("#table1")) {
        $("#table2").append(row);
    }
    else {
        $("#table1").append(row);
    }

    // draw the user's attention to it
    row.fadeOut();
    row.fadeIn();
});

Up and Down

Now apply a function to the click event of all the elements with class "rowUp." Again get the current row. Then get the previous element. If the previous element is itself a row, then there is room to move up – so detach the row and move it before the previous row. Otherwise do nothing. For moving down, simply switch from prev to next and before to after.

// Setup the "Up" links
$(".rowUp").click(function () {
    var row = $(this).closest("tr");

    // Get the previous element in the DOM
    var previous = row.prev();

    // Check to see if it is a row
    if (previous.is("tr")) {
        // Move row above previous
        row.detach();
        previous.before(row);

        // draw the user's attention to it
        row.fadeOut();
        row.fadeIn();
    }
    // else - already at the top
});
	

In your unit tests, do not test the validation – but rather test to see if the validation attributes have been applied. See Brad Wilson's blog post, DataAnnotations and ASP.NET MVC, for details.

Manual Validation

If you do manually validate, in a unit test or production code, using Validator.TryValidateObject (or other methods in the Validator class), then be sure to set the validateAllProperties = true. By default it is false and that means that only Required properties will be validated. <editorial>Why in the world is this false by default? The obvious and expected behavior is that all properties would be validated</editorial>. Validator.TryValidateObject(someObject, new ValidationContext(someObject, null, null), resultList, true).

Validating Related Objects

Validating inherited fields works nicely. But what about composition / delegation? If you are validating object A, and it is composed of objects B and C that are also validated, then you need some "deep" validation. There is a simple recipe for accomplishing this:

1. Implement IValidatableObject on class A, which will require you to add a method with signature public IEnumerable Validate(ValidationContext validationContext).
2. Set this method to run TryValidateObject on the instance of B and of C. Each call to TryValidateObject will need its own ValidationContext. Constructing a ValidationContext is simple – just pass in the object itself, and generally it is appropriate to pass null for the second and third parameter. Don't forget to throw in a true for validating all properties! Something like this will work (with opportunity for multiple refactorings):
   

   public IEnumerable<ValidationResult> Validate(ValidationContext validationContext)
   {
       var resultList = new List<ValidationResult>();
   
       Validator.TryValidateObject(this.InstanceOfB, new ValidationContext(this.InstanceOfB, null, null), resultList, true);
       Validator.TryValidateObject(this.InstanceOfC, new ValidationContext(this.InstanceOfC, null, null), resultList, true);
   
       return resultList;
   }
   

   
   Now, you wrote a failing unit test for this Validate method before coding it, right?

Caution: if any of the properties directly in A are invalid, then those will be detected and Validate(ValidationContext validationContext) will never be called.

Validation Summary in an MVC Partial View

When using partial views for AJAX support in an MVC application, a @Html.ValidationSummary() should be put inside the partial view. If you put it inside the hosting view, then it will not be populated when you submit a form inside the partial view.

Moles is a powerful and useful framework for unit testing. Or was. But even then, it was overused (at least by me). But no more!

Next: Growing Object-Oriented Software, Guided by Tests showed me how I was letting design guide tests far too much – and here I thought I was practicing test driven development (TDD) just by writing unit tests. No. Moles was a symptom of this – the framework is used to replace a concrete method in a dependent class. Well, in good TDD, the dependent class shouldn't exist yet. There should not be a concrete method to replace.

"There‘s a more subtle but powerful reason for not mocking concrete classes. When we extract an interface as part of our test-driven development process, we have to think up a name to describe the relationship we've just discovered--in this example, the ScheduledDevice. We find that this makes us think harder about the domain and teases out concepts that we might otherwise miss. Once something has a name, we can talk about it." (from Ch 20, unknown page)

Next: Microsoft's own documentation tells me that I'm doing it wrong, and that I would experience "significant performance degradation" from use of Moles. I should have been doing more with Stubs. From the Microsoft Moles Reference Manual" (v0.91, p4):

"In general, we recommend that you use stub types to isolate from dependencies. This can be achieved by hiding the components behind interfaces. Mole types can be used to isolate from third-party components that do not provide a testable API."

Finally: Microsoft has announced the end-of-the-line for Moles. In Visual Studio, the framework will be replaced with one called Fakes.

• devweb
• testweb
• prodweb
• devservices
• testsql
• et cetera

Of course, that scheme presumes that there is only one server in each category. If there are multiple servers in order to fit different needs, then find a name that explains that need: for example, testexternalweb and testinernalweb, for DMZ and internal web servers. I can't begin to reckon the number of times such an (aliased) naming convention would have helped avoid mistakes and improved communication over the years.

Bonus: if abcWEB01 needs to be retired and replaced by with abcWeb10, just change the CNAME so that prodweb points to the new server. Application configuration nightmare averted.