Sunday, June 1, 2008

Why to use PEX (Program EXploration for .NET). Sample 1. Reason 1.

Sample under test:

public static class XmlUtility
    {
        public static string Encode(char input)
        {
            switch (input)
            {
                case '\n': return "
";
                case '\r': return "
";
                case '&': return "&";
                case '\'': return "'";
                case '"': return """;
                case '<': return "&lt;";
                
                default: return new string(input, 1);
            }
        }
    }

Non-PEX test sample:

        [TestMethod()]
        public void EncodeTest()
        {
            Assert.AreEqual<string>("&#xA;", XmlUtility.Encode('\n'));
            Assert.AreEqual<string>("&#xD;", XmlUtility.Encode('\r'));
            Assert.AreEqual<string>("&amp;", XmlUtility.Encode('&'));
            Assert.AreEqual<string>("&apos;", XmlUtility.Encode('\''));
            Assert.AreEqual<string>("&quot;", XmlUtility.Encode('"'));
            Assert.AreEqual<string>("&lt;", XmlUtility.Encode('<'));
            
            Assert.AreEqual<string>("a", XmlUtility.Encode('a'));
        }

PEX test sample:

[PexMethod()]
        public void EncodeTest(char input)
        {
            if (input == '\n') { Assert.AreEqual<string>("&#xA;", XmlUtility.Encode(input)); return; }
            if (input == '\r') { Assert.AreEqual<string>("&#xD;", XmlUtility.Encode(input)); return; }
            if (input == '&') { Assert.AreEqual<string>("&amp;", XmlUtility.Encode(input)); return; }
            if (input == '\'') { Assert.AreEqual<string>("&apos;", XmlUtility.Encode(input)); return; }
            if (input == '"') { Assert.AreEqual<string>("&quot;", XmlUtility.Encode(input)); return; }
            if (input == '<') { Assert.AreEqual<string>("&lt;", XmlUtility.Encode(input)); return; }
            
            // Everything else should not be encoded
            Assert.AreEqual<string>(new string(input, 1), XmlUtility.Encode(input));
        }

Note, I had to use ifs instead of switch/case as I had a bit of an issue to make PEX working as expected  in case of switch statement in the test method.

So what the difference does it make?

In both tests we effectively provided parameters to make test coverage 100% (sorry for a bit of approximation). Both tests pass.

Though, there is a problem with algorithm and it doesn't encode the '>' input. TDD approach wise we would add a test that makes our encoding method fail, and fix our encoding to encode '>'. TDD purists are known for eating people alive for not following that "test first" strategy, but even following this TDD approach you may not be able to protect against simply overlooking something.

So lets be non TDD and add a line of code to fix our issue:

case '>': return "&gt;";
 
Now lets rerun the tests:
There is no reason why current non-PEX test would fail, so it really passes.
Lets see how PEX test is doing:

image
 
And now we can see a bit of the PEX magic in action!
Behind the scenes, looking at our code under test, PEX generated an extra test method for input of '>' and this failed our
Assert.AreEqual<string>(new string(input, 1), XmlUtility.Encode(input));

"assert for the rest of cases" statement.

So now, I'm actually explicitly forced to add the test case for '>', which I'm doing:

if (input == '>') { Assert.AreEqual<string>("&gt;", XmlUtility.Encode(input)); return; }

To make the PEX tests pass:

image

Conclusions:
1. I should be better on writing unit tests, but I can never be perfect.
2. PEX can really give me a hand for catching some of my possible mistakes in unit tests.

Ultimate link for PEX (Program Exploration for .NET) resources: http://research.microsoft.com/Pex/
First user experience: http://blog.benhall.me.uk/2008/05/microsoft-pex-05-released.html
TDD purists may be watching you! http://intellij.net/forums/thread.jspa?messageID=5215367 (scroll down a bit until the Gabriel Lozano's  post, I had fun reading ...)

Post a Comment