codeflood logo

Automated Testing and Sitecore - Part 8

Well. When I started writing this multi-part series on automated testing and Sitecore I never intended for it to flow over into 8 parts. But I think what I've covered in this series are the fundamental pieces any developer requires to be able to affectively automate their tests for Sitecore code.

So, now that we're at the end of the series, I'll leave you with some tips.

  • Don't get hung up on "unit" testing. As you've seen, it's very difficult to completely isolate any code we write against the Sitecore API, as that API is a dependency. Also at times, the entire Http Request becomes a dependency when we're testing components which are context sensitive. Don't worry too much about ensuring you're testing is "unit". Unit testing makes it easier to pinpoint which piece of code is misbehaving, but all we really need is an atomic, repeatable test harness for our code to ensure it behaves as expected. Once you let go of pure units, you'll be able to write your tests a lot easier.
  • Plan for testing. I always start writing new pieces of code by thinking about how I could test that code. (Actually, I start by writing the tests so I'm TDD.) Is it presentation? Is it static in behavior? Do I need the custom test runner to provide a Sitecore context to the running tests? How will I make the test repeatable? What dependencies on the environment are there, such as, do I need to create any items to test the component?
  • Design for testing. Take note of how you write your code and how you design the architecture of your solution to make sure components are easily testable. For example, complex functionality should never be in the code behind of a sublayout or layout. It should always be in a different class. This makes the functionality available to other parts of the solution as well as making the testing of the logic of that function much easier as the logic is not mingled with presentation code.
  • Ensure your tests are repeatable. You want to be able to retest and retest and get consistent results. Part of this will most likely be setting up your content tree before the test so you know what results to expect. Being able to pragmatically setup the environment is paramount to making your tests repeatable. You don't want to have to rely on a person creating certain items or setting up the environment manually before a test. Ensure there are as few steps as possible in the testing strategy. The closer you can get to "single click" testing, the less likely you are to have developers not running the tests to make sure they didn't break anything.
  • Ensure your tests are atomic. You need to cleanup any test items created for the test at the conclusion of the test (or test fixture). You don't want tests to impact on each other as these ends up leading to very messy maintenance. You need to ensure your tests can grow as your solution does and the biggest impact on this is the atomicity of your tests.

I hope the techniques I've shown in this series will help you on your way to better testing. But these techniques are still evolving. As I'm challenged by a new dev about the complexity of testing against Sitecore, or as I find better ways to test, I incorporate those into my testing techniques. I'll keep you posted on what I find in the future. And of course if you think I'm completely off the mark about any of this, let me know. Only through challenging the technique do we truly assess it's effectiveness and suitability.

Comments

Anonymous

You mention TDD - do you have any other thoughts on this in regard to Sitecore? For example:
A custom test runner in a Web Form running in Sitecore context... it feels cumbersome. Is there perhaps a way to reintegrate the test runs within Visual Studio?
Creating Sitecore data (templates, items) for test fixtures... can you rely on the things you create purely programatically, including their security etc? Or is there some reason why you would "drop out to Sitecore GUI" to create data used in tests?

Anonymous

Regarding "The closer you can get to 'single click' testing, the less likely you are to have developers not running the tests to make sure they didn’t break anything" - usually we would have our build server (such as TeamCity with mstest.exe) run all tests. Do you have any experience with this using Sitecore?

Alistair Deneys

I know the custom test runner is quite heavy from the runner perspective, but Sitecore requires an HttpContext to run. Remember that any test that calls into the Sitecore API is not a unit test, but an integration test. I have tried previously to hack up an HttpContext and trick Sitecore into running properly but faced too many issues. (perhaps I need to spend some more time on this.)
And even though it's a custom test runner, there is nothing extra for the developer to do when writting tests. It's still just NUnit. But I can see you're aiming for automating your tests. In that case I'd probably still have the custom runner but have the results presented in XML, text, CSV or some other format other than HTML that you can easily parse. Then you could incorporate a call to the test runner in your build process (or wherever you're calling it) and have that read the results.
As for creating the data items; I like to do this in code to ensure the setup is repeatable and doesn't rely on a person to do it. Because Sitecore exposes a full API there is nothing the UI does that cannot be done in code (another reason I like Sitecore so much). You might construct pieces of your test data using the UI, but then grab the raw data of the item using the API and reconstruct it using the API as well for the tests.

Alistair Deneys

I've only used NUnit (default runner and my custom runner) and VSUnit (MSUnit? MSTest? Whatever comes with Visual Studio 2008+). I know you want your tests run from your build, but you'd also want your devs to execute tests for code they have changed before checking in to capture issues before they even get to the server.

[...] Automated Testing and Sitecore part 8 – Testing guidelines [...]

Leave a comment

All fields are required.