Testing – it is more than code coverage….

As a Force.com developer, you are probably already aware that before you can deploy your code into production, you should have atleast 75% test coverage.  Though necessary (since the platform enforces this requirement), it is hardly sufficient to ensure that your code meets the requirements and expectations of the users.  I will briefly describe the different types of testing you should consider before rolling out a Force.com application.

Unit Testing – Unit testing is used to verify that each individual piece of code (trigger, a method of a class, custom controller code etc.) is behaving as expected.  Force.com provides a testing framework which helps in the creation and execution of such unit tests.  This article provides a great overview of how to write and execute such unit tests.  As mentioned earlier, the platform enforces that atleast 75% of your code has test coverage before custom code can be deployed into production.  Unit testing can be done by individual developers either in their developer sandboxes or DE orgs. Since these tests are run in the production org before the code can be deployed, it is important that these unit tests are self contained i.e. not have any dependency on data that may exist in a particular org.  Also see Apex Test Coverage Best Practices  for a good coverage of best practices for writing unit tests on Force.com.

Integration Testing -  The goal here is to test the interface between the different units of code.  This testing can be done in a configuration only sandbox or a developer sandbox with some sample data.  Integration testing will surface any mismatched assumption made about the behavior of code that need to talk to each other. 

System Testing – This is testing that is conducted on the complete, integrated system to verify that all functional requirements are met.  The system is treated as a black box and along with the functional requirement, performance testing and security requirement verification should also be carried out.  A full copy sandbox provides the best environment for system testing.

User Acceptance Testing - This is the final step before rolling out the application. The end users who will be using the application should have the confidence that the application meets their requirement.  Most of the bugs should be fixed before you roll it out for User Acceptance Testing.  The focus is on usability of the application rather than the technical implementation.  This test should be carried out on a full copy sandbox since it simulates the production environment. 

These tests should be fully described in your test plan.  A test plan is a comprehensive document that describes the objectives and scope of testing, the stakeholders, the testing methodology and tools used, the metrics that will be captured and the testing schedule.  It should be a living document that has the active participation of all the stakeholders.  If you are a consultant implementing a system for your customer, you should have a formal sign off process.  

Let's look at a few best practices….

  1. Have a testing strategy and a test plan document. 
  2. Address all the different types of testing as described above. 
  3. Do not push testing to the very end – testing shouldn't be seen as a check mark item that stands in the way of deploying the code but rather as a way of improving the software quality and verifying that the end user expectations are met. We recommend using a test driven development methodology
  4. Plan and schedule any resource dependencies ahead of time – the resource dependency may be end users who have to do acceptance testing or access to some external system to do system testing.  If there is any change in schedule communicate to all parties involved as quickly as possible and agree on a new schedule.
  5. Test early, test often – a bug caught earlier in the development cycle is cheaper and easier to fix vs. if it is discovered later in the cycle.

Finally, don't forget to budget time and resource to develop and execute on a test plan.  Though there may be a temptation to just push the code into production and hope for the best, experience with numerous software projects has shown that it rarely works.

tagged , Bookmark the permalink. Trackbacks are closed, but you can post a comment.
  • http://blog.jeffdouglas.com Jeff Douglas

    Good article Nick. I would also recommend that EVERYONE involved in testing vote for my Idea: “Provide Apex Testing Functionality Similar to JUnit”
    http://ideas.salesforce.com/article/show/10096208/Provide_Enhanced_Apex_Testing_Functionality#skin=adn

  • http://johnfmoore.wordpress.com John Moore

    Really good article for any developer. Developers are always the first line of defense from a code quality perspective. As authors of the code we own initial responsibility and must be aware of how to properly test code.
    John
    http://twitter.com/JohnFMoore

  • http://gtuerk.blogspot.com Gabe Tuerk

    I’m a big fan and recently wrote a post about TDD and what to test, how. See that on my blog.
    This article needs one addition: Regression Testing
    Continuous verification takes a back seat to new Functionality and the only thing keeping an eye on things are good Unit Tests. Formalizing a regression plan (from a business standpoint) and allocating some time for that before each major release is quite important

  • Luke C

    I agree with Gabe about regression testing. Since I don’t have a full copy sandbox, I rely on a couple extra test classes that I keep on production. Regression test classes are useful because they become pass/fail rather than 75% threshold dependent. I use mine not to test all code coverage but a very wide range of possibilities on a specific part of the code. I’ve got 4 other Sf admins and it’s too easy for some innocent change to undermine the code, so this is an easy way for all admins to check whether they just broke it; and for me to feel confident that maintenance changes I made are stable.

  • Nick Simha

    Gabe,
    Good point on Regression testing – it should be scoped into your build process. And could you please post the link to your article on Test Driven Development that you mention?
    Thanks

  • Gary B

    Agree with the article and the comments, especially around regression testing.
    Quick question to canvass opinion though – has anyone else found that as your test classes grow in number and complexity, so deployment to production environment takes longer and longer? This is because when you’re deploying code, all your test code is run.
    My current feeling is that production environments should hold enough test code to provide enough coverage but other tests should be held in other sandboxes. What are other peoples thoughts?

  • http://www.navatargroup.com Ketan Khandkar

    Thanks, Nick, for stressing the importance of the effort required for testing. Good add about Regression testing.
    We at Navatar have a developed a testing model for Force that can drastically reduce your testing time. Write me at kkhandkar@navatargroup.com for more info. You can also download a free best practice paper titled “Top 10 Mistakes Architects Make When Building a Force.com Application” at
    http://navatargroup.com/download1a.php?download=10MistakeWhitepaper
    Regards
    Ketan Khandkar

  • Caryn Sesta

    Hello
    I just got off the phone with Salesforce support and they instructed me to post my issue here. I have a few reports I am looking to run and need to be able to calculate # of days and avg. # of days between 2 date fields. They said that I will need to create a custom formula and that the developers could provide assistance. Please contact me via e-mail when more information is needed.
    Thanks
    Caryn
    csesta@masonicare.org