Quiz – how can you have 100% test coverage in Sandbox and 0% in production?

By Nick Simha | Published: October 1, 2009

Answer – if your test suite is not self contained i.e. it depends on specific data in Sandbox.

I got a call from one of our customers today about not being able to deploy into their production org due to inadequate test coverage even though they were getting 100% coverage in their development sandbox.  Our support tracked it down to hard coded test methods.  If your test methods depend on specific data in an org and you move such a test method to a different org, it will fail.

Here is an example where the record Id is hardcoded…..

static testMethod void myTestHardcodedIds(){	 // INCORRECT - By hardcoding this Account Id in the test method, the test method  // will fail in every other org where this code is deployed because the hardcoded  // Account Id won't exist there.  Account testAccount = [select id,name from Account where Id='001300000040lMM']; testAccount.billingState='CA';	  update testAccount;	

 // Verify that the billingState field was updated in the database. Account updatedAccount = [SELECT billingState FROM Account WHERE Id = :testAccount.Id]; System.assertEquals('CA', updatedAccount.billingState);

The right way to write test methods is to make them self contained i.e. not depend on data in a particular org.  I have taken the above example from this great article on testing.  I encourage you to go through that to find out how to write the above test method correctly as well as other best practices.

tagged , , Bookmark the permalink. Trackbacks are closed, but you can post a comment.
  • Caleb

    While a great concept to have self contained test suites, sometimes you have to query for data: Users are always hard to create: you can run into license limits for standard users and for the Customer Portal I still cannot save a user record…at first I though it was an issue with mixed DML – insert contact and user isn’t normally allowed at the same time but I can create Partner Portal users. And Share/Group and other “setup” type records are always sticky and often times easier to leverage existing data.
    But definitely try to write self contained tests if possible and if you know you’ll be doing a lot of Apex in an org spend the time up front to create generic getData() methods so that if a field on an Account becomes required you only have to change your code in one spot. And if you do have to query for data…make sure the same data (criteria) exists in both Sandbox and Production (i.e. don’t use record Ids use names instead).
    And also do include at least one System.assert() for each test…otherwise you’re not really testing anything you’re just running your code to fake code coverage. Tests take a while to write, but can be invaluable when you go back to your code a few weeks/months later to make an update.
    All of course IMHO,
    Caleb

  • http://developinthecloud.wordpress.com Wes

    To ensure a clean test data suite I usually delete data from the areas I’m working with. This removes all ‘live’ data for the duration of the test, and let’s you create you own contained data set for the scope of the test method. I think I have a blog post about it somewhere.

  • Arun

    I usually create the test data (even the static ones) in my test coverage class itself. This way my test cases never failed during migration between orgs.

  • Nick Simha

    Great tips. I am wondering if we should have a micro-site / page for testing related tips.

  • Jon Keener

    This is also really important when using a config only sandbox, since when refreshing the sandbox, there is no data in the environment.

  • thecrmninja

    I, like Arun, create the necessary data elements within the Test Class.
    Additionally, I have had great success previously with creating formula fields that guide my Apex creations. In this way, you can create formula fields in both orgs (dev and sandbox) that respond to separate and distinct data elements without concern that your test class won’t clear.
    For example, you have a trigger that reacts to events in particular queues. You can create a formula field that sets itself to 0 or 1 depending on the DB ID value of a Queue. Create the same field FOR DIFFERENT DB ID’S in Production and Apex only needs to worry about Zero’s or One’s.

  • Mike Eaton

    Here’s another situation that you may find interesting. It is also possible to have existing code already in PRD that does not meet the minimum test requirements. This will also prevent you from deploying to PRD because Salesforce automatically runs all the TestMethods in PRD before allowing any further deployment.
    This may not sound like a big issue – and Salesforce tech support initially suggested why not update the PRD affected code. Anyone see the glitch yet? If you can’t deploy to PRD because the existing code IN PRD doesn’t pass the TestMethod requirements, how do you deploy the updates?
    Well there are several questions you might have at this point, like how did code that didn’t have minimum coverage get into PRD in the first place? Very good Question!
    This happened about a year ago to a client of mine. Not surprising, many companies start evaluating SFDC with a trial org. A trial org is more akin to a Developer Edition supporting code development without any TestMethod coverage restrictions. During that trial period, there were numerous demo applications and code samples added for obvious evaluation reasons. Once the business decided to convert their Trial Org to a Full PRD Org, Salesforce just flip the switch (or whatever magic they perform) and now the Trial Org is a PRD org – WITH all the restrictions in place.
    How we managed to resolve the issue is another story entirely. Suffice it to say, it is possible – albeit accidental – to get into this tricky situation.