Unit Test Scoping in Spring ’12

There are several pitfalls that developers can fall into when writing unit tests, and traditionally speaking one of the easiest to fall into can be coding with a reliance on existing data.  So for instance, you want to to prove that your SOQL is correctly getting data from your custom object – this unit test might seem relatively innocent:

List<Furniture__c> dressers = [SELECT ID, Name, Type__c from Furniture__c where Type__c =: dresserFilter LIMIT 10];
System.assertNotEquals(dressers.size(),0);

Which works great in the sandbox where you’ve been developing your custom furniture application with test data, but might instantly break when you move to production where no furniture data as been loaded.  More egregious scenarios may include referencing specific ID’s from one org – which won’t match up to the ID in another org.  I have myself, and I’m sure other development teams as well, found themselves at the late end of a Friday night deploy because the code I thought had excellent test coverage fails completely when moved to a different environment.

With API 24.0, or Spring ’12 to the seasonally minded, the data scope for unit testing will default so that it will not see any pre-existing data from the org.  Unit tests referring to older versions of the API will still run as expected.  With Spring ’12, the above code will fail as it doesn’t have access to the system data – but can properly be re-written to handle the situation:

List<Furniture__c> furniture = new List<Furniture__c>();
Furniture__c dresser = new Furniture__c(Name='Three Drawer',Type__c='Dresser');
Furniture__c sofa = new Furniture__c(Name='Sleeper Sofa',Type__c='Couch');
furniture.add(dresser);
furniture.add(sofa);
insert furniture;

String dresserFilter = 'Dresser';
List<Furniture__c> dressers = [SELECT ID, Name, Type__c from Furniture__c where Type__c =: dresserFilter];
System.assertEquals(dressers.size(),1);

This unit test will run as expected anywhere that has my custom object defined and since I control the test data relevant to the unit test – I can refine my system asserts for the specific use case.  If you have unit tests, however, where accessing the system data is exactly what you intend – then you can annotate the method itself with isTest(SeeAllData=true) to allow that access.  So to use the original example with API 24.0 but have it operate as before, simply add the annotation:

@isTest(SeeAllData=true)
static void testFurnitureSystemWide() {
   List<Furniture__c> dressers = [SELECT ID, Name, Type__c from Furniture__c where Type__c =: dresserFilter LIMIT 10];
   System.assertNotEquals(dressers.size(),0);
}

You can also annotate the Apex class itself if you want to apply the data access to every method within the class.  This new setup should help developers write better, more portable unit tests and aims to also improve the speed of running unit tests in general.  If you are looking for ways to easily create test data, check out Matthew Botos‘ excellent SmartFactory project – which speeds the creation of test data by using describe to fill out required fields dynamically.

tagged , , Bookmark the permalink. Trackbacks are closed, but you can post a comment.
  • Anonymous

    Josh,

    It’d be great to have something like (BypassValidationRules=true) to allow unit tests bypass the validation rules. I know it is debatable and there are good reasons not to have this feature. But in many scenarios it’d be useful, e.g. there might be a unit test in a package that creates a Contact record. If the installer created custom validation rules around Contact the unit test would fail.

  • http://twitter.com/cirruscg Ryan Huff

    This is probably my favorite feature in Spring ’12. I used to have code that would delete data like Price Books and Products so that it could re-create it for testing. This got difficult when I started getting a lot of data in the production org and started getting errors about “too many related records”. This feature was a welcome addition.

  • http://twitter.com/cirruscg Ryan Huff

    Unit tests now don’t need to run on package install, so this is solved for that scenario. Ensuring that VRs run during unit tests helps to identify issues that a subscriber’s org may be introducing…

  • David Claiborne

    This is all well and good, but what if you have code that operates on the self-service portal user. And there is no way to create a self-service portal user via the API. What do you do then?

  • David Claiborne

    This is all well and good. But I have had the case in the past when I have created code that operates on the self-service portal user. And, I discovered, there is no way to create a self-service portal user via apex. So what do I do then?

  • Anonymous

    Is there a typo in the second code snippet for the assert? It seems like it should be one of the following instead since the unit test is not using pre-existing data and only one record was inserted that matches the query.

    System.assertEquals(dressers.size(),1);
    System.assertNotEquals(dressers.size(),0);

  • http://twitter.com/joshbirk Josh Birk

    You would be correct, thanks for the catch. Updated now.

  • http://twitter.com/joshbirk Josh Birk

    David, does appear DML isn’t allowed on SelfServiceUser so it looks like this would be a use case where you would need to rely on existing system data.

  • http://twitter.com/rich_unger Rich Unger

    One other major advantage to this feature is that you can’t inadvertently lock records in a test that might be needed for a production request during a deploy.

  • http://profile.yahoo.com/HIGEZVVFMQACQD2YFKICJ5A6KA LiuMouse

    I think, If you try to use the SmartFactory in large org, it will cause a bit of problem, for example, if you just use part of fields in object, SmartFactory will assign values to other fields in other apps, so it will cause some validation rules or something else, so I think you should have consideration before use this smart test library.

  • Graeme Ing

    It would be nice to limit unit tests just to the code you are deploying. Running a full unit test on our installation takes 50 minutes, due to the amount of custom code we have. :( I’m also running into it starting to time out and I’m worried there might come a day when I can’t deploy at all. What kind of deployment times are you folk running with?

  • Thomas Gagne

    Is there an option that can be set at the org level to allow access to data, so old unit tests may run successfully while trying to get a deploy to work with new ones?

  • http://twitter.com/joshbirk Josh Birk

    You can associate unit tests with the older API version to effectively “deprecate” to the old access without having to change any code.