Asynchronous Testing

Writing concurrent code has many pitfalls and because of the inherit complexity, testing that code may also prove difficult. By leveraging XCTest, the test cases default to synchronous tests (i.e. when the test hits the last line in scope, it ends). Now, some developers may try to solve this by adding sleeps or manipulating the run loop, but those methods are unreliable or can cause side effects. To properly handle parallel scenarios, XCTest provides an API that allows you to create expectations for the outcome of an asynchronous operation by adhering to the XCTWaiterDelegate protocol.

XCTestExpectation

The simplest way of creating an asynchronous-aware test is to use XCTestExpectation. This is the base class for all of the other expectations and provides the ability to set a required number of fulfillment calls, create an inverted expectation (success if it isn’t fulfilled), fail the test if too many fulfillments are triggered, and expectations can be set to only be successful if they are fulfilled in order.

Example:

- (void)someTest
{
  XCTestExpectation *expectation = [self expectationWithDescription:@"My Expectation"];
  [someObject doWorkWithCompletionHandler:^{
    [expectation fulfill];
  }];
    
  /**
  * This method waits on expectations created with XCTestCase's convenience methods only.
  * This method does not wait on expectations created manually via 
  * initializers on XCTestExpectation or its subclasses.
  * Timeout is in seconds.
  */
  [self waitForExpectationsWithTimeout:2 handler:^(NSError * _Nullable error) {
    // Test cleanup
  }];
}

Inverse Example:

- (void)someTest
{
  XCTestExpectation *expectation = [self expectationWithDescription:@"My Expectation"];
  expectation.inverted = YES;
    
  [someObject doWorkWithCompletionHandler:^{
    // Something failed, so no fulfill
  }];
    
  [self waitForExpectationsWithTimeout:2 handler:^(NSError * _Nullable error) {
    // Test cleanup
  }];
}

Multiple Fulfillment Example:

- (void)someTest
{
  XCTestExpectation *expectation = [self expectationWithDescription:@"My Expectation"];
  expectation.expectedFulfillmentCount = 3; // Default is 1
    
  [someObject doWorkWithCompletionHandler:^{
    [expectation fulfill];
  }];
  [someObject doWorkWithCompletionHandler:^{
    [expectation fulfill];
  }];
  [someObject doWorkWithCompletionHandler:^{
    [expectation fulfill];
  }];
    
  [self waitForExpectationsWithTimeout:2 handler:^(NSError * _Nullable error) {
    // Test cleanup
  }];
}

Over Fulfill Example:

- (void)someTest
{
  XCTestExpectation *expectation = [self expectationWithDescription:@"My Expectation"];
  /**
  * If set, calls to fulfill() after the expectation has already been fulfilled
  * Exceeding the fulfillment count will raise an exception. 
  * This is the legacy behavior of expectations created through APIs on XCTestCase
  * but is not enabled for expectations created using XCTestExpectation initializers.
  */
  expectation.assertForOverFulfill = YES;

  [someObject doWorkWithCompletionHandler:^{
    [expectation fulfill];
  }];
  [someObject doWorkWithCompletionHandler:^{
    [expectation fulfill]; // Exception will be thrown
  }];
    
  [self waitForExpectationsWithTimeout:2 handler:^(NSError * _Nullable error) {
    // Test cleanup
  }];
}

Ordering Example:

- (void)someTest
{
  XCTestExpectation *exp = [self expectationWithDescription:@"My Expectation"];
  XCTestExpectation *exp2 = [self expectationWithDescription:@"My Expectation2"];

  [someObject doWorkWithCompletionHandler:^{
    [exp2 fulfill]; // Causes failure due to it completing first
  }];
  [someObject doWorkWithCompletionHandler:^{
    [exp fulfill];
  }];

  [self waitForExpectations:@[expectation, expectation2] timeout:2 enforceOrder:YES];
}

XCTNSNotificationExpectation

For work that interacts with NSNotificationCenter, you can use XCTNSNotificationExpectation. This type of expectation allows you to wait for a certain notification to be fired from a certain sender and allows you to wait on another instance of a notification center that isn’t the default.

Example:

- (void)someTest
{
  XCTestExpectation *expectation = [self expectationForNotification:@"com.ex.notification" 
                                                             object:someObject
                                                            handler:^BOOL(NSNotification *_Nonnull notification) {
    /**
    * If not provided, the expectation will be fulfilled by the first 
    * notification matching the specified name from the observed object.
    */
    // Return YES/NO based on introspection
  }];
    
  [someObject doWork];
    
  [self waitForExpectations:@[expectation] timeout:2];
}

Custom Notification Center Example:

- (void)someTest
{
  XCTNSNotificationExpectation *expectation = [[XCTNSNotificationExpectation alloc] 
                                                initWithName:@"com.example.notification" 
                                                object:someObject
                                                notificationCenter:someOtherCenter];
    
  [someObject doWork];
    
  [self waitForExpectations:@[expectation] timeout:2];
}

XCTKVOExpectation

Another powerful feature of Objective-C is KVO. Since KVO notifies an interested party in the event of a value change (thanks to KVC), you can use XCTKVOExpectation to wait until your work has completed with the expected result by specifying the NSKeyValueObservingOptions you want.

Example:

- (void)someTest
{
  NSKeyValueObservingOptions options = NSKeyValueObservingOptionNew;
  XCTKVOExpectation *expectation = [[XCTKVOExpectation alloc] initWithKeyPath:@"somePath" 
                                                              object:someObject
                                                              expectedValue:someValue 
                                                              options:options];
    
  [someObject doWork];
    
  [self waitForExpectations:@[expectation] timeout:2];
}

XCTNSPredicateExpectation

Finally, you can use the NSPredicate API to create logical conditions that can be tested for a YES or NO value. Therefore, you can create a XCTNSPredicateExpectation that the system will test until timeout (or success) in order to wait until some operation(s) completes.

Example:

- (void)someTest
{
  NSPredicate *predicate = [NSPredicate predicateWithBlock:^BOOL(id _Nullable object, NSDictionary<NSString *,id> *_Nullable bindings) {
    // Perform logical validation
  }];

  XCTNSPredicateExpectation *expectation = [[XCTNSPredicateExpectation alloc] 
                                             initWithPredicate:predicate 
                                             object:someObject];
  
  [someObject doWork];

  [self waitForExpectations:@[expectation] timeout:2];
}

Using expectations is an easy way of making good asynchronous-aware tests. Now, there are other ways of handling asynchronous code such as using OCMock to stub and then execute arbitrary NSInvocations or to use the Objective-C runtime to swizzle methods to do something else during test executions. However, the expectation API allows you to avoid the complexity of adding a 3rd-party library and avoid the runtime while still being powerful.