Advanced Topics and Best Practices

This article covers essential best practices for building a robust integration, such as logging, exception handling, and performance optimization.

Logging

Logging is useful for both users and integrators to troubleshoot issues. A consistent approach to logging involves:

Logging API calls and their responses. The template integration project includes standard logging for these, but ensure that your integration will utilize the existing log calls correctly.
- Successful API calls should log both the request and response at a Detail level. Errors should log both request and response at the Error level.
- LogRequest includes code to log your request parameters, including headers and URL parameters. Since the logs may be customer-facing and will be stored in plain text, ensure that any API keys or username/password details are omitted.
Include logging at any place where action is terminated, but users might expect further action. For example, if a polling request has no entries, add a log indicating that no entries were found rather than exiting the call. If a transfer might normally have prerequisite logic applied, but it found no prerequisite calls were required, log that as well.
Most informational log calls should be at the Detail level.
Verbose-level logs will not be visible in any output generated by iPaaS.com. They will only be visible in your internal, local testing.
Include your integration name in the log location parameter. This can be easily accomplished with the static variable Identity.AppName (in the template integration). For example, a log thrown from the method PollRequest in TranslationUtilities should have the location set as: $”{Identity.AppName}.TranslationUtilities.PollRequest”.

Exceptions

Do not ignore and discard errors in a try/catch block. Any caught errors must be handled (including a comment on why and a detailed log entry) or rethrown.
A common error scenario is an external system returning a 404 for valid calls when no data is present (e.g., you check to see if a product has any variants and you get a 404 in response when there are none). In this case, use the notFoundAction variable in the APICall class to handle this.
Prerequisites and post actions calls using DataHandlerFunctionAsync do not currently throw errors. Check the HasErrors field after the call is complete to find out if errors occurred. This will not show you the error message, but will tell you that an error occurred. If this is a fatal error for your transfer, include the id and type of data that experienced an error when you throw an exception.

Other

Auto-reauthorizing tokens: In systems that have expiring tokens, especially if the tokens are short-lived, you should be able to detect and recover from tokens expiring mid-transfer. Ideally, you can store the token expiration (persistent data provides a built-in way to do this), check it before each call, and refresh the token before making your next call. In some cases, you may not know the token has expired until it is used and the use fails. In that case, you should modify your APICall.HandleResponse method to account for this, refresh the token, and re-run the failed call without interrupting the current transfer. Be aware that other transfers may also be running concurrently for the same integration.
For any calls that return multiple entries, ensure you properly handle pagination and do not just return the first page. Often APIs will return metadata in the form of headers or response fields indicating that there is more data on other pages. However, this will not always be the case, and you must properly handle cases where multiple pages are needed. This is generally handled in the ApiCall class so that it can be handled in one place without requiring other methods to worry about how pagination is handled.
Look for efficiencies in how calls to your external API are made.
- If data is likely to be needed across multiple child mappings, consider caching it. For example, if you are mapping products with variants, you may notice that each variant mapping will likely require a call to an options endpoint. If the calls are the same for each variant, you can cache the results locally and save repeated API calls. We have seen transfer times drop by as much as 75% by targeted application of simple caching strategies.
- Look for ways you can engage the batch processing feature. If you are sending a product with multiple variants where each variant must be sent separately, you can use the BatchExecution class to execute multiple API calls at once. This feature can make troubleshooting and debugging harder, as well as potentially hitting external API resource limits, so use it only when essential for reasonable performance.
Some external APIs have complex throttle limit algorithms and may have very low resource limits. In these cases, you may be able to add a delay into your integration to handle these automatically, rather than fail the transfer once a limit is reached. For example, an external API with a leaky bucket throttling algorithm may end up rejecting a call, but including metadata that you may retry the call again in a few milliseconds. By convention, any throttling delay that is less than 1 second should be handled internally, while any delay over 1 second should be handled by throwing a QuotaException. Use the method below to sleep on delays less than 1 second. This will allow you to still support iPaaS.com-initiated cancellations during your delays:
```
public static async Task SleepWithCancelSupport(int milliseconds)
{
    var cancellationToken = CallContext.UniversalCancellationToken;
    //If there is no cancellation token set, just use the normal sleep process
    if (!cancellationToken.HasValue)
    {
        await Task.Delay(milliseconds);
        return;
    }
    
    var cancelled = cancellationToken.Value.WaitHandle.WaitOne(milliseconds);
    if (cancelled)
    {
        throw new TaskCanceledException("Thread cancelled while sleeping");
    }
}
```

Be aware of cases where you may need handling for special characters.

If you use RestSharp or GraphQL to handle your calls, most special character issues related to body or url encoding should be handled automatically. If you choose to use a simple HttpClient for your calls, you will need to handle special character encoding yourself.
Path segment parameters will have issues with special characters (such as / or ?) if they are allowed.
For numeric types, unless null and 0 are interchangeable, use nullable datatypes. Avoid using the following decorator unless necessary, since it may lead to sending null when a 0 is intended:
```
DefaultValueHandling = DefaultValueHandling.Ignore
```

API Throttling in iPaaS.com

External Collision Handling

Working with API Rate Limits

Data Modeling