Overview
When integrating with APIs like NetSuite, responses often contain complex, nested data structures. A single field can be a JSON object with its own properties, such as an id and a refName. This hierarchical structure can complicate data mapping, particularly in integration platforms or user interfaces where a simple, flat key-value structure is expected.
This document outlines a design pattern to "flatten" these complex objects into a set of simplified, primitive properties (e.g., strings). This approach simplifies data mapping for end-users while preserving the original complex data structure for internal use.
Problem/Challenge
Directly exposing nested objects from an API response to an end-user for data mapping presents several challenges:
Complexity for Users: End-users, who may not be developers, often struggle to understand and navigate hierarchical object structures, making it difficult for them to map data correctly.
Mapping Tool Limitations: Many integration platforms or mapping tools are designed to work best with flat data structures (e.g., mapping
source.field_Atodestination.field_B). They may lack the UI or functionality to easily handle nested objects.Increased Risk of Errors: The complexity of mapping to nested fields increases the likelihood of user error, leading to incorrect data transformations and integration failures.
Sample API response
{
"creditHoldOverride": {
"id": "AUTO",
"refName": "Auto"
}
}
Requiring a user to map to creditHoldOverride.id and creditHoldOverride.refName is less intuitive than mapping to simple, top-level fields.
Solution Design
The proposed solution is to create a data model that presents a flattened view of the complex object, while maintaining the original nested structure internally. This is achieved through a combination of primary and flattened properties with specific logic in their accessors.
Complex Property (
CreditHoldOverride): A primary property is defined to represent the original nested object from the API response. This property is typically hidden from the end-user interface to prevent confusion.Flattened Properties (
CreditHoldOverride_Id,CreditHoldOverride_RefName): For each field within the complex object, a corresponding "flattened" property is created. These are the simple, primitive fields that are exposed to the user for mapping.Getter Logic: When the hidden complex property is accessed, its
getaccessor first checks if any of the flattened properties have been populated by the user.If they have, it reconstructs the complex object on the fly from these flattened values. This ensures user input is correctly reflected in the underlying object.
If the flattened fields are empty, it retrieves the original complex object that was deserialized from the API response.
Setter Logic: When a new complex object is assigned to the primary property (typically during the initial deserialization of the API response), its
setaccessor stores the original complex object and populates the corresponding flattened properties with the values from that object.
This design ensures that the simple, user-friendly fields and the underlying complex object are always synchronized.
Benefits
Simplified User Experience: Users can map to simple, intuitive fields (e.g.,
CreditHoldOverride_Id) without needing to understand or navigate complex object hierarchies, reducing complexity and the potential for errors.Data Integrity: The original complex object is preserved internally. This ensures no data is lost and that the complete object remains available for any necessary programmatic access.
Reusability: The helper methods that manage the synchronization between the complex and flattened properties are generic and can be reused for any similar complex object, promoting a consistent and efficient implementation.
Compatibility: This approach maintains compatibility with systems that may expect either the full complex object or the flattened representation, as both are available within the data model.
Implementation Notes
The implementation relies on helper methods to manage the state between the flattened properties and the original complex object.
The
getaccessor for the complex property (CreditHoldOverride) calls a helper method (GetLinkRefProperty) that dynamically constructs the object from the flattened fields if they have been modified. This allows the system to capture user-defined values from the simple fields.The
setaccessor for the complex property (CreditHoldOverride) calls a helper method (SetLinkRefProperty). This method is responsible for two things:Storing the original, complete object in a private dictionary or field.
Populating the public flattened properties (
CreditHoldOverride_Id,CreditHoldOverride_RefName) with the values from the original object.
Attributes like
[iPaaSIgnore]can be used to hide the complex property from the UI, ensuring that only the flattened, simplified properties are visible to the end-user during data mapping.
Example of flattening an API response to simplify mapping
* Example Netsuite response
*
* {
* "creditHoldOverride": {
* "id": "AUTO",
* "refName": "Auto"
* }
* }
*
* Example implementation in class:
*/
/// <summary>
/// A helper method to get a complex link reference object.
/// It constructs the object from flattened properties if they exist.
/// </summary>
/// <param name="propertyName">The name of the property to get (e.g., "CreditHoldOverride").</param>
/// <returns>A CommonLinkRefClass object.</returns>
private CommonLinkRefClass GetLinkRefProperty(string propertyName)
{
// Check if either of the flattened properties (_Id or _RefName) have a value.
if (!string.IsNullOrEmpty(GetPropertyValue<string>($"{propertyName}_Id")) ||
!string.IsNullOrEmpty(GetPropertyValue<string>($"{propertyName}_RefName")))
{
// If they do, create a new link reference object from their values.
return new CommonLinkRefClass
{
Id = GetPropertyValue<string>($"{propertyName}_Id"),
RefName = GetPropertyValue<string>($"{propertyName}_RefName")
};
}
else
{
// Otherwise, try to get a previously stored complex object from a dictionary...
return linkRefProperties.TryGetValue(propertyName, out var linkRefClass)
? linkRefClass // ...if found, return it.
: deserializedCommonLinkRefClass; // ...otherwise, return a default deserialized value.
}
}
/// <summary>
/// A helper method to set a complex link reference object.
/// </summary>
/// <param name="propertyName">The name of the property to set.</param>
/// <param name="value">The CommonLinkRefClass object to set.</param>
public void SetLinkRefProperty(string propertyName, CommonLinkRefClass value)
{
// Store the complex object in a dictionary for later retrieval.
linkRefProperties[propertyName] = value;
// Update the corresponding flattened fields (_Id and _RefName) with values from the complex object.
UpdateLinkRefFields(propertyName, value);
}
public class CustomerCompany : AbstractIntegrationData
{
#region CreditHoldOverride
// This attribute likely tells a UI-generation tool to ignore this complex property,
// so the user only sees the simple, flattened ones.
[iPaaSIgnore]
// This is the property that directly maps to the nested JSON object from NetSuite.
[JsonProperty("creditHoldOverride", NullValueHandling = NullValueHandling.Ignore)]
public CommonLinkRefClass CreditHoldOverride
{
// The getter uses the helper method to construct the object.
get => GetLinkRefProperty(nameof(CreditHoldOverride));
// The setter uses the helper method to store the object and update flattened fields.
set => SetLinkRefProperty(nameof(CreditHoldOverride), value);
}
// This is the flattened property for the 'id' part of the object. It's what a user might map to.
[JsonProperty("creditHoldOverride_Id", NullValueHandling = NullValueHandling.Ignore, DefaultValueHandling = DefaultValueHandling.Ignore)]
public string CreditHoldOverride_Id { get; set; }
// This is the flattened property for the 'refName' part of the object.
[JsonProperty("creditHoldOverride_RefName", NullValueHandling = NullValueHandling.Ignore, DefaultValueHandling = DefaultValueHandling.Ignore)]
public string CreditHoldOverride_RefName { get; set; }
#endregion
...
}