Background
This document outlines a method for synchronizing large datasets—either by pushing data to iPaaS.com or pulling data from iPaaS.com—by leveraging the Postman Runner feature. This technique is particularly useful when seeding an entire product catalog, migrating historical orders, or handling datasets that exceed the standard limits of the manual sync interface.
Understanding the Standard iPaaS.com Sync Process
Typically, iPaaS.com processes data synchronization as follows:
Webhook Notification: For incoming data, a webhook notifies iPaaS.com about new or updated records in the source system.
Queue Entry: The webhook triggers the creation of an entry in a processing queue.
Data Retrieval & Normalization: As the entry reaches the top of the queue, iPaaS.com makes a GET request to the source system API to retrieve the full record details. Based on the defined scope (e.g.,
CREATE
,UPDATE
,DELETE
) and mapping configurations, the data is normalized into the iPaaS.com data models.Outbound Processing: Once the data is created or updated within iPaaS.com, any connected systems listening for changes to that data type will have the corresponding record queued for outbound synchronization, following a similar process.
Polling Alternative: Systems using polling follow a similar pattern; the polling process identifies new or updated record IDs, which are then added to the processing queue.
Manual Sync: The manual sync page enables users to add records directly to the queue. Using Debug during a manual sync prioritizes the record in the queue and provides additional logging. However, the manual sync page typically has a limit (e.g., 3,000 IDs).
This Postman Runner method essentially automates the process of manually adding many records to the iPaaS.com queue via API calls, bypassing the standard interface limitations.
Before You Begin: Alternative Methods
Consider these alternatives before using the Postman Runner method, as they might be simpler for your specific needs:
Polling with
/poll
Scope: If you need to import data TO iPaaS.com and the integration uses polling (not all do), you can often trigger a full data pull using the manual sync feature with a/poll
scope.
NOTE: Many polling integrations store a "last polled date." To ensure all data is captured, you may need to clear this date in the integration settings before initiating the |
Initialization Feature: Check if the specific data type and direction support Initialization within iPaaS.com, which is designed for bulk data loading.
Step-by-Step Guide to Using Postman Runner for Bulk Sync
Step 1: Prepare Your Data
To move data TO iPaaS.com (importing into iPaaS.com):
Obtain a complete list of the unique identifiers (IDs) for the records you want to import from the source system. You may receive this information via an export feature or by querying the source system's API.
NOTE: Ensure these IDs are in the exact external ID format that iPaaS.com expects for this specific integration, as documented in the subscription's documentation. |
To move data FROM iPaaS.com (exporting from iPaaS.com):
This is only necessary if you need to trigger a sync from iPaaS.com to a single specific target system, bypassing the standard outbound flow that follows an import. This updates any systems listening. However, you can turn off the FROM webhooks from other systems and have the same effect.
NOTE: Remember that TO iPaaS.com hooks will then result in FROM iPaaS.com hooks, so this is only relevant when you only need to sync from iPaaS.com to one system. |
Use the iPaaS.com API to retrieve the internal iPaaS.com record IDs you need to push. You will likely need to parse these IDs from the JSON API response.
If the data type and direction support initialization, you can use that feature as well.
Step 2: Identify the Correct Scope
The scope tells iPaaS.com what action to perform (e.g., product/create
, customer/update
, order/delete
).
The supported scopes for your integration and data type are found on the Inbound/Outbound Data Flows page within a subscription's settings or on the Manual Sync page in iPaaS.com.
NOTES:
|
Step 3: Create Your CSV File
Prepare a CSV file with at least one column,
id
.(Optional): Include a second column,
scope
. This is useful if you need to perform different actions for different records. If all records use the same scope, you can omit this column and specify the scope directly in your Postman request setup later.
The column headers must match the variable names you use in Postman (e.g., Example CSV:id,scope |
Step 4: Configuring Postman
Authentication
You need the Webhook API key associated with the iPaaS.com subscription you are working with. Find this key in iPaaS.com under: Subscription Management > Subscriptions. Find the relevant subscription in the list and click the Edit icon.
This key will be used as a Bearer Token for authentication in your Postman request header (Authorization: Bearer YOUR_API_KEY
).
API Call Details
Use the following settings for the API call details.
Method:
POST
URL: You will target the
/v2/ipaas
endpoint. The base URL depends on the environment:Staging:
https://stagingapi.ipaas.com/v2/ipaas
Production:
https://api.ipaas.com/v2/ipaas
Body: Construct the request body according to the iPaaS.comAPI documentation (refer to the Swagger page).
Here is the Swagger request body to POST a new customer.
Request Body to POST a New Customer
{
"id": "string",
"customer_number": "string",
"tracking_guid": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
"first_name": "string",
"last_name": "string",
"email_address": "string",
"company": "string",
"categories": [
{
"id": "string",
"name": "string"
}
],
"comment": "string",
"addresses": [
{
"id": "string",
"tracking_guid": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
"type": "string",
"phone_number": "string",
"first_name": "string",
"last_name": "string",
"company": "string",
"address_1": "string",
"address_2": "string",
"address_3": "string",
"city": "string",
"region": "string",
"country": "string",
"postal_code": "string",
"is_primary_billing": true,
"is_primary_shipping": true,
"customer_id": 0,
"custom_fields": [
{
"id": "string",
"type": "string",
"custom_field_name": "string",
"value": "string",
"tracking_guid": "3fa85f64-5717-4562-b3fc-2c963f66afa6"
}
],
"external_ids": [
{
"system_id": 0,
"external_id": "string",
"internal_id": "string",
"tracking_guid": "3fa85f64-5717-4562-b3fc-2c963f66afa6"
}
]
}
],
"relationships": [
{
"id": "string",
"type": "string",
"tracking_guid": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
"start_date": "string",
"end_date": "string",
"is_primary": true,
"related_to_id": "string",
"custom_fields": [
{
"id": "string",
"type": "string",
"custom_field_name": "string",
"value": "string",
"tracking_guid": "3fa85f64-5717-4562-b3fc-2c963f66afa6"
}
],
"external_ids": [
{
"system_id": 0,
"external_id": "string",
"internal_id": "string",
"tracking_guid": "3fa85f64-5717-4562-b3fc-2c963f66afa6"
}
]
}
],
"custom_fields": [
{
"id": "string",
"type": "string",
"custom_field_name": "string",
"value": "string",
"tracking_guid": "3fa85f64-5717-4562-b3fc-2c963f66afa6"
}
],
"external_ids": [
{
"system_id": 0,
"external_id": "string",
"internal_id": "string",
"tracking_guid": "3fa85f64-5717-4562-b3fc-2c963f66afa6"
}
]
}
Use Postman variables
{{id}}
and{{scope}
} (if using the scope column) as placeholders for the data from your CSV file. The exact structure will depend on the expected payload format for triggering syncs via this endpoint.
To create a call for a TO/FROM iPaaS.com webhook:
Ensure your
POST
request is saved within a Postman collection.Click File > New Runner tab, or find the Runner button in Postman.
Drag the collection containing your prepared API request into the Runner window.
Select the specific request you configured.
Click Select File and upload the CSV file you created in Step 3: Create Your CSV File.
Preview the data to ensure it's parsed correctly.
Ensure the iteration count matches the number of data rows in your CSV.
Double-check that your request includes the correct Authorization header with the Bearer Token (Webhook API Key). If you set authentication at the collection level, it should be inherited.
Confirm that the variable names in your request body (
{{id}}}, {{scope}}
) match the headers in your CSV file.(Optional): In Advanced Settings, add a delay between requests (e.g., 100-500 milliseconds). This helps prevent overwhelming the iPaaS.com queue and potentially impacting other processing tasks.
Click Start Run.
Important Considerations
|