TABLE OF CONTENTS
- Overview
- Key Features
- Trigger Execution Framework (HTTP)
- Real-Time Execution Models
- When to Use Event-Based vs API-Based Execution
- Configuration: Event-Based Real-Time Execution
- Callback Configuration
- Best Practices
- Use Cases
- Backward Compatibility
- Important Notes and Limitations
- Prerequisites
- Scheduler & Exchange Recommendations
- Pagination & Data Volume Recommendations
- Real-Time Processing Guidelines
- Inventory Synchronization Recommendations
- Transmission Modes & File Processing
- Infrastructure Sizing Recommendations
- Known Limitations & Constraints
- Operational Constraints
- Recommended Data Volume Limits
- Unsupported Configurations & Anti-Patterns
- Performance Recommendations
- Summary
Overview
The Commerce Connector with real-time, multi-step data exchange capabilities enables organizations to build flexible, event-driven, and callback-based integrations with external systems such as ERPs, pricing engines, inventory systems, and file-based platforms.
This feature includes:
- Multi-step integration pipelines
- Real-time event triggering
- Real-time storefront callbacks
- HTTP-based execution framework
- Backward compatibility with existing scheduled exchanges
These capabilities allow Znode to automatically react to business events, retrieve live data during storefront requests, and orchestrate complex workflows across APIs, SFTP, and Blob Storage.
Key Features
Multi-Step Data Exchanges
Data Exchanges support unlimited sequential steps.
Capabilities
- Add unlimited integration steps
- Execute steps sequentially
- Configure an independent mapper per step
- Reuse data from previous steps
- Apply transformations only where required
- Reference prior step outputs using step prefixes
Example
Step1.Account.NameStep2.SalesOrderNumberSupported Transmission Modes
| Mode | Usage |
|---|---|
| API | Real-time and lightweight integrations |
| SFTP | Legacy ERP file-based integrations |
| Blob Storage | Large file exchanges |
| Import Handler | Bulk Znode imports |
Trigger Execution Framework (HTTP)
A new Trigger Execution (HTTP) section is available in Manage Data Exchange.
Trigger Execution must be enabled for all multi-step exchanges.
Supported trigger types:
- Znode Event (Push Model)
- Znode Callback (Pull Model)
Real-Time Execution Models
Znode Event – Push-Based Execution
Automatically executes a Data Exchange when a configured Znode system event occurs.
Supported Event Modules
- Account
- Order
- Product
- Category
- Price
- Inventory
- Catalog
- Portal
Typical Events
- Account Updated
- Order Created
- Product Changed
Example Use Case
Account Updated → Fetch Account Details → Transform Data → Send to ERP or SFTP
Znode Callback – Pull-Based Execution
Retrieves live pricing or inventory data from external systems during storefront requests.
Supported Callback Types
- Real-time Pricing
- Real-time Inventory
Supported Storefront Scenarios
- Real-time pricing on Product Detail Page (PDP)
- Real-time inventory availability
Callback Behaviour
- Shopper initiates storefront request
- Znode triggers callback
- Commerce Connector calls external API
- Response is transformed using Global Mapper
- Live data is returned to the storefront
Required Global Mapper Fields
- Sales Price
- Retail Price
- SKU
- Line-Item ID
When to Use Event-Based vs API-Based Execution
| Criteria | Event-Based | API-Based |
|---|---|---|
| When to Use | When event exists in Znode | When event does not exist |
| Trigger | Automatic system event | Manual API invocation |
| Dependency | Znode events | External system |
| Control | Limited | High |
| Use Cases | Order Created, Registration | Custom workflows |
Configuration: Event-Based Real-Time Execution
Prerequisites
- Administrator access
- Commerce Connector enabled
- Events configured
- External APIs reachable
Steps
- Navigate to Commerce Connector.
- Create a Data Exchange.
- Enable Trigger Execution.
- Select the required Event.
- View the Event Payload.
- Understand Step 0 (read-only event payload).
- Configure the Return Payload option.
- Configure the processing steps.
- Configure the API request.
- Map data between steps.
Example Mapping
Step2.SalesOrderName : Step0.Payload.EventDataCodeWorkflow
- Znode event occurs.
- Trigger validation executes.
- Data Exchange starts.
- Steps execute sequentially.
- Data is transformed.
- Operations execute.
- Status is logged.
Callback Configuration
- Enable the Znode Callback trigger.
- Select the required callback action (for example, Real-Time Inventory).
- Configure ERP processing steps.
Global Mapper
The Global Mapper executes after all configured steps and formats the final response returned to the storefront.
Mappings should reference the next step prefix.
Step2.QuantityAvailable : Step1.InventoryQuantityStep2.SKU : Step1.ProductSKUExecution Flow Summary
- Shopper opens Product Detail Page (PDP).
- Callback is triggered.
- Configured processing steps execute.
- ERP API is called.
- Global Mapper formats the response.
- Live data is returned to the storefront.
Best Practices
- Enable callback-based integrations wherever applicable.
- Disable legacy real-time integrations when using callbacks.
- Keep ERP APIs lightweight and responsive.
- Use caching wherever possible to reduce external API calls.
Use Cases
Real-Time Order Creation
- Trigger: Order Created
- Header and line-item steps execute.
- ERP Order API is invoked.
Real-Time Pricing
- Trigger: Product Detail Page load.
- External pricing API returns the latest price.
Account Updates
- Trigger: Account Updated.
- Customer information is synchronized with the ERP.
Multi-Step Enrichment
- Tax API
- Shipping API
- Data from multiple systems is combined before returning the final payload.
Backward Compatibility
- Existing scheduled exchanges continue to function without modification.
- No migration is required for existing scheduled integrations.
Important Notes and Limitations
Important Conditions
- Callback Data Exchanges must be enabled.
- Legacy real-time Pricing and Inventory integrations must be disabled for callback execution.
- Disabling callback exchanges reverts storefront pricing and inventory to stored Znode values.
Current Limitations
The following capabilities are currently not supported:
- Advanced retry monitoring UI.
- Pagination and aggregation in multi-step flows.
- Copying multi-step Data Exchanges.
- Custom processors in multi-step flows.
- Individual step-level data logging.
- Advanced monitoring dashboards.
Prerequisites
Before configuring Real-Time Eventing or Callback:
- Administrator access to the Znode Admin Console.
- Commerce Connector must be enabled.
- External APIs or endpoints must be reachable.
- ERP, SFTP, or Blob Storage configurations must be available.
- Legacy real-time Pricing and Inventory integrations must be disabled for callback scenarios.
Scheduler & Exchange Recommendations
Supported Scheduler Types
| Scheduler Type | Purpose |
|---|---|
| Scheduled | Automated recurring exchanges |
| OnDemand | Manual execution |
| RealTime | Event-triggered immediate processing |
Recommended Production CRON Frequencies
| Exchange Type | Recommended Frequency |
|---|---|
| Inventory Sync | Every 15 minutes |
| Order Export | Every 15–30 minutes |
| Price Sync | Every 4 hours |
| Product Import | Daily |
| Full Catalog Sync | Weekly |
Important Scheduler Guidelines
- Do not configure scheduler intervals below 5 minutes.
- High-frequency jobs may overload Hangfire queues.
- Large exchanges should be scheduled during off-peak hours.
- Long-running jobs should use dedicated queues whenever possible.
Pagination & Data Volume Recommendations
Recommended Page Sizes
| Data Type | Recommended Page Size | Maximum Recommended |
|---|---|---|
| Inventory | 500 | 5,000 |
| Orders | 200 | 3,000 |
| Products | 250 | 3,000 |
| Prices | 1,000 | 5,000 |
| Customers | 250 | 3,000 |
Pagination Guidelines
- Pagination should always be enabled for API-based exchanges.
- Page sizes above 1,000 may cause memory pressure.
- Very small page sizes increase HTTP overhead.
- Large payloads increase serialization and deserialization costs.
Recommended Processing Limits
| Parameter | Recommended Maximum |
|---|---|
| Export records per file | 5,000 |
| Export file size | 50 MB |
| API payload size | 10 MB |
| Concurrent export jobs | 3 per server |
Real-Time Processing Guidelines
Real-Time Processing Characteristics
Real-time exchanges execute as synchronous HTTP operations.
The request lifecycle includes:
- Request received
- Processor execution
- ERP/Znode API communication
- Data transformation
- Immediate response returned
Recommended Usage Patterns
| Recommended Usage | Not Recommended |
|---|---|
| Single order processing | Bulk catalog synchronization |
| Single SKU inventory lookup | Large inventory exports |
| Order acknowledgment | Full price synchronization |
| Lightweight API workflows | Large batch operations |
Recommended Performance Targets
| Metric | Recommended |
|---|---|
| Response Time | < 2 seconds |
| Payload Size | < 1 MB |
| Concurrent Requests | < 20 |
| ERP API Latency | < 500 ms |
Real-Time Operational Risks
- Real-time exchanges are blocking operations.
- Slow ERP APIs directly impact storefront performance.
- No queue buffering exists for real-time requests.
- High concurrency may saturate API thread pools.
Inventory Synchronization Recommendations
Recommended Inventory Modes
| Scenario | Recommended Mode |
|---|---|
| Product page inventory check | Real-Time |
| Warehouse inventory sync | Scheduled |
| Full inventory reconciliation | File-based |
Inventory Synchronization Risks
- Real-time inventory calls execute inline with storefront requests.
- Slow ERP APIs increase storefront latency.
- No native in-process inventory caching exists.
- High inventory traffic can overload ERP systems.
Recommended Mitigation Strategies
- Enable Redis caching for inventory lookups.
- Use short cache expiration windows (2–5 minutes).
- Prefer scheduled synchronization for large catalogs.
- Avoid bulk real-time inventory requests.
Transmission Modes & File Processing
Supported Transmission Modes
| Mode | Recommended Usage |
|---|---|
| API | Real-time and lightweight integrations |
| SFTP | Legacy ERP file integrations |
| Blob Storage | Large file exchange |
| Import Handler | Bulk Znode imports |
Recommended File Processing Limits
| Import Type | Recommended Batch Size |
|---|---|
| Product Import | 1,000–10,000 |
| Price Import | 2,500–10,000 |
| Inventory Import | 2,500–10,000 |
| Category Import | 500–1,000 |
| Promotions | 500 |
File Processing Recommendations
- Split large files into smaller batches.
- Avoid processing files larger than 50 MB.
- Prefer Blob Storage or SFTP for high-volume integrations.
- Increase timeout values for large imports where necessary.
Infrastructure Sizing Recommendations
Recommended Production Sizing
| Environment | CPU | Memory | Recommended Instances |
|---|---|---|---|
| Small | 1 vCPU | 2 GB | 1 |
| Mid-Market | 2 vCPU | 4 GB | 2 |
| Enterprise | 3+ vCPU | 8–16 GB | 3+ |
Known Limitations & Constraints
Architecture Limitations
| ID | Limitation | Impact |
|---|---|---|
| L1 | No hard-coded maximum record limit per exchange | Large exchanges may cause memory pressure. |
| L2 | Real-time processing is synchronous. | Caller blocks until processing completes. |
| L3 | Event queues may be temporary. | Events may be lost during application restart. |
| L4 | No built-in Dead Letter Queue (DLQ). | Failed events may be discarded. |
| L5 | Shared Hangfire queue. | Long-running jobs may block short jobs. |
| L6 | Pagination retry counts are partially hardcoded. | Limited runtime tuning. |
| L7 | SQL parser timeout constraints. | Large parsing jobs may fail. |
| L8 | No built-in real-time rate limiting. | ERP APIs may become overloaded. |
| L9 | SFTP reconnects for every file. | May be slow when processing many small files. |
| L10 | Real-time calls execute inline. | No buffering or asynchronous queueing. |
Operational Constraints
| Constraint | Details |
|---|---|
| Minimum Scheduler Interval | 5 minutes recommended. |
| Real-Time Payload Size | 10 MB maximum recommended. |
| Export File Size | 50 MB recommended maximum. |
| API Page Size | 5,000 recommended maximum. |
| Concurrent Real-Time Requests | More than 20 concurrent requests may impact system stability. |
Recommended Data Volume Limits
| Exchange Type | Recommended Limit |
|---|---|
| Product Import | 10,000 rows per batch |
| Price Import | 10,000 rows per batch |
| Inventory Import | 10,000 rows per batch |
| Category Import | 2,000 rows per batch |
Unsupported Configurations & Anti-Patterns
The following implementation patterns are not recommended and may negatively impact performance or system stability.
| Anti-Pattern | Risk |
|---|---|
| Real-time full catalog synchronization | API thread exhaustion. |
| Real-time bulk inventory updates | ERP overload. |
| Page sizes greater than 5,000 | Memory spikes. |
| CRON execution every 30 seconds | Hangfire queue saturation. |
| Single-node RabbitMQ deployments | Single point of failure. |
| Very large XML payloads | High memory allocation. |
| Running all workloads on a single queue | Queue starvation. |
| Large synchronous ERP calls | Storefront slowness. |
Note: The recommendations and limits documented in this article are intended as general best practices for production environments. Actual values may vary depending on infrastructure sizing, ERP performance, network latency, integration complexity, and overall system workload. These recommendations should be validated and adjusted based on your deployment requirements and performance testing.
Performance Recommendations
| Metric | Recommended Target |
|---|---|
| Response Time | < 2 seconds |
| Payload Size | < 1 MB |
| ERP API Latency | < 500 ms |
Summary
Znode’s real-time, multi-step data exchange framework enables organizations to build scalable, event-driven integrations that support real-time automation, live data retrieval, and complex multi-step workflows across APIs, SFTP, and Blob Storage. The framework maintains backward compatibility with existing scheduled data exchanges, allowing organizations to modernize integrations without impacting existing implementations.