Maersk contributes carrier rate data to the NYFI index and, in return, pulls back aggregated market intelligence — including the NYFI forward curve that prices index-linked deals. Two flows, one published benchmark.
The NYSHEX Freight Indices (NYFI) are global benchmarks for containerized freight pricing. This integration enables Maersk to transition from SCFI index deals to NYFI index deals by establishing an automated, API-driven data exchange with NYSHEX.
As co-founder of NYSHEX, Maersk provides carrier data that supports the generation of NYFI index prices at market rate. The internal team extracts loaded volumes and revenue data from HeartbeatCube (Maersk's operational reporting tool) and deposits it into Azure Blob Storage for automated processing.
Data ProviderNYSHEX generates the NYFI freight indices with transparent methodology, published standards, and oversight by an independent board. NYFI provides visibility into the prices at which container loading occurred over the prior week, serving as the foundation for index-linked contracts, benchmarks, and futures.
Index GeneratorPreviously, Maersk shared carrier data with NYSHEX through Excel files delivered via email, which was neither scalable nor consistent. Additionally, Maersk constructed its own forward curve and manually compared it against NYFI. This integration replaces both manual processes with automated API-driven data exchange.
RationaleThis integration delivers distinct value to each stakeholder across the container shipping ecosystem.
Transition from SCFI to NYFI index deals with automated data contribution and market intelligence retrieval.
The Maersk internal team extracts carrier data (loaded volumes and revenue) from HeartbeatCube and deposits the resulting JSON file into Azure Blob Storage. The Index Price Engine retrieves this file on its scheduled cron (every Friday at 08:30 UTC).
Maersk receives the NYFI forwarder curve from NYSHEX, enabling execution of NYFI index deals. This replaces the previous manual process of constructing an internal forward curve and comparing it against NYFI data.
Each rate record is submitted in two variants: inclusive and exclusive of Destination Handling Charge (DHC). This enables granular surcharge analysis across trade lanes and container types.
The pipeline employs Kafka with Avro serialization, Confluent Schema Registry validation, and exponential-backoff retry (3 attempts, 500ms to 5s interval) to guarantee data delivery with full audit trail.
Receives structured carrier data to generate NYFI indices and distributes aggregated market intelligence back to participants.
Carrier rates are received via the NYFI PRO Inbound Rates API in a validated, schema-compliant format. This replaces the previous Excel-based email process with a scalable, programmatic data delivery mechanism.
The automated pipeline with OAuth2 authentication ensures data provenance and quality, supporting the independent board's governance requirements for NYFI index generation.
NYSHEX publishes computed NYFI indices back to Maersk, including MARKET_RATE (industry aggregate) and MY_RATE (carrier-specific) metrics with full percentile distributions for forward curve construction.
Machine-readable data arrives pre-validated, eliminating manual ingestion delays. Rates are immediately available for processing into weekly NYFI benchmarks and forwarder curves.
Improved upstream data quality delivers more reliable market benchmarks for freight spend management and volatility mitigation.
NYFI indices derived from automated, auditable data pipelines provide greater confidence for index-linked contracts, reducing pricing disputes and improving commercial transparency.
Weekly percentile distributions (P10–P90) and trend indicators (week-over-week, month-over-month) support proactive freight budget planning and hedging strategy formulation.
Indices are available at multiple levels: major trade routes (Trans-Pacific, Asia-Europe), sub-routes (Far East to USWC), and specific port pairs (CNNBO to USLAX) for detailed procurement analysis.
Index-linked contracts anchored to NYFI benchmarks reduce information asymmetry between carriers and shippers, establishing a neutral pricing reference for contract negotiations.
The integration comprises two directional flows: outbound (Maersk to NYSHEX) and inbound (NYSHEX to Maersk). The inbound flow is a direct GET: the internal Maersk team calls the NYSHEX PRO API and stores the response in the datalake. Select a flow to view the pipeline, and click any node for detailed specifications.
Measurable improvements delivered by transitioning from manual Excel-based exchange to automated API integration.
The engine-driven pipeline processes data end-to-end without manual formatting or email transmission. The cron triggers automatically every Friday at 08:30 UTC.
EfficiencyFrom blob extraction to NYSHEX ingestion, the complete outbound pipeline executes in under five minutes with zero manual intervention.
SpeedMaersk receives NYFI forwarder curve data via the inbound feed, enabling automated NYFI index deal execution without manual curve construction.
Market IntelligenceExponential-backoff retry with Kafka persistence ensures zero data loss. Every submission is tracked through the event pipeline with observability metrics.
ReliabilityEach source record produces two rate variants (with and without DHC surcharge), providing both parties with granular surcharge visibility per trade lane and container type.
TransparencyConfluent Schema Registry enforces data contracts between services. Breaking schema changes are detected at serialisation time, preventing downstream processing errors.
GovernanceKey operational scenarios supported by the Maersk–NYSHEX data integration pipeline.
The Maersk internal team extracts the current week's carrier data (loaded volumes and revenue) from HeartbeatCube and deposits the resulting JSON file into Azure Blob Storage. Every Friday at 08:30 UTC, the Index Price Engine retrieves this file, maps each record into NYFI PRO OAS-compliant format with DHC decomposition (producing two rate entries per source record), and transmits the data to NYSHEX via Kafka and the Generic Sender service.
NYSHEX processes Maersk's submitted rate data in combination with broader carrier market data, then transmits the computed timeframe indices to the external-integration-index-price-api via POST /timeframes. The API validates the payload, transforms it to Avro format, and publishes to the finance Kafka topic for ingestion into the Maersk datalake. This provides both MARKET_RATE (industry aggregate) and MY_RATE (Maersk-specific) indices with full percentile distributions, enabling forward curve utilisation without manual comparison.
Shippers negotiating long-term contracts with Maersk can reference NYFI benchmarks as a neutral pricing anchor. The automated pipeline ensures indices reflect the most current carrier data, updated weekly, so that contract pricing adjustments are based on verified market movements rather than stale or disputed figures.
Risk management teams utilise the P10–P90 percentile distribution from NYFI indices to quantify freight rate volatility across trade lanes. Week-over-week and month-over-month trend indicators enable early detection of rate movements, informing hedging strategies and budget forecasts in advance of market shifts.
Freight forwarders evaluating carrier offers can isolate the Destination Handling Charge (DHC) component using the dual-rate structure. Each record is submitted with both inclusive and exclusive DHC pricing, providing transparent visibility into surcharge contribution per container type and trade lane for accurate cost-plus quotation to end customers.