Freight Index Integration — Production

Automated Container Freight Index Integration

Maersk and NYSHEX exchange container freight rate data through a fully automated, bidirectional API-driven integration — powering trusted NYFI benchmarks with scheduled, auditable pipelines.

FRI 08:30
Weekly Schedule (UTC)
×2 Rates
With & Without DHC
P10P90
Percentile Depth
Business Context

Why This Integration Matters

The NYSHEX Freight Indices (NYFI) are global benchmarks for containerized freight pricing. Maersk, as a founding carrier member, provides the rate data that powers these indices.

🚢

Maersk — Founding Carrier

As the world's largest container shipping company and co-founder of NYSHEX, Maersk provides weekly container rate data from its global operations. This data — covering trade lanes, port pairs, and equipment types — forms the foundation of trusted freight indices.

Data Provider
📊

NYSHEX — Index Provider

NYSHEX generates the NYFI freight indices: transparent, independently governed benchmarks used for index-linked contracts, futures, and market analysis. Built with published methodology and oversight by an independent board.

Index Generator

API-Driven Automation

An automated, bidirectional API integration: Maersk pushes rate data to NYSHEX's NYFI PRO endpoint, and NYSHEX returns timeframe index data to Maersk's API — all through secure, auditable, scheduled pipelines.

Architecture
Stakeholder Perspectives

Different Lenses, One Integration

See how this integration serves different roles across the shipping ecosystem.

🚢

Maersk — Carrier Perspective

As data provider and co-founder, operational efficiency and market intelligence matter most.

📤 Automated Rate Submission

Rate data is extracted from Azure Blob Storage and processed automatically. No manual compilation or formatting errors. Data flows from internal systems to NYSHEX every Friday at 08:30 UTC.

📈 Market Benchmarking

Receive NYFI indices back — including percentile distributions (P10–P90), week-over-week, and month-over-month trends — to benchmark Maersk's pricing against market rates.

🔀 DHC Transparency

Every rate is split into "with DHC" and "without DHC" variants, enabling granular analysis of surcharge impact across trade lanes and container types.

🔄 Reliable Delivery

Kafka with Avro serialization, Schema Registry validation, and exponential-backoff retry ensures every data point reaches NYSHEX — with full auditability.

📊

NYSHEX — Index Provider Perspective

Reliable data inputs and trusted index generation are the foundation of NYFI's market credibility.

📥 Structured Data Ingestion

Receives rates via the NYFI PRO API in a standardized, schema-validated format — ensuring consistent, machine-readable data ingestion every cycle.

🏛️ Index Integrity

Automated, tamper-proof data pipeline with OAuth2 authentication strengthens the independent board's confidence in data quality and provenance.

📤 Timeframe Distribution

Publishes computed indices back to Maersk via POST /timeframes — including MARKET_RATE and MY_RATE types with full percentile distributions across trade lanes.

⏱️ Faster Index Publishing

Eliminates the delay between receiving data and ingestion. Rates arrive in machine-readable format, ready for immediate processing into weekly NYFI benchmarks.

🏭

Shippers & Forwarders — End User Perspective

Better data quality upstream means more reliable benchmarks for managing freight spend and volatility.

📋 Trusted Benchmarks

NYFI indices powered by automated, auditable data pipelines are more trustworthy for index-linked contracts — reducing disputes and improving transparency.

📉 Volatility Management

Weekly percentile distributions (P10–P90) and trend indicators (WoW, MoM) enable proactive freight budget planning and hedging strategies.

🌍 Trade Lane Visibility

Indices cover major trades (Trans-Pacific, Asia-Europe), sub-routes (Far East to USWC), and specific port pairs (CNNBO to USLAX) — deep granularity for procurement.

⚖️ Contract Fairness

Index-linked contracts anchored to NYFI reduce the information asymmetry between carriers and shippers, fostering a more balanced negotiation landscape.

Architecture

Interactive Data Flow

Trace the journey of container rate data between Maersk and NYSHEX. Click any node to explore its internals.

01 ☁️
Azure Blob Storage
NYSHEX folder
Single JSON file per cycle
explore →
02 ⚙️
Index Price Engine
Blob → NYFI PRO OAS
DHC split mapping
explore →
03 📡
Kafka Topic
Avro serialized
Push notification events
explore →
04 🚀
Generic Sender
Kafka consumer
HTTP POST dispatcher
explore →
05 🌊
NYSHEX NYFI PRO
POST endpoint
Index price submission
explore →
01 🌊
NYSHEX
Timeframes index data
Market & custom rates
explore →
02 🔗
Index Price API
POST /timeframes
Validates & processes
explore →
03 🗺️
Avro Mapping
REST DTO → Avro
IndexPriceEvents schema
explore →
04 📡
Kafka Topic
Finance index price
Internal distribution
explore →
05 📊
Maersk Internal
Downstream consumers
Analytics & pricing
explore →
Business Impact

Measurable Outcomes

Quantified business improvements from API-driven automation of freight index data exchange.

100%

Elimination of Manual Work

Zero manual data extraction or formatting. The scheduled pipeline handles everything end-to-end, every Friday at 08:30 UTC.

Efficiency
< 5min

Data Delivery Time

From blob extraction to NYSHEX ingestion in under 5 minutes — automated, end-to-end, with zero manual steps.

Speed
P10–P90

Statistical Depth

Full percentile distributions, WoW/MoM trends, and MARKET_RATE vs MY_RATE comparison returned per trade lane, route, and port pair.

Data Quality
3× Retry

Guaranteed Delivery

Exponential-backoff retry with Kafka persistence ensures no data loss. Every rate submission is tracked through the event pipeline with metrics.

Reliability
×2 Rates

DHC Surcharge Split

Each record produces two rate variants (with/without DHC), giving both parties granular surcharge visibility for every lane and container type.

Transparency
AVRO

Schema Governance

Confluent Schema Registry enforces data contracts. Breaking changes are caught at serialization time — not after index calculation errors.

Scalability
Use Cases

Real-World Scenarios

How different stakeholders benefit from the integrated freight index pipeline.

📦

Weekly Rate Contribution to NYFI Benchmark

Carrier

Every Friday at 08:30 UTC, the Index Price Engine automatically picks up the week's container rate data from Azure Blob Storage — covering all trade lanes, equipment types, and port pairs. It maps each record into NYFI PRO format with DHC-split logic, wraps it in Avro envelopes, and delivers to NYSHEX via Kafka + Generic Sender.

Azure Blob Price Engine Kafka NYSHEX NYFI PRO
📈

Receiving Market Index Data for Benchmarking

Analyst

After NYSHEX processes Maersk's rate contribution alongside other carriers, it returns computed timeframe indices via POST /timeframes. The Index Price API validates the payload, maps it to Avro, and publishes to the finance Kafka topic — where internal consumers access MARKET_RATE benchmarks (industry-wide) and MY_RATE indices (Maersk-specific) with full percentile breakdowns.

NYSHEX POST /timeframes Avro Mapping Kafka Internal Systems
⚖️

Index-Linked Contract Pricing

Shipper

Shippers negotiating long-term contracts with Maersk can reference NYFI benchmarks as a neutral pricing anchor. The automated pipeline ensures indices reflect the most current carrier data — updated weekly — so contract adjustments track actual market movements rather than stale or disputed figures.

Maersk Rates NYFI Index Contract Pricing
📉

Freight Rate Volatility Analysis

Risk Manager

Risk managers use the P10–P90 percentile distribution from NYFI indices to quantify freight rate volatility across trade lanes. Week-over-week (WoW) and month-over-month (MoM) trends enable early detection of rate swings, informing hedging strategies and budget forecasts before volatility impacts the bottom line.

NYFI P10–P90 Index API Kafka Risk Analytics
🔍

DHC Surcharge Impact Assessment

Forwarder

Freight forwarders comparing carrier offers can isolate the DHC (Destination Handling Charge) component using the dual-rate split. Each record comes with both "with DHC" and "without DHC" pricing, making it transparent exactly how much the surcharge adds per container type and trade lane — critical for accurate cost-plus quoting to end customers.

DHC Split NYFI Index Cost Analysis