AI Tariff Intelligence Platform

Summary

This document describes the design and implementation of an Intelligent Tariff Processing System built using Microsoft Fabric, Azure AI, and OpenAI services. The proposed solution automates the extraction, validation, and management of tariff elements to form regulatory documents, supporting supervised learning workflows, and preparing high-quality, simulation-ready data.

Problem Statement

The tariff and regulatory documents are normally provided in highly unstructured forms and may include complex legal terms, conditional regulations, and frequent changes. The interpretation and implementation of these documents are essential and time-consuming tasks for organizations, where even a small mistake can lead to serious consequences.

With the increasing number of tariff documents and the rate of changes in the regulations, the manual and semi-automated processes cannot provide consistent and timely results.

Traditional tariff processing systems are completely dependent on human interpretation of unstructured PDFs, which results in inconsistent output, limited auditability, and latency in downstream analysis. To address these challenges, the proposed solution uses AI-powered intelligence combined with human validation and supervised learning workflows to achieve accurate, traceable, and simulation-ready tariff data at scale.

Business Objectives

• Automate Tariff Extraction:
  Reduce manual efforts by leveraging AI-powered document intelligence to extract tariff rules and elements from unstructured PDF documents.
• Ensure Accuracy:
  Achieve high extraction accuracy through structured validation workflows and continuous improvements via supervised learning.
• Enable Simulation:
  Deliver curated, versioned, and simulation-ready tariff elements to support downstream analytics and scenario modeling.
• Maintain Compliance:
  Track complete document lineage, version history, and audit trails to support compliance and governance requirements.
• Scalability:
  Build on Microsoft Fabric to handle growing volumes of tariff documents and complex regulatory changes.

Solution Architecture

The solution comprises six core stages orchestrated through Microsoft Fabric Pipelines and Notebooks:

1. Document Ingestion:
   Tariff PDF files are uploaded by users via UI to initiate the tariff processing pipeline.
2. Text Extraction:
   Azure AI Document Intelligence extracts text and layout information from the uploaded documents.
3. Rule Extraction:
   Azure OpenAI analyzes the extracted text to extract tariff-related business rules, conditions, and constraints.
4. Element Generation:
   The extracted rules are converted into standard tariff element models through LLM-based processing.
5. Validation & Feedback:
   Extracted tariff elements are validated through a UI-driven validation process, and user feedback is collected to enable supervised learning and improvement.
6. Simulation Preparation:
   Users select tariff elements which are stored for use in subsequent tariff simulation and analysis tasks.

Key Components

1. Data Ingestion & Storage

• OneLake Storage:
  Centralized lakehouse for raw PDFs, extracted text, and intermediate data
• Fabric Pipeline:
  Orchestrates document upload, processing triggers, and data movement
• Versioning:
  Maintains timestamped history of all tariff documents and extracted elements

2. AI-Powered Extraction

• Azure AI Document Intelligence:
  OCR and layout analysis for complex tariff PDFs
• Azure OpenAI:
  GPT-based extraction of business rules, conditions, and tariff logic
• Fabric Notebook:
  Data cleaning, chunking, and prompt engineering for optimal LLM performance

3. Data Models & Persistence

• Business Rules Table:
  Extracted rules with metadata (source, confidence, version)
• Tariff Elements Table:
  Normalized element models (rates, conditions, applicability)
• History Tables:
  Immutable audit log of all document versions and element changes

4. User Interfaces

• Validation UI:
  Compare expected vs. actual extracted elements, flag errors, and provide corrections
• Simulation Selection UI:
  Browse and select tariff elements for downstream simulation workflows
• App Service:
  Hosted on Azure App Service for secure, scalable access

5. Feedback Loop

• Supervised Learning Store:
  Capture user corrections and validation outcomes
• Prompt Refinement:
  Iteratively improve LLM prompts based on feedback
• Accuracy Tracking:
  Monitor extraction accuracy per document type and adjust models

Process Flow

The following diagram illustrates the end-to-end tariff extraction and processing pipeline using Microsoft Fabric:

Parallel Processes

• Archival:
  Pipeline archives each document version to History Tables
• Version Control:
  Elements are versioned and linked to source document versions

Technology Stack

Data Models

Tariff Elements Table

• element_id (PK), tariff_document_id (FK), element_type, rate_value, condition,
  applicability_rule, effective_date, confidence_score, created_at, version

Business Rules Table

• rule_id (PK), tariff_document_id (FK), rule_text, rule_category, extracted_by,
  confidence_score, validated, created_at

Tariff History Table

• history_id (PK), tariff_document_id, document_version, upload_date,
  processed_date, status, user_id, file_path

Feedback Store

• feedback_id (PK), element_id (FK), user_correction, feedback_type, timestamp, user_id

User Workflows

Workflow 1: Document Upload & Extraction

1. User uploads tariff PDF via web interface.
2. Fabric Pipeline triggers the Document Intelligence extraction process.
3. Extracted text stored in OneLake, archived for audit and version control.
4. Notebook prepares data and prompts Azure OpenAI.
5. Business rules and elements are stored in respective tables.

Workflow 2: Validation & Feedback

1. User accesses Validation UI to review extracted tariff elements.
2. System displays extracted elements with confidence scores.
3. User compares against expected results and provides corrections as needed.
4. User feedback and validation are stored in feedback repository, which is used
   to refine prompts and trigger re-extraction for low-confidence documents.

Workflow 3: Simulation Selection

1. User browses validated tariff elements in Selection UI.
2. Tariff elements can be filtered by date, type, applicability, and version.
3. User selects the tariff elements required for simulation scenario.
4. Selection elements are saved to the simulation store.
5. Optionally triggers a simulation pipeline.

Quality Assurance & Feedback Loop

• Supervised Learning:
  Continuous improvement through controlled user validation and correction.
• Confidence Scoring:
  Azure OpenAI provides confidence scores for extracted rules, with low-confidence
  items flagged for review and re-extraction.
• A/B Testing:
  Test multiple prompt strategies and extraction approaches to determine the most
  effective approach.
• Metrics Tracking:
  Track key metrics such as extraction accuracy, processing time, and user correction rate.
• Model Tuning:
  Regularly update prompts and extraction logic based on accumulated feedback,
  validation results, and metrics.

Future Enhancements

• Automated Classification:
  Pre-classify tariff documents by type for specialized extraction logic and processing paths.
• Multi-Language Support:
  Expand extraction and validation functionality to include non-English tariff documents.
• Real-Time Processing:
  Facilitate stream-based ingestion and processing for high-velocity tariff updates and changes in real time.
• Advanced Simulation:
  Integrate tariff element store directly with tariff simulation engines to support large-scale scenario analysis.
• Regulatory Alerts:
  Implement automated detection of conflicting rules and potential issues during extraction and validation.

Success Metrics

Deployment & Operations

• Environment:
  Microsoft Fabric Workspace with dedicated capacity
• Access Control:
  Azure AD integration for role-based access
• Monitoring:
  Fabric Pipeline monitoring and Application Insights for UI
• Backup & Recovery:
  OneLake automatic snapshots and time travel
• Compliance:
  GDPR-compliant data handling and audit logs for all operations

Conclusion

This solution delivers a fully automated, AI-driven approach for tariff processing that significantly reduces manual effort and increases efficiency. By combining Microsoft Fabric’s unified analytics capabilities with Azure AI services, the solution provides a scalable and transparent system for extracting, validating, and managing complex tariff data with supervised learning.

Organizations can improve decision-making through this foundation, which also enhances automated simulations and classification, ensuring long-term adaptability as business needs evolve.

Document Version: 1.0

Date: January 27, 2026

Author: Vincent Susai