Conference proceeding
An LLM-Based ETL Architecture for Semantic Normalization of Unstructured Data
2025 IEEE MIT Undergraduate Research Technology Conference (URTC), pp 1-5
10 Oct 2025
Abstract
Automating extract-transform-load (ETL) pipelines for scanned business documents typically demands costly, finetuned, layout-aware models. We present a cloud-native architecture that transforms heterogeneous documents into a unified, structured JSON schema-without any model fine-tuning. Our pipeline combines off-the-shelf optical character recognition (OCR) (Azure Document Intelligence) with a schema-constrained large language model (LLM), guided by type-checked Pydantic [1] outputs and a one-pass swap heuristic for efficient fewshot prompting. Evaluated on the FUNSD (form) and CORD (receipt) corpora, the system achieves 0.60 (FUNSD) and 0.83 (CORD) fuzzy KV-F1 scores, respectively, while processing each page in under eight seconds at under0.004 on standard cloud quota. Scaling to a larger LLM boosts the CORD accuracy to 0.89 F1 at under 0.02 per page. The entire pipeline-code, prompts, and metric scripts-is open-sourced, enabling lightweight, fullydeployable semantic ETL for small-to medium-scale workloads.
Metrics
1 Record Views
Details
- Title
- An LLM-Based ETL Architecture for Semantic Normalization of Unstructured Data
- Creators
- Shreyan Gupta - Drexel University
- Publication Details
- 2025 IEEE MIT Undergraduate Research Technology Conference (URTC), pp 1-5
- Publisher
- IEEE
- Number of pages
- 5
- Resource Type
- Conference proceeding
- Language
- English
- Academic Unit
- College of Computing and Informatics
- Scopus ID
- 2-s2.0-105041886466
- Other Identifier
- 991022197287904721