Azure Document Intelligence/Data Engineer

Description:

Role Summary

This role owns the document ingestion and extraction pipeline: turning scanned forms, PDFs, and other unstructured records into clean, structured, validated data ready for indexing, search, and downstream workflows. The person in this role builds the confidence-scoring and human-in-the-loop review mechanisms that keep extracted data trustworthy at scale.

Key Responsibilities

• Build and maintain document ingestion pipelines using Azure AI Document Intelligence (Form Recognizer).
• Implement OCR, document classification, and metadata/field extraction for scanned PDFs and mixed-format records.
• Design confidence-scoring logic and exception/review queues for low-confidence extractions.
• Build batch processing pipelines using Azure Functions, Logic Apps, or Data Factory.
• Model and maintain extraction and metadata data structures in SQL.
• Implement data quality checks, duplicate detection, and validation rules across the pipeline.
• Partner with the Search Engineer to ensure extracted data feeds cleanly into indexing.

Requirements:

Required Qualifications

• Production experience with Azure AI Document Intelligence / Form Recognizer.
• Strong OCR and document classification experience, including scanned/handwritten forms.
• Experience with confidence scoring and human-in-the-loop validation design.
• Proficiency in Python and SQL for data processing and pipeline development.
• Experience with Azure Functions, Logic Apps, or Data Factory (or equivalent batch/serverless tooling).
• Demonstrated experience handling data quality issues: duplicate detection, exception handling, reconciliation.

Preferred Qualifications

• Experience with LLM-assisted structured extraction (JSON schema outputs, prompt-based field extraction).
• Healthcare, insurance, legal, or government document-processing background.
• Familiarity with HIPAA-, FedRAMP-, or similarly regulated data-handling practices.