SoftBytes
Plan Your Project
AI Automation & Agents

Self-improving invoice extraction pipeline

An invoice extraction pipeline on a fine-tuned Gemini model that retrains every week on newly processed and corrected invoices, reaching 96% extraction accuracy.

  • Sector Finance operations, Procurement
  • Client B2B operations software company
See Fine-Tuning case studiesSee Data Extraction case studies
Extraction accuracy after fine-tuning
96%
Automatic retraining on newly processed invoices
Weekly
Digital, scanned and handwritten invoices
Any format
Self-improving invoice extraction pipeline: extracted purchase order items and ERP push workflow
The starting point

The challenge

Finance and operations teams received invoices in every format: clean PDFs, scans and handwritten documents. Staff re-keyed vendors, line items, dates, totals and tax into downstream systems, and off-the-shelf extraction handled the clean documents but broke on the long tail.

The solution

What we built

  • Designed the extraction layer on Gemini 2.0 Flash, fine-tuned on the client's own invoices through Google Cloud Vertex AI.

  • Built the first model by hand: assembled and labelled the initial dataset, pushed it to Google Cloud and ran the first fine-tune.

  • Converted every invoice into page images, so multi-page, scanned and handwritten documents go through the same pipeline.

  • Automated a weekly retraining loop in which newly processed invoices, and any corrected outputs, become the next training set.

  • Returned strict, schema-defined JSON, so extracted data drops straight into ERP workflows.

Architecture

How the system works

Pipeline overview: initial fine-tuning, weekly automatic fine-tuning, and production inference to ERP output
The three pipelines: initial fine-tuning, the weekly retraining loop and production inference to ERP.

The last step loops back: each week's processed and corrected invoices fine-tune the next model version.

Engineering

How it was engineered

  • Validated every new model version before it was promoted to production.

  • When a model got a document wrong, the corrected output became training data for the next version, so accuracy compounded instead of drifting.

  • Measured extraction accuracy after fine-tuning: 96%.

Scope

What was delivered

01

Initial fine-tuning dataset and model

02

Automated weekly fine-tuning pipeline

03

Production extraction with JSON schemas

04

Model versioning and validation

05

ERP-ready export

Technology

Stack

Gemini 2.0 FlashVertex AIGoogle Cloud StoragePythonDjangoStreamlitCeleryRedis
More work

Related case studies

How it works: Question → Django API → Documents → Claude on Bedrock → Answer
AI & Automation

Private Q&A over investment notebooks

Financial services, Wealth management

A natural-language Q&A interface, built on Claude through Amazon Bedrock, over proprietary investment notebooks and financial content stored in S3.

See all AI & Automation case studies

Need something similar?

We can scope and build a system with comparable capabilities, shaped around your own workflows, data and integrations.

Plan a similar project More case studies