Skip to content
← Back to work

Manufacturing Data Extraction Pipeline

Reading customer engineering packages with a vision-language model, and writing the results straight into the systems that drive production.

Role

Sole engineer — design through production

Context

750+ person PCB manufacturer

Status

In production

Verified by

Manual audit · 1,000+ production jobs

Results

98.8%

Extraction success · n = 1,000+ production jobs

Accuracy measured by manual audit: quoting engineers verified pipeline output against the source packages across 1,000+ production jobs.

~280

Jobs per week · Sustained in production

30 → 5 min

Entry time per job · Remaining time is source-linked verification

At ~280 jobs a week that is roughly 117 hours returned to the team — an estimated ~2.9 FTEs, derived from the measured per-job saving rather than counted directly.

Figure

Simulation of the extraction pipeline, using synthetic data, in four stages. One: intake. A customer package arrives as a folder of 18 files of mixed kinds — drawings, spreadsheets, an archive, notes, an email thread, a photograph. Nobody opens the folder; an order landing in the ERP puts the package on a queue that hands it to the extractor. Two: identify. The system works out which three files matter from their contents rather than their names — the fab drawing, the stackup and the Gerber data. Three: extract. Six specifications are read out of those files, and each value is shown with the file it came from. Material FR-4, surface finish ENIG and minimum trace and space 0.10 mm come from the fab drawing alone. Three more are checked against a second file: layer count 8 agrees between the fab drawing and the Gerber data (eight copper layers in the file set) and board thickness 1.60 mm agrees with the stackup, and copper weight does not agree — the fab drawing reads 1 oz where the stackup reads 2 oz, so that one is flagged for a person. Four: publish. The five confirmed values are written to the ERP and the CAM system automatically, and the flagged one waits for a person. Nobody retypes anything. End state: 18 files in, 3 identified, 6 specs extracted, 1 flagged for review.

This visual demo shows how the extraction tool works — all running on an on-prem NVIDIA DGX Spark box hosting a 122B vision-language model.

Illustrative simulation with synthetic data. Not a customer package.

The problem

Every order arrives as a package: a fab drawing, a stackup, and the CAD data that describes the board itself. Before anything can be built, someone has to turn that package into structured data — roughly thirty manufacturing spec fields per job, typed into the systems that drive production.

That work fell to quoting engineers, and in practice they read the drawing and the stackup. The CAD data is not something a person reads: Gerber describes copper geometry, not specifications. So the human pass was always working from a subset of what the package actually contains.

It ran about thirty minutes per order, it was tedious enough to invite mistakes, and because it sat at the very front of the process, it set the pace for everything behind it. Throughput was limited by how quickly people could read documents and type.

The obvious move — hand the package to a capable model and let it read — was the one move the environment did not allow. So the interesting problem was never extraction accuracy on its own. It was building something accurate enough to trust, inside a network nothing is allowed to leave.

Constraints

  • ITAR: no cloud APIs, and nothing customer-related leaves the controlled network. Every hosted model was off the table before design started.
  • It had to fit the systems already running the floor — the ERP that receives orders, the SQL Server behind it, and a CAM system that consumes XML.
  • Accuracy had to beat a person doing the same task. Anything less and engineers would re-check every field, which costs more than doing it by hand.

How I built it

The pipeline starts where the work actually starts: an order landing in the ERP triggers it. From there it pulls every part of the package — the drawing, the stackup, and the CAD data — including the sources a person was never going to read. Reading more of the package than a human can is the point: the same spec is often stated in more than one place, and disagreement between those places is the most useful signal available.

None of that path is interactive. The order and its package are written to a database, and an intake spooler works the queue one job at a time, continuously — so a week of orders moves through a single box without anyone scheduling anything, and each job has the whole machine while it runs.

Extraction runs against the on-prem vision-language model. Each field comes back with its source, and the readings are cross-validated against each other before anything is written down. Values that agree pass through. Values that disagree are the ones a human should look at, which is a far better use of an engineer than typing values that were never in question.

Then the part that actually bought the time back: the review interface. Most of a fab drawing is irrelevant to a quote — roughly ninety-five percent of it — and a quoting engineer is looking for a specific handful of attributes buried in it. So the original thirty minutes was mostly search, not judgement: reading a long document to find a few values.

Because extraction records where every value came from, the interface can show each extracted value beside its source and point the engineer straight at the callout or line it was read from. The job changes from hunting for values to confirming citations. Thirty minutes of reading became five minutes of checking.

That checking step is deliberate, not residual. At 98.8%, roughly one value in eighty is wrong, and a wrong value on a fab package is expensive. The system was built to make verification fast, not to remove the verifier.

The confirmed result is written to SQL Server as structured data, and a headless spooler renders the XML the CAM system expects. No parallel process to maintain — the data shows up where it always had to be, and the humans moved from transcription to audit.

What I’d do differently

I would invest in a labeled regression set much earlier. For too long, accuracy changes were something the team felt rather than measured — and a number you cannot measure is a number you cannot defend.

Stack

Python · vLLM · vision-language models · review UI · Docker · SQL Server · XML integration · ERP integration · isolated-network deployment