Drawing analysis — structuring with VLM and OCR

The problem

Drawings carry a lot of information, but not in a form a machine can use. Before they can be searched or generated against, they have to be read and structured. This is a different problem from document retrieval — it is about solving what is drawn, as an image.

Design decisions

• Built around reading and structuring visual information rather than combining retrieval and generation
• PDF conversion depends on external binaries, so the runtime environment was pinned declaratively, producing the same result locally and in the execution environment
• Introduced CI first, so verification ran on every change, then added features on top
• When connectivity to the customer environment failed, isolated which layer it stopped at and made clear which side of the boundary each issue fell on

Approach

• Built around reading and structuring visual information rather than combining retrieval and generation
• PDF conversion depends on external binaries, so the runtime environment was pinned declaratively to keep results reproducible
• Introduced CI first, so verification ran on every change before features were added
• When connectivity to the customer environment failed, isolated which layer it stopped at and made the boundary of responsibility explicit