Blueprint AI — Computer Vision & OCR for Architectural Blueprints

A computer vision and OCR pipeline that finds countertops and appliances in blueprints, reads the drawing scale, and turns pixels into real-world dimensions and cost estimates.

Focus: object detection, instance segmentation, OCR scale extraction, and measurement-driven cost estimation.

Training set: 10,000+ images
Detection: Faster R-CNN
Segmentation: Mask R-CNN
OCR: docTR

Problem

Estimating countertop material from a blueprint is slow, manual work. Someone reads the scale, measures each surface, and builds a cost sheet by hand. The goal was to automate that from the drawing itself.

What I built

Detection: Faster R-CNN trained on 10,000+ annotated blueprint images to locate countertops, sinks, and appliances with confidence scores.
Segmentation: Mask R-CNN (ResNet-50-FPN) producing pixel-accurate masks of each countertop, not just a bounding box.
Scale and measurement: docTR OCR reads the scale text, computes inches per pixel at 200 DPI, and converts each region to real dimensions, area, and a per-object material cost table.

Result

A working end-to-end pipeline with a Streamlit front end. Upload a blueprint and get detected objects, measured surfaces, and an automatic cost estimate.

Screenshots

Stage 1

Detect — Blueprint Object Detection

Streamlit demo detecting countertops, sinks, and appliances in architectural blueprints utilizing a trained Faster R-CNN model.

Stage 2

Scale — OCR Measurement Extraction

Extracting blueprint scale via OCR (docTR) to compute real-world dimensions, areas, and generate material cost estimates.

Stage 3

Segment — Instance Segmentation

Pixel-perfect instance segmentation masks isolating exact countertop shapes for area computation and material estimation, leveraging a trained Mask R-CNN model.

Stack

Python, PyTorch, Faster R-CNN, Mask R-CNN, OpenCV, docTR OCR, Streamlit.