Cloud Vision APIs: When to Build vs Buy

The Best Code Is Sometimes No Code

Here’s a truth that experienced engineers know well: building a custom machine learning model is often the wrong choice.

It takes weeks to collect and label data, days to train and evaluate, and ongoing effort to maintain. If a cloud API can solve your problem today, that’s usually better, especially for early-stage products, MVPs, or use cases that aren’t your company’s core differentiator.

The major cloud providers, Google, Microsoft (Azure), and Amazon (AWS), have invested billions of dollars in training vision models and packaging them as easy-to-call APIs. These models are trained on hundreds of millions of images and can detect a vast range of objects, scenes, text, faces, and content types.

The question isn’t “should I use an API or build custom?” It’s “does the API solve my problem well enough, or do I need something specific?”

What Cloud Vision APIs Offer

All three major providers offer similar core capabilities:

Label / Object Detection: detect and label objects, scenes, and activities in an image. “This image contains: dog (96%), outdoor (92%), grass (88%).”

Text Recognition (OCR): extract printed and handwritten text from images.

Face Analysis, detect faces, estimate age/gender/emotion, check for face blur/exposure.

Explicit Content Detection: flag adult or violent content. Critical for user-generated content platforms.

Safe Search, similar to explicit content, but with finer-grained categories.

Logo / Landmark Detection: identify brand logos or famous locations.

Image Properties, dominant colours, overall brightness/sharpness.

Document Intelligence: extract structured data from forms, receipts, and invoices (often a separate, higher-tier product).

Using the Google Vision API

Google Cloud Vision is the most widely used cloud vision API. Here’s how to get started.

First, set up authentication:

pip install google-cloud-vision
# Create a service account at console.cloud.google.com
# Download the JSON key file
export GOOGLE_APPLICATION_CREDENTIALS="path/to/service-account.json"

Load and call the API:

from google.cloud import vision

client = vision.ImageAnnotatorClient()

# Read image file into bytes
with open('product.jpg', 'rb') as f:
    content = f.read()

# Create an Image object with the binary content
image = vision.Image(content=content)

The vision.Image object wraps your image bytes. From here, you can call different analysis methods.

Label detection: what’s in the image:

response = client.label_detection(image=image)

print("Labels detected:")
for label in response.label_annotations:
    print(f"  {label.description}: {label.score:.1%} confidence")

Each label has a description (the detected thing) and a score (confidence, 0.0-1.0).

Text detection: extract all text:

response = client.text_detection(image=image)
texts = response.text_annotations

if texts:
    # First element contains ALL detected text as one block
    print("Full extracted text:")
    print(texts[0].description)
    
    # Subsequent elements are individual words with positions
    for text in texts[1:]:
        bounds = text.bounding_poly.vertices
        print(f"Word '{text.description}' at ({bounds[0].x},{bounds[0].y})")

Object detection with localisation: find objects and where they are:

response = client.object_localization(image=image)

for obj in response.localized_object_annotations:
    print(f"Object: {obj.name} ({obj.score:.1%})")
    
    # Bounding box in normalised coordinates (0.0 to 1.0)
    verts = obj.bounding_poly.normalized_vertices
    print(f"  Top-left: ({verts[0].x:.2f}, {verts[0].y:.2f})")
    print(f"  Bottom-right: ({verts[2].x:.2f}, {verts[2].y:.2f})")

Note the coordinates are normalised (0.0 to 1.0 of the image dimensions). Multiply by image width/height to get pixel coordinates.

The Build vs Buy Decision Framework

Use this framework every time you’re deciding whether to use an API or build a custom model:

Use a cloud API when:

• Your objects are common, cars, people, text, faces, emotions
• You need to ship quickly and don’t have labelled training data
• Your volume is moderate (under ~1-5M requests/month)
• You’re building an MVP or prototype
• You don’t have the ML engineering skills in-house

Build a custom model when:

• Your objects are domain-specific (your specific product defects, proprietary equipment, custom logos)
• You have or can acquire labelled training data
• You’re processing very high volumes and API costs become significant
• Data privacy requirements prohibit sending images to third-party servers
• You need to run offline or at the edge (no internet connection)

The grey zone: you need better accuracy than the API provides on your specific use case but don’t quite have the resources for a full custom model. In this case, fine-tuning (starting from a pretrained model and adapting to your data) is the middle path.

Cost Estimation: Know What You’re Signing Up For

Cloud vision APIs charge per request. Typical pricing (always check current rates):

Quick mental math:

• 10,000 images/month = ~$10-15/month. Trivial.
• 1 million images/month = ~$1,000-1,500/month. Meaningful but reasonable.
• 10 million images/month = ~$10,000-15,000/month. Probably worth running your own model.

For most startups and internal tools, cloud APIs are far cheaper than the engineering cost of building and maintaining a custom model. The economics shift only at high volumes.

A Practical Testing Script

Before committing to any API, test it on a representative sample of your actual data:

from google.cloud import vision
import os, json

client = vision.ImageAnnotatorClient()

def test_api_on_image(image_path: str) -> dict:
    """Run multiple Google Vision analyses on one image."""
    with open(image_path, 'rb') as f:
        image = vision.Image(content=f.read())
    
    labels   = client.label_detection(image=image).label_annotations
    text     = client.text_detection(image=image).text_annotations
    objects  = client.object_localization(image=image).localized_object_annotations
    
    return {
        'labels':  [(l.description, round(l.score, 3)) for l in labels[:5]],
        'text':    text[0].description if text else '',
        'objects': [(o.name, round(o.score, 3)) for o in objects],
    }

# Test on your sample images
for filename in os.listdir('test_images'):
    if filename.endswith('.jpg'):
        result = test_api_on_image(f'test_images/{filename}')
        print(f"\n{filename}:")
        print(f"  Top labels: {result['labels'][:3]}")
        print(f"  Objects: {result['objects']}")

Run this on 20-30 representative images before deciding whether the API meets your needs.

Exercise: If you have a Google Cloud account (they give $300 free credit), call the label detection API on 5 images from different domains: a street scene, a product, a document, a food photo, and something from your specific industry. Are the labels accurate and useful for your use case? If not, what would a custom model need to know that the API doesn’t?