How to Offer Customer-Hosted Document AI Without Maintaining Two AI Stacks

Description and use

By consolidating model calls behind a single client, configuring targets via settings, pinning models, running unified evaluations, and aligning release cadence, the vendor eliminated duplicated code and releases, reduced support complexity, and enabled a clear, documented support boundary while keeping documents within the customer’s environment.

Problem

Software vendors need to support both SaaS and customer‑hosted deployments of document AI (OCR, extraction, embeddings, etc.) without maintaining separate AI stacks, which leads to duplicated code, duplicated releases, and duplicated support.

Outcome

By consolidating model calls behind a single client, configuring targets via settings, pinning models, running unified evaluations, and aligning release cadence, the vendor eliminated duplicated code and releases, reduced support complexity, and enabled a clear, documented support boundary while keeping documents within the customer’s environment.

Workflow

• 1. Create a single inference client that exposes one interface for all model calls (OCR, extraction, embeddings, reranking, generation).
• 2. Ensure no vendor SDKs are imported outside the client and that request/response shapes are identical across targets.
• 3. Configure deployment target via settings: endpoint, credentials, model identifiers.
• 4. Maintain a model manifest per target, pinning exact model versions and recording substitutions.
• 5. Run a unified, labeled evaluation suite against every target before each release and record results per target.
• 6. Release on a single cadence, packaging application, inference software, and model manifest together; provide offline packages for customers without internet access.
• 7. Define and document the support boundary between vendor and customer responsibilities.
• 8. Deploy SIE Cloud for SaaS and self‑hosted SIE for customer installations, using the same client API with different endpoints.

Evidence

Put every model call behind one inference client and choose the deployment target by configuration, not code.

Source

The deal was signed on one condition: the software had to run in the customer's own cloud account, because the documents were contracts and the contracts could not leave.

Source

Two rules make this work: No vendor SDK outside the client. The same request and response shapes in every target.

Source

Run one evaluation suite against every target before every release.

Source

Set one release train for the product and treat the customer install as a delivery channel for it.

Source