What Does the MLOps Deployment Lab Actually Look Like?
Four students in our April 2025 cohort made it all the way to week 12 with a trained model they were proud of and absolutely no plan for how to serve it. That's not a knock on them. ML model deployment is the part most online courses skip, wave at vaguely, or reduce to a Colab share link. We don't do that.
The MLOps & AI Infrastructure course ends with a live deployment exercise inside our MLOps Deployment lab environment. Students take a real trained model, containerize it, and ship it to Cloud Run. The whole thing runs in one three-hour session. This post is a walkthrough of what that lab covers and why we built it the way we did.
Why Cloud Run Beats Kubernetes for Your First ML Deployment
Cloud Run is the right starting point for serverless ML deployment because it removes the cluster management overhead that trips up new MLOps engineers. You write a container, define a port, set a few environment variables, and Google Cloud handles the rest. No Kubernetes YAML archaeology on day one.
The real tradeoff is cold starts. We actually measure them during the lab. A clean 500MB image on Cloud Run cold-starts in roughly 1.2 to 2.8 seconds. Get that image under 400MB and you're reliably under 1.5 seconds. We walk students through multi-stage Docker builds specifically for this reason, trimming dev dependencies and keeping only what the inference path needs.
For internal tools, demo APIs, or anything below a few hundred requests per minute, Cloud Run's scale-to-zero behavior is genuinely useful. One student's capstone project was a churn prediction model for a local SaaS business. Running it on an always-on VM cost around $380/month. Cloud Run dropped that to under $20. Real number, from a capstone review in May 2025.
The Lab Setup: FastAPI, Docker, and Artifact Registry
We use FastAPI with Uvicorn as the serving layer. Not Flask. Flask's synchronous request handling becomes a bottleneck fast once you're past 40 or 50 concurrent prediction requests, and students deserve to learn the pattern that actually scales.
Here's the core of the prediction endpoint students write in the lab:
from fastapi import FastAPI
from pydantic import BaseModel
import joblib
import numpy as np
app = FastAPI()
model = joblib.load("/app/artifacts/model.pkl")
class PredictRequest(BaseModel):
features: list[float]
@app.post("/predict")
async def predict(req: PredictRequest):
X = np.array(req.features).reshape(1, -1)
prediction = model.predict(X)
return {"prediction": int(prediction[0])}
@app.get("/health")
async def health():
return {"status": "ok"}
Simple on purpose. The /health endpoint matters because Cloud Run uses it for startup probes. Students who skip it spend 20 minutes debugging why their service shows "not ready" even though their prediction logic is fine.
The Dockerfile the lab provides uses a python:3.11-slim base, copies in only the requirements and app files, and exposes port 8080. Cloud Run expects 8080. That's one of those small things that burns 30 minutes if nobody tells you upfront.
After building locally and testing with docker run -p 8080:8080, students push to Artifact Registry and deploy with:
gcloud run deploy ml-predict \
--image us-central1-docker.pkg.dev/PROJECT_ID/ml-repo/predictor:v1 \
--region us-central1 \
--memory 2Gi \
--cpu 2 \
--max-instances 10 \
--allow-unauthenticated
The --allow-unauthenticated flag is for lab purposes only. In the next section students replace it with IAM-based authentication. We make a point of this because shipping an open prediction endpoint to production is a bad habit to start with, and it's surprisingly easy to forget under time pressure.
Where Students Get Stuck When They Deploy Machine Learning to Google Cloud
Deploying a machine learning model to Google Cloud sounds clean in a tutorial. In practice, the same things break every cohort.
Model artifact paths come up first. Students who didn't complete the week 10 versioning lab properly have artifacts sitting in random local folders with hardcoded paths that simply don't exist inside the container. We added a preflight checklist to week 11 specifically to catch this before week 12. It has helped a lot.
Dependency bloat is the second one. Someone installs the full torch package (2.4GB) because that's what they trained with, but their inference path only needs torch.nn and a state dict load. The image balloons, cold starts get painful, and Cloud Run sometimes times out on startup. The fix is either to export to ONNX for CPU inference or to pull torch with --extra-index-url to grab the CPU-only wheel. We cover both paths.
The third issue is subtler. Students forget to set a minimum instance count on anything that's supposed to feel responsive. Scale-to-zero is great for cost. It is genuinely bad for a demo where your first request arrives cold in front of a client. That's a conscious configuration choice, not a bug, but students don't always realize they're the ones who get to make it.
Load Testing and Monitoring Before You Call It Done
The lab doesn't end at gcloud run deploy. We run Locust against the live endpoint, 50 simulated users ramping over two minutes, and look at Cloud Monitoring latency percentiles. P50 and P99 tell very different stories. A model that responds in 180ms at median can blow out to 4 seconds at P99 under load if the container is undersized. I've seen students genuinely surprised by that gap. Every time.
Students also wire up a basic alert: if P99 latency exceeds 3 seconds for two consecutive minutes, fire a Pub/Sub notification. It takes about 15 minutes in the Cloud Console. The point isn't the specific threshold. It's that shipping without observability isn't shipping, it's hoping.
If you're working through MLOps & AI Infrastructure and want to run this yourself, the MLOps Deployment lab environment has everything pre-configured: the model artifacts, the starter FastAPI app, the Dockerfile, the Locust test script. You bring the Google Cloud project.
The question we end week 12 with isn't "did your model deploy?" It's "do you know what it will do under load next Tuesday when you're not watching it?"
