Coming soon. This course is planned; lessons and labs are not available yet. No release date has been announced.
Run a small inference endpoint, inspect queue and response behavior, then recover from an artifact or runtime mismatch.
Intermediate ยท Inference service operations
Know Linux services, HTTP APIs and basic Python. The planned baseline runs on a CPU host.
The planned project asks you to operate a small endpoint, explain its latency and recover useful responses after a runtime or artifact change.
A CPU-hosted service provides the baseline. GPU diagnostics are optional and require compatible hardware; model quality is assessed separately.
3 planned chapters and 6 lesson outlines.
Separate endpoint behavior from claims about the model's quality.
Planned task: Specify input shape, output, failure response and a client latency budget.
Changed case to investigate: A new input shape arrives; reject or handle it explicitly rather than silently changing meaning.
Planned task: Pin the service code, inference runtime, dependencies and model artifact for a second deployment.
Changed case to investigate: A library version changes on one host; isolate the compatibility failure.
Measure queueing and compute under a controlled CPU baseline.
Planned task: Trace requests through admission, worker execution and response with measured timings.
Changed case to investigate: A short burst increases queue delay while compute time stays steady; revise the diagnosis.
Planned task: Compare latency and throughput under two batch settings while preserving response checks.
Changed case to investigate: An interactive client has a tighter deadline; retest the batch policy.
Restore a useful endpoint and keep optional accelerator claims tied to real hardware.
Planned task: Restore a known runtime/artifact pair and verify the API contract and operational signals.
Changed case to investigate: The artifact changes but the endpoint still returns HTTP 200; check shape and version, not just status.
Planned task: On a compatible device, compare driver and runtime evidence before enabling acceleration.
Changed case to investigate: The workload falls back to CPU; report actual device use without inventing GPU performance or model quality.