[Programming question]: MLOps

[haesleinhuepf]

I'm preparing a semester (14 lectures) about MLOps for students who already know the basics about machine learning and deep learning. Which contents should these lectures cover? Note: it is supposed to be an applied lecture where students learn how to do things with python.

Git-bob answer

[github-actions]

^{This message was generated by git-bob (version: 0.27.2, model: openai:gpt-5-2025-08-07, log), an experimental AI-based assistant. It can make mistakes and has limitations. Check its messages carefully.}

Great topic. Here’s a concrete, applied 14-lecture plan with weekly hands-on deliverables and a coherent Python-first tooling stack that runs locally on student laptops.

Baseline stack (kept minimal, runs offline/local):

• Python 3.10+, conda, pip-tools or Poetry
• scikit-learn or PyTorch (choose one and stick to it; I’d default to sklearn for speed)
• Git + pre-commit + black + isort + flake8 + mypy + pytest
• DVC for data/versioning (remote: local dir or MinIO S3)
• MLflow (tracking + model registry, backend: SQLite, artifacts: local or MinIO)
• Prefect 2 for orchestration
• FastAPI + Uvicorn for serving
• Docker + docker-compose (optional: devcontainers)
• Great Expectations and Evidently for validation/monitoring
• Optional advanced: Feast (feature store), Prometheus/Grafana, Kubernetes (kind/minikube)

Semester-long project

• One dataset/problem per team (classification or regression). By the end: a reproducible repo with data pipeline, training pipeline, CI, model registry, containerized service, monitoring, and a short postmortem.
• Milestones are aligned with lectures below.

Lecture 1. What is MLOps + local stack setup

• Outcomes: Why MLOps, lifecycle, roles, failure modes, stack overview.
• Lab: Create repo, conda env, pre-commit hooks, basic Makefile, run hello-world notebook and script.
• Deliverable: Repo skeleton committed; dev env verified.

Lecture 2. Reproducibility and configuration

• Outcomes: Env pinning, seeds, deterministic training, config management.
• Lab: Pin env (conda-lock or poetry lock), Hydra for configs, Pydantic models for I/O schemas.
• Deliverable: Reproducible training run with a config file baseline.

Lecture 3. Data management and versioning

• Outcomes: Data lineage, splits, DVC basics, storage backends.
• Lab: Add raw -> processed pipeline, DVC track datasets, set remote (local dir or MinIO).
• Deliverable: DVC pipeline with locked version of data and documented split strategy.

Lecture 4. Data quality and validation

• Outcomes: Data contracts, tests, schema drift.
• Lab: Great Expectations suite, basic data unit tests with pytest, small golden sample dataset.
• Deliverable: CI passes data checks on pull request.

Lecture 5. Experiment tracking

• Outcomes: MLflow tracking, metrics/params/artifacts, experiment hygiene.
• Lab: Log metrics, params, figures; compare runs; export best model artifact.
• Deliverable: MLflow experiment with at least 5 runs and a selected best model.

Lecture 6. Project structure and testing for ML

• Outcomes: Modular code, fast tests, fixtures, smoke tests on small data.
• Lab: Refactor notebooks to src package; add unit/integration tests; pre-commit formatting.
• Deliverable: >80% test coverage on core utils; tests run under 2 minutes locally.

Lecture 7. Orchestration and pipelines

• Outcomes: Task graph, retries, caching, artifacts handoff.
• Lab: Prefect flow for data->train->evaluate->register; parameterized runs.
• Deliverable: One-click pipeline execution with cached steps and CLI params.

Lecture 8. CI for ML with GitHub Actions

• Outcomes: CI strategies for data-dependent workflows, caching, secrets.
• Lab: Workflow that installs env, pulls small DVC sample, runs tests, lints, and builds Docker image.
• Deliverable: Passing CI badge; artifacts uploaded for inspection.

Lecture 9. Model packaging and registry

• Outcomes: Reproducible packaging, versioning, model promotion.
• Lab: MLflow model flavors/pyfunc; register model; stage transitions with rules.
• Deliverable: Model in “Staging” with metadata (dataset hash, metrics, params).

Lecture 10. Serving: batch and real-time

• Outcomes: Service contracts, input/output schemas, latency vs throughput.
• Lab: FastAPI service that loads MLflow model; request/response Pydantic models; Dockerize.
• Deliverable: docker-compose up exposes a /predict endpoint with health checks.

Lecture 11. Monitoring and observability

• Outcomes: Data drift, performance drift, live metrics, logging/tracing.
• Lab: Evidently for drift reports; basic request logging; optional Prometheus metrics endpoint.
• Deliverable: Automated drift report on a held-out or time-split dataset.

Lecture 12. Deployment strategies

• Outcomes: Environments, blue/green/canary, rollbacks, infra as code.
• Lab: Local “staging” and “prod” via docker-compose profiles; optional: kind/minikube deploy.
• Deliverable: Documented deploy procedure with rollback steps.

Lecture 13. Security, governance, and responsible AI

• Outcomes: Secrets management, supply chain, PII handling, model cards, lineage.
• Lab: Secrets in GH Actions, dependency scanning, basic model card, lineage via DVC/MLflow links.
• Deliverable: SECURITY.md + MODEL_CARD.md + risk checklist.

Lecture 14. Capstone, SLAs, incident response, and postmortems

• Outcomes: Operability, SLOs/SLAs, on-call basics, postmortems.
• Lab: Load test the service; trigger a controlled failure; write a short incident report and postmortem.
• Deliverable: Final demo + repo + postmortem.

Assessment

• Weekly deliverables (40%)
• Midterm checkpoint (20%): working pipeline to model registry with CI
• Final project (40%): E2E system, demo, documentation, monitoring, and postmortem

Starter repo checklist

• src/, tests/, notebooks/, configs/, data/ (DVC), docker/, .github/workflows/
• Makefile targets: setup, lint, test, train, pipeline, serve, docker-build, docker-up
• Pre-commit config with black/isort/flake8/mypy
• MLflow tracking server config (local), DVC remote config (local/MinIO)
• Example Prefect flow and FastAPI app with schemas
• Sample Evidently/GE reports scripts

Happy to provide a minimal starter repo structure or a notebook-based template if you want to standardize the stack for students.

[haesleinhuepf]

Awesome! Go ahead and create the starter repo in the current repository / folder with powerpoint slides and example notebooks for each lesson. Make sure that notebooks install requirements using pip in one of the first code cells.

git-bob implement this