Chap 2. Hardware

Hardware is the base for the prototype development.

Estimating your needs in the near future, evaluating the hardware options, and selecting the appropriate hardware ( the server, the data center, etc) are the basic skillsets for an MLOps developer.

Hardware, HW bottlenecks, different types of workloads (memory intensive, compute intensive, memory bandwidth intensive). CPU vs GPU vs TPU. Tools to find the bottlenecks (is it IO/CPU compute/GPU compute/ GPU mem bandiwidth / GPU mem - CPU mem bandwidth). Scaling

It various much for a different scenario, for example, if you're a lab admin, startup's tech lead, or a solo performer,

Resources in the reference:

cornell's lecture: http://www.cs.cornell.edu/courses/cs6787/2017fa/Lecture11.pdf

berkley's courses:

https://inst.eecs.berkeley.edu/~ee290-2/sp20/

MIT's course:

Tutorial on Hardware Accelerators for Deep Neural Networkseyeriss.mit.edu

Personal development: https://www.mrdbourke.com/notes-on-building-a-deep-learning-pc/

https://blog.inten.to/hardware-for-deep-learning-current-state-and-trends-51c01ebbb6dc