in the later videos I used an LLM to generate bg slides as its more useful than explaining the concept with no visual, check out the notes tho no ai there
I filmed over 6+ Hours of footage and hand wrote 85+ pages of notes to teach absolute beginners about AI Inference.
Over the last 2 months, whilst working as a full-time SWE intern, I was grinding after hours to create this video series. I learned to code, amongst many other things, through YouTube videos aimed at beginners. Now, with the rise of AI, these videos tend to show up less, so I decided I wanted to change that.
This is . A lot of thinking and consideration went into creating this, and all of my notes are written by me. For transparency almost all code was also written by me with exception of small bug-fixes, readme updates and stuff like that. There is also extra notes pertaining to each video which are AI generated, but main notes are all me.
The series walks you through building a small Llama 3.2 1B inference engine in C++ and CUDA, starting with what happens when you send an LLM a prompt.
I cover:
How text becomes tokens, and tokens become embeddings.
Loading model weights into GPU memory.
CUDA basics, writing your own GPU kernels, and matrix multiplication with cuBLAS.
The transformer step by step: RMSNorm, RoPE, attention, and the SwiGLU feed-forward network.
Prefill and decode: processing your prompt and generating a response one token at a time.
KV caching and how it saves repeated computation.
Static and continuous batching to handle multiple requests.
Paged KV caching and paged attention to manage and read cached data on the GPU.
Here is the github repo as well as a link to my youtube:
I’d also add, software engineering pros get there in a lot of ways. That’s what is so cool about the field. But whenever I see pure technical grind, I always have respect. It is in many ways the essence of the field.
These slides are obviously generated by an llm.
I think those are just the video thumb nails. There seems to be a lot of effort in the videos :)
in the later videos I used an LLM to generate bg slides as its more useful than explaining the concept with no visual, check out the notes tho no ai there
This embodies laziness. Please write it yourself. LLM slop is not desirable.
I filmed over 6+ Hours of footage and hand wrote 85+ pages of notes to teach absolute beginners about AI Inference.
Over the last 2 months, whilst working as a full-time SWE intern, I was grinding after hours to create this video series. I learned to code, amongst many other things, through YouTube videos aimed at beginners. Now, with the rise of AI, these videos tend to show up less, so I decided I wanted to change that.
This is . A lot of thinking and consideration went into creating this, and all of my notes are written by me. For transparency almost all code was also written by me with exception of small bug-fixes, readme updates and stuff like that. There is also extra notes pertaining to each video which are AI generated, but main notes are all me.
The series walks you through building a small Llama 3.2 1B inference engine in C++ and CUDA, starting with what happens when you send an LLM a prompt.
I cover: How text becomes tokens, and tokens become embeddings. Loading model weights into GPU memory. CUDA basics, writing your own GPU kernels, and matrix multiplication with cuBLAS. The transformer step by step: RMSNorm, RoPE, attention, and the SwiGLU feed-forward network. Prefill and decode: processing your prompt and generating a response one token at a time. KV caching and how it saves repeated computation. Static and continuous batching to handle multiple requests. Paged KV caching and paged attention to manage and read cached data on the GPU.
Here is the github repo as well as a link to my youtube:
https://github.com/samir-sarwar/samir-vllm https://www.youtube.com/@samir-sarwar
(if anyone hiring for summer 2027 interns focused on inference or mle hmu please!! )
Holy hell. This guy is working hard. Please hire him!
I’d also add, software engineering pros get there in a lot of ways. That’s what is so cool about the field. But whenever I see pure technical grind, I always have respect. It is in many ways the essence of the field.
thanks for acknowledging the hustle :)
if you like what you see, lets connect! : https://www.linkedin.com/in/samir-sarwar-swe/
or x: https://x.com/samirswr