Build Cartoon Avatar Diffusion Model using HuggingFace diffusers

Cartoon Avatar Diffusion Model Memory Anatomy

Following up the question "where has the GPU memory gone?" raised in the last section of the blog "build-avatar-diffusion-model-from-scratch". It's so puzzling to see that a relatively small model (~290M) with small input (~6M per batch) consumes almost the entire 24G GPU memory during training.

Build Cartoon Avatar Diffusion Model from Scratch