Optional: Whipped cream or marshmallows for topping
In a saucepan, mix cocoa powder, and sugar.
Add water and heat over medium heat, stirring constantly until the mixture becomes a smooth paste.
Pour in the milk gradually while continuing to stir to avoid lumps.
Heat the mixture until it's hot but not boiling, and then remove from heat.
Stir in vanilla extract.
Pour into mugs and top with whipped cream or marshmallows if desired.