Preheat your oven to 350°F (175°C). Line two large baking sheets with parchment paper.
In a medium bowl, whisk together the flour, cocoa powder, baking soda, baking powder, and salt. In a large mixing bowl, beat the softened butter and sugar together until light and fluffy, about 2 minutes. Add the egg and vanilla extract, and beat until combined.
Add the dry ingredients to the butter mixture, alternating with the milk, beginning and ending with the dry ingredients. Mix just until combined. Stir in the hot water until the batter is smooth and slightly runny.
Drop heaping tablespoons of batter onto the prepared baking sheets, spacing them about 2 inches apart. You should have about 24-28 individual cakes. Bake for 10-12 minutes, or until the cakes spring back when lightly touched.
Let the cakes cool on the baking sheet for 5 minutes, then transfer them to wire racks to cool completely.
In a heatproof bowl, melt the marshmallows and 1 tablespoon of water in the microwave in 15-second intervals, stirring between each, until fully melted and smooth. Let it cool slightly.
In a separate large mixing bowl, beat the softened butter until creamy. Add the powdered sugar and beat until fluffy, about 2 minutes. Slowly add the cooled melted marshmallows and vanilla extract, beating until well combined. If needed, add 2-3 tablespoons of heavy cream to reach a spreadable consistency.
Spread or pipe a generous amount of marshmallow filling onto the flat side of half of the cakes. Top with the remaining cakes to create a sandwich.