In a separate large bowl, whisk together your milk, butter, eggs, vanilla, and food coloring until combined. Mix in your flour mixture until just combined, do not overmix.
Evenly divide your batter between the prepared muffin liners and bake for 18-20 minutes or until a toothpick inserted into the middle comes out almost clean. Allow the muffins to cool completely before making your filling.
Once your muffins have cooled, use a tsp measure to scoop out 1 tsp of muffin out of the top middle of your muffins, set those scooped out parts aside.