In a pot or large skillet, heat the diced apples, sugar, butter, cinnamon, and nutmeg on medium-low heat until the apples are soft and tender. This will take about 10 minutes. Stir the apples occasionally to make sure they are all cooking evenly.
Mash the apple mixture as much as you can with a fork or a potatoes masher. Use roughly 2 tbsp worth of filling and place it in the middle of the rectangles that are on your baking sheet. Leave about 1/4 inch around the edge.
Place the other rectangles on top of the filled ones. Crimp the edges and pierce the tops with a fork. You should leave a little bit of space between each pop tart.
Carefully transfer your pop tarts to a cooling rack to firm up. They may be a little soft fresh out of the oven but don’t worry – they won’t stay that way.
Once your pop tarts have completely cooled, make your icing by whisking together all of your ingredients until smooth. Add in extra milk if it is too thick, 1 tbsp at a time.