Prepare the Caramel Layer:

1. In a skillet or saucepan, melt 1/4 cup unsalted butter over medium heat.
2. Stir in brown sugar, cinnamon, and nutmeg until the sugar dissolves and the mixture is bubbly.
3. Pour the caramel mixture into a greased 9-inch round cake pan. Arrange the sliced apples in an even layer on top of the caramel. Refrigerate while preparing the cake batter.

2. Make the Cake Batter:

1. Preheat oven to 350°F (175°C).
2. In a medium bowl, whisk together flour, baking powder, cinnamon, salt, and nutmeg. Set aside.
3. In a large mixing bowl, beat softened butter, granulated sugar, and brown sugar until light and fluffy.
4. Add eggs one at a time, beating well after each addition. Stir in vanilla extract.
5. Gradually add the dry ingredients to the wet mixture, alternating with the milk. Mix until just combined.
6. If desired, fold in the diced apple for added texture and flavor.

3. Assemble and Bake:

1. Remove the caramel and apple layer from the refrigerator. Pour the cake batter evenly over the top.
2. Place the cake in the preheated oven and bake for 40-45 minutes, or until a toothpick inserted into the center comes out clean.

4. Flip and Serve:

1. Let the cake cool in the pan for 10-15 minutes.
2. Carefully invert the cake onto a serving plate, allowing the caramel and apples to form the topping.
3. Slice and serve warm with a dollop of whipped cream or a drizzle of caramel sauce.