Cut cake into 2-inch-thick slices and cut slices into 1-by-4-inch strips. Spread preserves over one side of each strip. Arrange half the cake, jam-side up, in the bottom of a 12-cup serving bowl. Sprinkle with half the sherry. Arrange half the fruit over the cake layer and spoon half the custard over the fruit. Repeat with remaining cake, sherry, fruit and custard. Cover and refrigerate for at least 1 hour.
Just before serving, sprinkle almonds over the top and arrange fresh fruit decoratively in the center.
Make Ahead Tip: Prepare through Step 1, cover and refrigerate for up to 8 hours.