An image sharing server provides several ways of sharing images between users. After a user contributes images to the image sharing server, the user can interact with the image sharing server to identify and tag people in the images, share the images with other users, and organize the images into memory boxes. Memory boxes can also be shared between users, and multiple users can be granted the ability to add images to a shared memory box. In addition, the image sharing server can prompt a user to share his or her images with other users who contributed related images. The image sharing server also performs facial recognition to automatically identify people in images, and facial recognition models can be shared between users.