Part 1: Image Aggregator

Project: Image Aggregator

About this Project:

More times that I would like to admit, I have found myself lost in the endless amount of pictures sitting in my photos album. Once, while frustrated over not being able to find an image which I could picture every detail of in my head, I decided that I wanted to work on a project that would allow me to do just that. I wanted to create a platform where an individual would be able to query an image just by describing it how they would to any other person. The users would be able to input information such as the names of the individuals in the pictures, the background, and details about the location which would allow my software to determine what image they are looking for way faster than having to spend several minutes scrolling through the photo album on the phone.

Initial Stages:

To first understand more about the process, I started working with the face_recognition library in Python. One of the advantages of this library are the pre-trained models that can be used for a variety of tasks. These models are able to identify unique faces in a given image and then determine an encoding for a face. This way, I am able to ask the user to name certain individuals based on face, then later on use this information to identify the person in a later image. This is the first basic step that I needed to implement in my project before I would be able to continue expanding. 

The above image is what I was able to produce by allowing the facial recognition model to learn the "vector encodings" for Niti, Nishka, and Rhea's face. Then I used the image above as an unknown image test and the face_recognition library was able to match the faces from the known images to the ones above. I further modified the code to output the names of the individuals who are in the unknown images for each image that is inputted with the following code.

Currently, I am working on incorporating both a frontend with React.js and a backend using the Flask library in order to deploy a platform that is able to collect a set of known and unknown images from a user and correctly classify those individuals in the unknown images. Later on, I can work with models to train the model on input images and then allow it to learn patterns about the background. After that, I will incorporate NLP to parse through a query that a user inputs and output an image representing that query.