Week 1 (June 27th - July 1st)

June 27th:

I began my research this week in person where I met with Professor Vicente Ordóñez Román and we went over onboarding tasks and started to figure out a more specific project that I would be working on this summer. Initially we discussed three different topics: vision+language generation, image bias removal and analysis, image and text retrieval. However, we found that starting with bias removal and analysis would be good.

The first machine learning model introduced was the CLIP Model:

Here is the link to the CLIP Model article used.

• The CLIP Model uses both image and text retrieval.

Moreover, we looked at different research articles regarding image or text biases in machine learning models.

The first article was regarding text biases. The WinoBias dataset addressed the problem of conference systems exhibiting gender bias. In order to evaluate the biases they created a new challenge corpus, WinoBias, to show if the different systems where gender biased through the use of pronouns and different occupations.

The second article was regarding image biases. The problem addressed was that in visual recognition tasks such systems using images containing people allow the risk of amplifying societal stereotypes. The social stereotypes addressed where gender (and race, or age). They stated that the model tested exhibited bias amplification if it leaked more info about gender than a classifier of equivalent accuracy.

The articles I read where to familiarize myself with machine learning models, image and/or text retriaval, and bias removal and analysis.

June 29:

On this day, my mentor and I looked at some other helpful research articles for my summer research as well as went over some of the major parts of Python coding that I would need for this internship.

Some of the more complex aspects of Python coding when applied to artificial intelligence are understanding tensors and how they are displayed in images. Along with manipulating images using different aspects of the tensor. For instance, in order to change the saturation of an image using python and numpy, one must convert from rgb (red, green, blue) to hsv (hue, saturation, value) and then enhance the s (saturation) channel.

In order to learn the Python information and coding for this research we use one of Professor Vicente Ordóñez Román’s Python tutorials from a Google Collab Notebook.

July 1:

On this day we discussed progress on reading and comprehending articles and learning about machine learning from the Professor’s recorded lectures. As well as discussed what should be done for next week.

Upcoming

• Run on Google Collab – the CLIP model
• Try some experiments with this model using your own data your own images and text prompts
• Report here how well is the model doing for some of these.