Principal Research Scientist
Northeastern University

Professor
Universitat Oberta de Catalunya

I am a Principal Research Scientist at the Institute for Experiential AI at Northeastern University, where I lead research on AI for Health and Responsible AI. I am also an Affiliated Professor at the Bouvé College of Health Sciences at Northeastern University, a Professor at Universitat Oberta de Catalunya (UOC), and a Research Affiliate at MIT Medialab. At UOC I lead the Computer Vision group at UOC eHealth Research Center.

My research interests are related to Computer Vision, Natural Language Processing, Affective Computing, AI for Health and Wellness, Social Robotics, and Responsible AI. More concretely, my research focuses on Human-Centric AI, which refers to creating AI that meets human needs, enhances human capacities, and aligns with human values. Her research program includes fundamental research on explainable, contextualized, and multi-modal systems for emotion and social signal perception, and their use cases for Health and Human Wellness.

From 2012 to 2015 I was a Visiting Professor at MIT CSAIL, where I worked on Object Detection, Scene Category and Attribute Recognition, and Explainable AI. From 2017 to 2020 I was a Visiting Professor at MIT Medialab Affective Computing group, where I worked on Emotion Perception, Emotionally-Aware Dialog Systems, and Human-Social Robot Interaction. During 2020-2021 I was a Visiting Faculty at Google (USA). Currently, I'm a contractor at Apple Machine Learning Research.

I did my PhD in Computer Science at the Universitat Autonoma de Barcelona and my BS degree in Mathematics at the Universitat de Barcelona.

To check my publications please visit my Google Scholar profile.

Highlights

Check our work on CCN Interpretability (accepted at PNAS 2020)

An analytic framework to systematically identify the semantics of individual hidden units within image classification and image generation networks (PDF).

Our work on detecting incidents in the wild accepted at ECCV 2020

Our paper presents a database and trained models to recognize incidents in scenes (PDF). Find more information in the website of the website of the Incidents project. Check also our online demo.

Our work on emotionally-aware chat-bots accepted at NeurIPS 2019

Our paper proposes new methodology on how to evaluate open-domain dialog systems (PDF and available code repository). Check also our paper on using user's feedback in an off-policy reinforcement learning setting to improve the quality of the bots (PDF).

"Context Based Emotion Recognition using EMOTIC dataset" (at TPAMI 2019)

Extended dataset and extended experiments with different types of context features and loss functions on IEEE Transactions on Pattern Analysis and Machine Intelligence (PDF). The second release of the Emotic dataset is available at the website of the Emotic project.

Our paper on "Emotions in Context" accepted at CVPR 2017

We present a database for studying how to model context to understand people emotional states. We show promising results at estimating 26 affective categories and continuous dimensions (PDF, Project Page)

Our work on "Class Activation Map" accepted at CVPR 2016

We revisit the global average pooling layer and shed light on how it explicitly enables the convolutional neural network to have remarkable localization ability despite being trained on image-level labels (Project Page).

Scene recognition demo

Given a picture our system predicts the scene category and some other attibutes. It also provides a heatmap that indicates the region of the image that supports the ouputs.

Understanding the representations learned by CNNs

We found that object detectors emerged in a CNN trained for scene recognition. For more information check our paper: B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba. “Object Detectors Emerge in Deep Scene CNNs.” International Conference on Learning Representations (ICLR) oral, 2015. (PDF).

Project page of Places Database

You can download the database and the pretrained network PlacesCNN. More details can be found in our paper: B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva. “Learning Deep Features for Scene Recognition using Places Database.” Advances in Neural Information Processing Systems 27 (NeuIPS), 2014. (PDF).