Lecture by O. L. de Lacalle Semantic text similarity with images

Dr. Oier Lopez de Lacalle is giving a talk next 16:00 June 26th, 2019 – Aula Anfiteatro Ex-Farmacologia Viale Morgagni 65. The talk will focus on “Evaluating Multimodal Representations on Sentence Similarity: The Visual Semantic Textual Similarity Dataset”.

In this talk Dr. Oier Lopez de Lacalle introduces vSTS, a new dataset for measuring textual similarity of sentences using multimodal information. The dataset is comprised by images along with its respectively textual captions and allows to study whether better sentence representations can be built when having access to corresponding images, e.g. a caption and its image, in contrast with having access to the text alone.

We describe the dataset both quantitatively and qualitatively, and claim that it is a valid gold standard for measuring automatic multimodal textual similarity systems.

vSTS extends the existing semantic textual similarity tasks with images, and aims at being a standard dataset to test the contribution of visual information when evaluating sentence representations.

We also describe the initial experiments combining the multimodal information. In the experiments we show that the dataset allows to explore two hypothesis:

  • H1) whether the image representations alone are able to predict caption similarity;
  • H2) whether a combination of image and text representations allow to improve the text-only results on this similarity task.

Biography: Dr. Oier Lopez de Lacalle is a B.Eng in Informatics (2003), received his Ph.D. in Computer Science in 2009 from the University of the Basque Country (UPV / EHU).

He is currently working as a post-doctoral researcher in the MUSTER project in the IXA group of the UPV / EHU. Previously,
he was a researcher hired by Ikerbasque at the University of Edinburgh. In these years he has published more than 40 articles in
journals and international conferences in the area of Natural Language Processing (NLP) and Artificial Intelligence (AI).

His research focuses mainly on semantic processing and information extraction using probabilistic modeling and deep learning algorithms.

The contributions various publications in JCR journals and high-level congresses (CORE A+). He is a regular reviewer in international journals and participates in international congress committee programs such as EACL, NAACL, ACL, EMNLP, and IJCNLP.

Stan Sclaroff

Lecture by Stan Sclaroff

Attention, Capabilities of Humans, Algorithms in Vision

Alberto Del Bimbo at AI DIVE 2018

AI DIVE 2018 Conference

Alberto Del Bimbo speaker on AI

Sho Takahashi

Lecture by Sho Takahashi

Video analysis for sports understanding

CBMI 2017

CBMI 2017 19-21 june 2017
Firenze, Italy