Verification-oriented comment detection

Challenge Description

Build a system that automatically retrieves comments that could be useful in the verification process.

Users tend to leave comments on Web videos expressing their personal opinion or experience in relation to what the video shows, convey public statements about the event shown in the video, and often challenge or support the credibility of the video. Several of these comments could be helpful for the verification process.

As a baseline approach, a predefined list of verification-related keywords is used to filter the video comments and create a subset with comments of relevance to the verification process. The list of verification-related comments is available in 7 languages (English, German, Greek, Arabic, French, Spanish and Farsi).

Verification-related keyword

English	French	German	Arabic	Spanish	Farsi	Greek
lies	mensonges	Lügen	أكاذيب	mentiras	دروغ	ψέματα
fake	faux	falsch	أكاذيب	falso	جعلی	ψεύτικο
wrong	erroné	falsch	غلطان	incorrecto	اشتباه	λάθος
lie	mensonge	Lüge	أكذوبة، كذب	mentira	دروغ	ψέμα
confirm	confirmé	bestätigt	مؤكد	confirmado	تأييد شده	επιβεβαιώνω
where	où	wo	أين	donde	كجا	που
location	lieu	Ort	مكان	lugar	محل، مكان	τοποθεσία
lying	mentir	lügend	كذب	mitiendo	دروغ گو	ψευδής
false	faux	falsch, fehlerhaft	خاطئ	falso	غلط، اشتباه، دروغین	εσφαλμένο
incorrect	inexact	unrichtig	غير صحيح	incorrecto	غلط، اشتباه	λανθασμένο
misleading	trompeur	irreführend	مضلل	engañoso	گمراه ‌ كننده	παραπλανητικό
propaganda	propagande	Propaganda	دعاية	propaganda	تبلیغات سیاسی، پروپاگاندا	προπαγάνδα
liar	menteur	Lügner	كاذب	mentiroso	کذاب	ψεύτης

---

Input

• YouTube API responses containing the video comments.
• Verification-related keywords in 7 languages.

---

Output

List of verification-related comments and/or keywords

---

Evaluation

The approach will be evaluated qualitatively since no ground-thruth is provided. You can select up to five videos from the dataset as test videos and apply the approach on them. As a result, the extracted verification comments should provide useful information for debunking the video.

Suggested test videos:

CAT 5 Hurricane Irma destroys St. John's, Antigua (WIND @ 185MPH!!!): The video claims that it shows footage of Irma hurricane but in fact it shows a tornado that struck Dolores, Uruguay in April 2016.

لحظة حدوث انفجار في مطار بروكسل: The video claims that it shows footage of Βrussels airport explosion ιν 2016. This video is a real video of an old event and is shared wth false contextual information. The video actually was shot in 2011 at Domodedovo Airport in Russia

Niño sirio salva a niña de francotiradores: The video claims that it shows a Syrian boy rescue girl in shootout. This video has been staged. It was filmed in Malta by Norwegian film makers as an experiment

Golden Eagle Snatches Kid: The video was created by a team of students in Montreal as part of their course on VFX.

Hawk throws snake at family eating lunch in Melbourne: This video has been staged and digitally manipulated. It was created by Melbourne-based video production company The Woolshed Company for Australian Football League club The Hawks (Hawthorn Football Club).

Arabs Destroy Christmas Tree In Italy: This video is a real video of an old event and is shared wth false contextual information. The video was originally taken in December 2015 and actually shows a group of Italian teenagers.

---

Hints:

Experiment with weakly supervised learning using the verification-keywords to label the comments as verification related ones.

---

Contact:

Olga Papadopoulou: olgapapa@iti.gr

Symeon Papadopoulos: papadop@iti.gr

---