Emory NLP is hiring Emory students for annotation related to conversational AI research. Accepted applicants are expected to go through a training session before the actual annotation. Please fill this form if you want to apply for this job.
Although human-computer interaction has recently made enormous progress, chatbots still struggle with certain conversational aspects natural to humans. This project involves talking with chatbots in text and evaluating various characteristics of dialogues such as:
Each annotator is expected to complete the following three tasks:
Thus, the total estimated time is 190 minutes for all three tasks. Once the annotation is done and passes our quality assurance, we will pay $32 with an Amazon gift card.
Here is an example of a dialogue between a human and a chatbot:
Here is an example of the evaluation: