04 Jan

This application is closed.

Emory NLP is hiring Emory students for annotation related to relation extraction research. Accepted applicants are expected to go through a training session before the actual annotation. Please fill this form if you want to apply for this job.

  • Prerequisite: background in Linguistics.
  • The project begins in the week of Jaunary 24th, 2022.
  • You must be a current Emory student to apply for this job.
  • Your name will be acknowledged in research papers describing this annotation work.



This project aims to extract relations for three types of NLP tasks, coreference resolution (CR), temporal ordering (TO), and discourse/causal inference (DCI), in 1,000 social media posts from Reddit.  Here are details about these tasks:

  • Estimated time for CR per post: 7.2 mins
  • Estimated time for TO per post: 12 mins
  • Estimated time for DCI per post: 12 mins
  • Minimum number of posts per task: 50 posts
  • Maximum number of posts per task: 500 posts

Thus, each annotator is expected to complete 50 - 500 posts per task (CR: 6 - 60 hours, TO: 10 - 100 hours, DCI: 10 - 100 hours). Note that you do not need to work on all three tasks. Once the annotation is done and passes our quality assurance, we will pay $1.2, $2, and $2 per post ($60 - $600, $100 - $1,000, $100 - $1,000) for CR, TO, and DCI respectively, with an Amazon gift card.

Coreference Resolution

This task aims to find a set of mentions that refer to the same entity. In the following example, "The boy" and "himself" refer to the same entity. Moreover, the pronoun "it" refers to the event described by "likes".

[The boy]E0 [likes]V0 [himself]E0too much. [It]V0is embarrassing.

We will also annotate part-whole relations between entities and events. In the following example, the pronoun "my" refers to the speaker who is a part of "my group". Moreover, the pronoun "they" refers to "some", which is a part of my group.

[Some of [[my]E0group]E0]E1 are useless, [they]E1never help.

Finally, generic nouns, typically used when making generalizations, are also annotated.

[Doctors]Ghelp [doctors]G

Temporal Ordering

This task aims to find the temporal order of events within the post. Specifically, we will identify the following relations:

  • before/after
  • contained/contains
  • overlap

TThe events are ordered by the Document Creation Time (DCT). In the following example, DCT is after the time the boy was eating, and the boy ate his dinner before he became sick:

The boy ate his dinner. He later was sick.
after(ate, DCT)
before(ate, sick)

In the following example, the bear comes up to the man during the time he is pretending; thus,  the contains relation is used:

He threw himself on the ground and pretended to be dead. The bear came up and sniffed all around him.
after(threw, DCT)before(threw, pretended)contains(pretended, came up)before(came up, sniffed)

Finally, the following example shows two events thatoverlap:

The insurer’s earnings from commercial property/casualty lines fell 59% in the latest quarter, while it lost $7.2 million in its personal property/casualty business.
after(fell, DCT)overlap(fell, lost)

Discourse and Causal Inference

This task aims to infer causal dependencies between events and discourse relations, such as elaboration, contrast, restatement, etc.

causeIf A did not occur, B would not have either.
enableIf A did not occur, B may have occurred anyway.
preventA reduces the likelihood of B occurring.
conditionOnly when B happens, can A happen
contrastA and B share a significantly different predicate or property
parallelA and B share a significantly similar predicate or property
concessionA should result in the occurrence of C, but B indicates the opposite of C occurring.
exceptionB is an exception to A; A is not true because of B
elaborationB is a more detailed description of A
restatementB restates the semantic meaning of A
optionA and B are alternative situations of each other
chosenoptionA and B are alternative situations of each other, but the author prefers A

The following example shows an instance of a contrast between the size of TCNJ and Rutgers.

TCNJ is a super small college. Rutgers is humongous, though, totally different vibe.
contrast(a super small college, humongous)

In the following example, the author is making a concession: even though Fauci has done so much, people dislike him.

People still dislike Fauci. Despite all he has done.
dislike(concession, done)

The following is an example of a restatement.

I wasn't entirely sure what to write for my essay. In fact, I was pretty confused about where to start.
restatement(sure, confused)
* The email will not be published on the website.