TRECVID Task Proposal Process

TRECVID consists of a series of "tasks", each focused on some facet of the multimedia retrieval problem. Examples of tasks include ad-hoc video search, instance search, video captioning, and event detection. The tasks invigorate TRECVID by encouraging research in new areas of multimedia retrieval. The tasks also support the multimedia research community by creating the infrastructure such as test collections necessary for task-specific research.

The set of tasks run in any particular year depends on the interests of the participants and sponsors, as well as on the suitability of the problem to the TRECVID environment. Selecting which tasks to include in a particular TRECVID in the past was done informally. However, TRECVID is receiving increasingly many proposals for new tasks. With a limit of at most seven tasks that can be run at one time, the requests for new tasks have surpassed the available space. Thus, the TRECVID program committee has decided to institute a more formal proposal mechanism for selecting tasks.

Call for New Task Proposals at TRECVID 2024

NIST invites proposals for tracks to be run in TRECVID 2024. Past TRECVID tracks have included ad-hoc and instance video search, concept detection, localization, video summarization, copy detection, video-to-text description, multimedia and surveillance event detection, activity detection, and disaster scene indexing. For TRECVID 2024, NIST invites track proposals that continue or build on previous TRECVID tracks, as well as proposals for new tracks in content-based video retrieval and related areas of research including video understanding.

Proposed TRECVID tracks should aim for the following timeline:

• February 2024: track draft guidelines are completed
• April : task guidelines finalized
• June to early September: test data released
• September: evaluation results released

Criteria for selection of tracks include:

• A strong advocate who is willing to volunteer as track coordinator.
• A large enough core of interested researchers to make the track viable.
• The availability of sufficient resources such as appropriate corpora, and assessors/annotators with expertise in the area.
• The track fits in well with existing tracks.

Proposals should be no longer than 3 pages (PDF) and include:

1. An explicit statement of the goals of the track (i.e., what is expected to be learned and/or what infrastructure would be created if the track were run).
2. A description of the task(s) to be run in the track, and possible methods of evaluation, including availability of any required tools.
3. What corpora would be necessary for the track, and whether there are any special constraints on distribution of the corpora.
4. What manual assessment or annotation is required (if any).
5. Where the assessing/annotation would occur (NIST or elsewhere) including any existing funding.
6. Whether there are any special qualifications or domain expertise required of the assessors/annotators.
7. An estimate of the amount of time required for such assessing/annotation.
8. Full contact details of the organizing committee, including designation of a track coordinator.

Please email track proposals to TRECVID Project Leader by November 1, 2023.
Selected track proposals will be presented at the TRECVID 2023 workshop this December.
Questions regarding track proposals may be directed to gawad@nist.gov
We welcome all ideas and feedback. If you would like to discuss in-person potential proposals please let us know and we can arrange for an online meeting.