Competitions are a great way to excel in machine learning. They offer various advantages in addition to gaining knowledge and developing your skill-set.
The problems and goals are very well defined. This saves you from the hassle of coming up with a problem, defining the goals rigorously, which are both achievable and non-trivial. You are also provided with data, which in most cases is ready for use. Someone has already done the painstaking work of collecting, preprocessing and organizing data. If it’s a competition on supervised learning, you also get labels for the data.
If you’re a procrastinator, you have deadlines to your rescue. They keep you focused and prevent you from going astray ;)
Competition leaderboards (if the competition has one), push you to do better. They keep things in perspective by giving continuous feedback on how you’re doing relative to others. You struggle to find better solutions, try to surpass yourself, and in the process keep growing.
Finally, the rewards. They come in various forms. Monetary rewards are one. The satisfaction of solving challenging problems and growing is another. But the main motivation for writing this post is the third kind of reward. If you’re a top performer in a competition organized under a conference, you get a chance to publish your results.
The scope of this list is limited to research based competitions. I was looking for a curated list of such competitions but couldn’t find any. So, decided to create one. However, there is a nicely curated Machine Learning Contests list by Harald Carlens which covers a broader scope than mine. The table below summarizes all the competitions I could find. They have been ordered according to their deadlines. I plan on updating the list on a regular basis. As more conferences release information about the competitions on their website, I’ll add them to the list.
If you know of any competition that is not on the list, please let me know in the comments or feel free to send a pull request.
| Name | Conference | Starts | Ends | Website | Sub-Challenges |
|---|---|---|---|---|---|
HEROHE
Unlike previous Challenges that evaluated the staining patterns present in IHC, this Grand Challenge new edition proposes to find an image analysis algorithm to identify with high sensitivity and specificity HER2 positive BC from HER2 negative BC specimens evaluating only the morphological features present on the hematoxylin and eosin (HE) slide.
Affective Behavior Analysis in-the-wild
This Competition aims at advancing the state-of-the-art in the problem of analysis of human affective behavior in-the-wild. Representing human emotions has been a basic topic of research. The most frequently used emotion representation is the categorical one, including the seven basic categories, i.e., Anger, Disgust, Fear, Happiness, Sadness, Surprise and Neutral. Discrete emotion representation can also be described in terms of the Facial Action Coding System model, in which all possible facial actions are described in terms of Action Units (AUs). Finally, the dimensional model of affect has been proposed as a means to distinguish between subtly different displays of affect and encode small changes in the intensity of each emotion on a continuous scale. The 2-D Valence and Arousal Space (VA-Space) is the most usual dimensional emotion representation; valence shows how positive or negative an emotional state is, whilst arousal shows how passive or active it is.
LNDb Challenge
The main goal of this challenge is the automatic classification of chest CT scans according to the 2017 Fleischner society pulmonary nodule guidelines for patient follow-up recommendation. The Fleischner guidelines are widely used for patient management in the case of nodule findings, and are composed of 4 classes, taking into account the number of nodules (single or multiple), their volume (<100mm³, 100-250mm³ and ⩾250mm³) and texture (solid, part solid and ground glass opacities (GGO)). Furthermore, three additional sub-challenges will be held related to the different tasks needed to calculate a Fleischner score.
EndoCV2020
Endoscopy is a widely used clinical procedure for the early detection of numerous cancers (e.g., nasopharyngeal, oesophageal adenocarcinoma, gastric, colorectal cancers, bladder cancer etc.), therapeutic procedures and minimally invasive surgery (e.g., laparoscopy). During this procedure an endoscope is used; a long, thin, rigid or flexible tube with a light source and camera at the tip to visualise the inside of affected organs on an external screen. Quantitative clinical endoscopy analysis is immensely challenging due to inevitable video frame quality degradation from various imaging artefacts to the non-planar geometries and deformations of organs.
After a great success of Endoscopy Artefact Detection challenge (EAD2019), this year EndoCV2020 is introduced with two sub-challenge themes this year.
Each sub-challenge consists of detection, semantic segmentation and out-of-sample generalisation tasks for each unique dataset.
Cell Tracking Challenge
The fifth challenge edition will be organized as part of ISBI 2020, taking place in Iowa City in April 2020. In this edition, the scope of the challenge will be broadened by adding two bright-field microscopy datasets and one fully 3D+time dataset of developing Tribolium Castaneum embryo. Furthermore, silver segmentation ground truth corpora will be released for the training videos of nine existing datasets to facilitate the tuning of competing methods. The submissions will be evaluated and announced at the corresponding ISBI 2020 challenge workshop according to the ISBI 2020 challenge schedule, with a paper that reports on the results collected since the third edition being published in a top-tier journal afterward.
Multi-organ Nuclei Segmentation And Classification Challenge
In this challenge, participants will be provided with H&E stained tissue images of four organs with annotations of multiple cell-types including epithelial cells, lymphocytes, macrophages, and neutrophils. Participants will use the annotated dataset to develop computer vision algorithms to recognize these cell-types from the tissue images of unseen patients released in the testing set of the challenge. Additionally, all cell-types will not have equal number of annotated instances in the training dataset which will encourage participants to develop algorithms for learning from imbalanced classes in a few shot learning paradigm.
Challenge on Learned Image Compression
We host a lossy image and video compression challenge which specifically targets methods which have been traditionally overlooked, with a focus on neural networks, but we also welcome traditional approaches. Such methods typically consist of an encoder subsystem, taking images/videos and producing representations which are more easily compressed than pixel representations (e.g., it could be a stack of convolutions, producing an integer feature map), which is then followed by an arithmetic coder. The arithmetic coder uses a probabilistic model of integer codes in order to generate a compressed bit stream. The compressed bit stream makes up the file to be stored or transmitted. In order to decompress this bit stream, two additional steps are needed: first, an arithmetic decoder, which has a shared probability model with the encoder. This reconstructs (losslessly) the integers produced by the encoder. The last step consists of another decoder producing a reconstruction of the original images/videos.
WebVision
The WebVision dataset is composed of training, validation, and test set. The training set is downloaded from Web without any human annotation. The validation and test set are human annotated, where the labels of validation data are provided but the labels of test data are withheld. To imitate the setting of learning from web data, the participants are required to learn their models solely on the training set and submit classification results on the test set. The validation set could only be used to evaluate the algorithms during development (see details in Honor Code). Each submission will produce a list of 5 labels in the descending order of confidence for each image. The recognition accuracy is evaluated based on the label which best matches the ground truth label for the image.
NTIRE
Image restoration, enhancement and manipulation are key computer vision tasks, aiming at the restoration of degraded image content, the filling in of missing information, or the needed transformation and/or manipulation to achieve a desired target (with respect to perceptual quality, contents, or performance of apps working on such images). Recent years have witnessed an increased interest from the vision and graphics communities in these fundamental topics of research. Not only has there been a constantly growing flow of related papers, but also substantial progress has been achieved.
Each step forward eases the use of images by people or computers for the fulfillment of further tasks, as image restoration, enhancement and manipulation serves as an important frontend. Not surprisingly then, there is an ever growing range of applications in fields such as surveillance, the automotive industry, electronics, remote sensing, or medical image analysis etc. The emergence and ubiquitous use of mobile and wearable devices offer another fertile ground for additional applications and faster methods.
SpaceNet
In the SpaceNet 6 challenge, participants will be asked to automatically extract building footprints with computer vision and artificial intelligence (AI) algorithms using a combination of these two diverse remote sensing datasets. For training data, participants will be allowed to leverage both the electro-optical and SAR datasets. However, for testing models and scoring performance only a subset of the data will be made available. We hope that such a structure will incentivize new data fusion methods and other approaches such as domain adaptation.
Challenge Website | Back «««< HEAD
AI City Challenge
Transportation is one of the largest segments that can benefit from actionable insights derived from data captured by sensors. Between traffic , signaling systems, transportation systems, infrastructure, and transit, the opportunity for insights from these sensors to make transportation systems smarter is immense. Unfortunately, there are several reasons why these potential benefits have not yet materialized. Poor data quality, the lack of labels for the data, and the lack of high-quality models that can convert the data into actionable insights are some of the biggest impediments to unlocking the value of the data. There is also need for platforms that allow for appropriate analysis from edge to cloud, which will accelerate the development and deployment of these models. The AI City Workshop at CVPR 2020 will specifically focus on ITS problems such as:
- Turn-counts used by DOTs for signal timing planning
- City-scale multi-camera vehicle re-identification w. real and synthetic trianing data
- City-scale multi-camera vehicle tracking
- Anomaly detection – detecting anomalies such as lane violation, wrong-direction driving, etc.
Emotionet Challenge
The EmotioNet Challenge 2020 (ENC-2020) evaluates the ability of computer vision algorithms to automatically analyze a large number of “in the wild” images for facial expressions.
Research groups that have designed or are developing algorithms for the analysis of facial expressions are encouraged to participate in this challenge.
Comments