A Multi-Stakeholder Assessment of AI-Based Judging in Gymnastics by Céline Decoster & Prof. Willem Standaert
The sport of gymnastics is one of the most-watched and attended events at the Olympic Games, but in the three years in-between the Games, it does not get much attention at all. An important reason for this is the difficulty of understanding how the judge-based scoring works.
For fans, it is hard to understand why one gymnast gets a higher score than another. Even if you are familiar with the rules, watching gymnastics can be frustrating because the score giving is often not perceived as fair.
However, Artificial Intelligence-based judging, which strictly and transparently matches gymnast exercises to the rulebook, could help in making the sport more accessible and popular.
To study the value of using AI-based judging systems to complement or substitute human-based judging, we have embarked on a study in which we consider the introduction of a technical artifact for judging and the impact on a set of stakeholders (i.e., federations, judges, gymnasts, coaches, fans, media).
The problems of Human-based Judging
As a consequence of human biases, the sport of gymnastics has had to deal with several judging scandals in its history. Such bias can be based on nationality, reputation order of performance, and so on. Also the viewing position of the judges and fatigue can play a role in the evaluation of gymnasts. Another problem for the attractiveness of the sport is the fact that human judges take a lot of time to come up with a score, consequently, competitions take a long time.
Artificial Intelligence to the rescue
These problems could soon come to an end. Similar to the introduction of the Video Assisted Referee (VAR) in football, the FIG (International Gymnastics Federation) and Fujitsu are collaborating to develop a new judging system that should be free of bias. It is important to specify that the AI system is rule based and that some rules need to be explicated in the process. Hence, this is not based on training data, which could introduce new types of bias.
Instead, the AI part is in the computer vision that gets input from 3D-laser sensors and helps to identify the exercise the gymnasts do and the execution thereof, based on image recognition of the joints. A system was announced that would combine human and AI judgment for use at the postponed Tokyo 2020 Olympic Games.
Figure 2 – Overview of high-speed, high-accuracy skeleton recognition technology (Sasaki, Masui, & Tezuka, 2018, p13)
Different stakeholders were asked to express their thoughts on the technology and what impact it could have on the popularity of the sport.
AI technology as a judging tool
The technology would be used to compensate for the current shortcomings of human judging. The interviews revealed that the technology as a support system would be positively received. The majority of the interviewees are convinced that the technology would benefit the judging process. It will be more transparent, objective and accurate, which will lead to less discussions about the scores.
“I really think that it will enhance the sport by making it more objective and that there will be less discussion like ‘is the angle good or not, is the turn fully completed or not?’, these are things that you can perfectly see with the technology.”
Former Belgian gymnast, Laura Waem
Given the controversy surrounding football’s VAR, this is a surprising statement. An important difference, however, is that for gymnastics each element can be specifically determined and thus recognized by the technology. Furthermore, the stakeholders indicate that the technology cannot completely replace the judges, because there are many questions about the evaluation of the artistic aspect of the sport. Former international gymnast and current elite coach Julie Croket has questions: “Gymnastics is an artistic sport, so the choreography, the appearance, how the gymnast profiles itself, I don’t think that the technology can judge that. I wouldn’t know how they would manage to do it?”
Making the judging process more transparent could cause a revolution in terms of the attractiveness of the sport for fans. The complexity of the points system in gymnastics is often cited as the main reason why not a lot of people follow the sport. The system could provide more explanation about the sport and the judging, during live broadcasts or in the arena. According to fan Hannah Mouillot, this could have an impact on the popularity of the sport:
“As a matter of fact, for people not practicing gymnastics, a competition is difficult to understand. First of all, the scoring system is complicated, also the deductions are quite difficult to understand. The technology would allow a big audience to understand gymnastics competitions. Unavoidably, it would make gymnastics more attractive, and more people would watch gymnastics competitions.”
AI technology as a training tool
Gymnasts and coaches believe that the technology could also be advantageous as a training tool. This is because they could then train with the same system that would judge them at big competitions. Gymnasts that train with the system could potentially achieve better results internationally, as it may give them an unfair competitive advantage.
Better performances often also lead to more media attention, knows international gymnast Florian Landuyt:
“Not many people watch the sport, you can change that primarily by achievements. When we look at Nina Derwael, and the women’s team that made it to the Olympics, they immediately got more media coverage.”
Thanks to media coverage, more people are reached who will eventually start to follow the sport. So using the system as a training tool could indirectly have a positive effect on the popularity of the sport. The new system is definitely an exciting development for the sport of gymnastics. It could create great opportunities, but it certainly also comes with many questions and uncertainties.
Call for participation
This research project was initiated by Céline Decoster, Master thesis student at the Faculty of Economics and Business Administration of Ghent University, under supervision of Prof. Willem Standaert. It is now followed up by a team of researchers from Belgium, Finland and Australia. Their aim is to further expand the qualitative research with the perspective of federations and committees, technology vendors, and other industry players, through interviews.
In addition, the researchers would like to establish a quantitative study/experiment that allows a direct comparison of human- and AI-based judging, as well as a hybrid combination of the two. This, in turn, could help in the development of the AI-based system in keeping with the perspectives of the entire ecosystem, provide third-party support in validating the system, and enhance the adoption and diffusion of AI-based and 3D-sensing technologies in sport and beyond.
Reach out to willem.standaert@ugent.be if you want to participate in or know more about this project.
For some related background materials, here’s more info on the previous Master thesis study (in Dutch) and a paper titled “Hybrid Sport Configurations: The Intertwining of the Physical and the Digital“.
UPDATE: the paper titled ‘Paradoxical Tensions Related to AI-Powered Evaluation Systems in Competitive Sports‘ has now been published in Information Systems Frontiers, more info & download option here.