Project: AI in Production

Experiment: Aged-Up Deepfakes

From this TETRA research project AI in production, we have prepared an introductory workshop on deepfakes to give to the DAE students of the VFX major. During the annual creative week, all regular courses for first and second year students are canceled and replaced by creative sessions. Our goal was to provide an introduction to deepfakes from the research group: what is deepfake technology, what can you do with standard software without heavy programming yourself, what are the points of interest if you want to get started with it, …
Due to the lockdown, the creation week was canceled this academic year, but the preparations for the workshop were already ready. The workshop with students will be moved to a later date in one of the VFX classes, but we are happy to share the approach of the workshop with the guidance group here. Read about the limits of this technology and how we tackled Star Wars in a hands-on use case with the combination of different tools.

Reinforcement Learning to find bugs

In this overview post we show how Reinforcement Learning (RL) can be applied to testing games. We do this by means of several papers and use cases. We show how you can detect errors in level design and game breaking bugs in this way. It may be helpful to read our introduction to RL first if you are unfamiliar with the concept.

Audio in Games

Audio is an essential element to the success of any game. It provides an immersive experience that can make the difference between the player enjoying a game or becoming frustrated and giving up.
Delayed, missing or incorrect sound effects can seriously affect the immersion of the game. Additionally, inconsistent, or sudden changes in volume of the background music can break immersion.
With this use case we want to investigate what the impact of AI for audio in games can be, and if problems such as error detection in sound effects and music can be addressed.
This may or may not bother players while playing, however, the right sound effects or background music can set the atmosphere while playing a game. Additionally, audio is an important element in determining what is happening in the game at any given moment. This can be illustrated by moments such as enemies sneaking up on you from the side, giving them the advantage to attack.
Unfortunately, audio issues still exist in games. They can be caused by several factors, including code changes, assets that are not properly optimized, or simply the way the game is played. We refer to these as bugs.
Actively detecting audio bugs requires quite some time. Furthermore, these are sometimes overlooked because game testers give a lower priority to audio. For example, people sometimes test without their headphones or with the volume set to 0, because detecting graphical and functional bugs involves a lot of repetition, which means that sound effects are also played ad nauseum.
In this use case, we explore ways to automatically detect audio bugs during development. This can help you identify and resolve them before they cause problems for the players. However, we’ll ignore the more nuanced challenges with audio, as detecting technical things like room temperature are too niche to delve into further.

AI in Production

Artificial Intelligence (AI) is a buzzword that has become hard to ignore lately. Despite much attention to the subject at conferences and in the media, it can remain vague what the realistic possibilities are today. In recent years, we have had AI breakthroughs in the field of self-driving cars, voice technology and playing various games. However, a clear picture of how this game, VFX and animation can improve production pipelines is still lacking.

Code synthesis with LLM

Good news for people without programming experience. You can now put together a game without even typing a letter code. We can now indeed automatically generate code with language models. This is done using simple commands, or the model simply supplements existing code. And for beginners: a language model can check your code for syntax errors and also optimize them automatically. Even for more experienced programmers, there is sometimes still rehearsal or research work. Especially when juggling between different languages. In this blog post we see how programming is now easier than ever before.

Automating Unit tests

Within this experiment a generic open-source model for generating code was created to avoid the repetitiveness of writing unit tests manually. This makes this use case ideal for automation. In the first example two methods for generating unit tests are presented. We will provide one full example and write down the name of the second method that we expect you to supplemented.

CoG 2021

Games offer a limitless domain for computational creativity, design, technology, education, social sciences, and artificial intelligence. The annual IEEE Conference on Games (CoG) is a unique forum for cutting-edge research related to game technologies and design, covering scientific, technical, social and human aspects of games.

CoG was expressly launched to reflect the changing nature of games as technology and media; where concerns merge, overlap, and cross-pollinate. Beginning as an evolution of the Conference on Computational Intelligence and Games (CIG) and later joined by the International Conference on Virtual Worlds and Games for Serious Applications (VS-Games), CoG brings together leading researchers and practitioners from academia and industry, to share recent advances and co-create future directions.

This year’s edition of CoG continues forging the path of attracting an evermore competent and diverse audience. Alongside our traditional types of submission (articles for peer review intended to be included in the conference proceedings), we seek proposals for presentations from both academia and industry in the form of abstracts. Furthermore, it is also possible to apply to present work previously published in the IEEE Transactions on Games journal.

AI Summer School 2021

In 2021, KU Leuven organised the first edition of the Summer School on the Law, Ethics and Policy of Artificial Intelligence (AI). Given the programme’s overwhelming success, three editions have taken place thus far. The fourth edition will take place from 1 to 10 July 2024.

The Summer School aims to provide a comprehensive overview of the various legal, ethical and policy-related issues around AI and algorithm-driven processes more broadly. As these technologies have a growing impact on all domains of our lives, it becomes increasingly important to map, understand and assess the challenges and opportunities they raise. This requires an interdisciplinary approach, which is why we are collaborating across faculties and departments to organise this Summer School. The programme’s goal is to offer participants the latest insights on AI from various perspectives, and in particular the fields of law, ethics and policy.

Overview: Speech Synthesis and Conversational AI

Over the past year, we’ve noticed a number of recent innovations around speech synthesis and conversational AI. So far, the state of the art in this area is not yet at the point where it can be realistically applied immediately in a game or VFX pipeline. That said, innovation never stops, and new techniques and interesting innovations emerge regularly within this field. In this blog post we bundle these for you.

Overview: Video Motion Capture Solutions

Although Motion Capture is now the international standard for animations of humanoid characters, it is often too expensive for smaller companies.
We regularly hear smaller companies in the game development industry ask the question: couldn’t we just do MOCAP with a camera or webcam, using AI?
The images we have already seen of AI that recognizes people on video images look promising. On top of that, a skeleton is recognized that matches the location and pose of the people in the image.
Within this overview we look at the state of the art of MOCAP systems that only use a webcam or video camera. We discuss how much potential for improvement there is and whether it is worth it for you.

Overview: Automatic Rotoscoping

Automatic rotoscoping could speed up a very labor-intensive part of a production. It is therefore interesting to look at where people are so far in research into algorithms that can make this happen. In this blog we look at 4 different, recent techniques and programs that strive for (semi-)automatic rotoscoping.

Overview: Automatic Rigging

To date, creating a rig for a model and assigning the correct blending weights has been a manual process. This rig is required to further animate your model, and is therefore a crucial step in the animation pipeline. In this overview we look at whether it is possible to automate this phase, which would be a big step forward in accelerating the animation pipeline.

Overview: Reinforcement learning in games

In this blog post we give a brief introduction to Reinforcement Learning (RL) and look at a number of applications within the gaming world. The goal is to stimulate your ideas, and also to demonstrate that you don’t have to be an AI expert to get started with these technologies. We look forward to questions or ideas arising from this blog post and are happy to take all input into account in preparation for the planned workshops.