Unraveling the Mysteries of the Hole and Clash (black Hole) Problem

What are Holes and Blackholes (Clashes)?

In the current version of the server, each cycle lasts for 100ms. During this period, clients can send requests to the server for player actions. However, it is only at the end of a cycle that the server executes the actions and updates the environment. The server uses a discrete action model; thus only execute one primary action command in each simulation cycle. When the agents send multiple primary commands during a cycle, the server chooses the first one for execution and discards the others. This situation is known as a black hole (clash).

On the other hand, sending no request during a cycle will mean that the agent misses an opportunity to act and remains idle*. This situation will be referred to as a hole (miss) and is also undesirable since, in real-time domains, this usually leads to the opponents gaining an advantage. We refer to black holes as clashes to prevent confusion between holes and black holes.

Why we are using Docker?

There was a need to slightly update how the competitions were held, considering there has not been much change in twenty years.

Using Docker in the Soccer Simulation 2D environment provides several advantages.

1. Easy setup and deployment: The utilization of Docker provides a convenient means of packaging and distributing the entire simulation environment, including all associated dependencies, libraries, and configurations. This approach streamlines the process of setting up and deploying the simulation on multiple machines or platforms while ensuring that the environments are consistent and reproducible. This streamlined process can run on various platforms, including Amazon AWS, Google Cloud Platform, VirtualBox, Rackspace server, or any other viable platform. It is essential to note that the host operating system must support Docker.

2. Isolation and consistency: Docker containers provide an isolated and consistent environment for running the soccer simulation. It encapsulates all the necessary components and dependencies, eliminating conflicts or compatibility issues that may arise when simulating teams on different systems. This isolation and consistency ensure the simulation runs reliably and consistently across any platform. This isolation gives us the perfect environment to terminate any unwanted connection between agents themself or the internet, making it more realistic and up to the rules.

3. Scalability and resource optimization: We can run several simulation instances simultaneously in separate containers, optimizing resource allocation and allowing for scalability. It can run multiple simulations simultaneously on a single machine or distribute them across a cluster of machines efficiently using computational resources.

4. Version control and reproducibility: Docker containers are based on images, which can be version controlled. Different versions of the soccer simulation teams can be stored, managed, and easily reproduced. It ensures that specific teams and simulation versions can be accessed and utilized when needed, maintaining consistency and reproducibility in research, development, and competitions.

5. Collaboration and sharing: Docker simplifies collaboration by providing a standardized, portable environment. Developers, researchers, and participants can share their simulation setups and team binaries by sharing Docker images. Enables seamless collaboration, sharing of ideas, and easy replication of experiments, fostering a more efficient and collaborative research community.

Overall, Docker in the Soccer Simulation 2D environment streamlines setup, deployment, consistency, scalability, reproducibility, and collaboration, enhancing the overall efficiency and effectiveness of the simulation for research, development, and competition.

Memory

Memory availability directly affects the performance and responsiveness of the game. Insufficient memory can lead to slow response times. Adequate memory resources are essential for smooth simulation, quick decision-making, and timely execution of actions.

Memory limitations impact resource allocation and scalability in distributed Soccer Simulation setups. In scenarios where multiple simulation instances are run concurrently, memory requirements increase. If memory is constrained, it may limit the number of simultaneous simulations that can be executed, affecting scalability and the ability to utilize available computational resources efficiently.

Fast memory speed is crucial for Soccer Simulation 2D. It enables quick data access, faster agent decision-making, and smoother gameplay. Slow memory speed can introduce delays in processing data, prolong simulation cycles, impact learning performance, and affect the coordination of agents.

The memory in the Docker system is limited to 2GB for each agent (each teams has access to 25GB of ram). We lowered this amount to test how it will affect the gameplay, and the number of H&C rises as expected.

Past tournaments

Unfortunately, there is no definite information on the RC22 memory, IO23 had 96GB, and IO22 had 128GB.

Network

The network can significantly impact Soccer Simulation 2D games, particularly concerning the occurrence of H&Cs. Here is how the network can affect holes in the gameplay:

1. Network Latency: Network latency is the time delay when data travels from one point to another on a network. In 2D Soccer Simulation games, network latency can cause delays in communication between the server and clients (agents). When there is high latency, it takes longer for the server to receive client requests and send game state updates back. This delay can cause H&C.

2. Packet Loss: When data packets do not reach their destination, it can interrupt communication between the server and clients. If a client's request packets are lost, the server will not receive the necessary actions for that simulation cycle. This PL can be especially problematic in real-time games like Soccer Simulation 2D, as it can immediately affect gameplay.

3. Network Congestion: When there is too much data traffic on a network, it can cause delays and increased latency, known as network congestion. Network congestion can make H&C happen more often because as more clients use the network, it takes longer for their requests to reach the server and receive updates back.

4. Unstable Connections: An unstable or unreliable network connection can cause inconsistencies and irregularities in gameplay. Frequent disconnections or fluctuations in network stability can lead to unreliable communication between the server and clients, resulting in agents experiencing frequent issues such as failed or delayed requests.

Each simulation server network will be handled separately in the Docker setup, as we did not encounter any network issues, and the Rcssservers are in one location or in one host, indicating that the network speed and stability are secured. However, we suspected that the usage of the WireGuard might be problematic. To ensure a secure and stable connection between our streaming system (rcssmonitor) and the host, we utilized WireGuard to establish a private network. WireGuard was necessary as many hosts do not permit repeated bidirectional UDP connections. The private network allowed the rcssmonitor to function properly.

To see the impact of the WireGuard, we changed the number of WireGuard connections, increased the client's number, limited the transmission speed, and changed server locations, and there were no changes in the number of H&Cs.

Fact: We checked that if we use one rcssmonitor, we transmit an average of 207.8 Kb of data per second. Using two rcssmonitor simultaneously reaches almost 591.8 Kb/s, and three monitors use 761.9 Kb/s, which means the transmission speed is linear. The average speed for monitoring every game is 250 Kb/s transmission speed.

Past tournaments

The network setups stayed the same on the Docker side of IO22, RC22 and IO23. We aim to remove WireGuard from the upcoming competitions.