Bug Hunting
Locating the nest
Bugs are collected through our In-Game Bug Reporting tool which players use to tell us when they’ve found something that isn’t as it should be. Alternatively, Bugs are caught internally and we make Bug reports through our studio communication. Then automated tickets are made within our task tracking system in order to better organize the severity of the situations.
How to decide what is a Bug and what is a local issue.
- We have a couple of ways. One is that local hardware problems would be more rare. It would be a problem that looks strange and only comes up 1 in 1000 reports. If it’s serious enough, we will make a ticket trying to reproduce the Bug ourselves and attach these reports together to do an investigation when we have time. Another comes down to experience most of the time. In this line of work you see it all, and slowly form connections between Bugs present to roughly understand where it comes from, whether it’s a visual, mechanical or hardware problem, gets easier.
- The other is just trying to find a reproduceable case. If it’s a local problem you will most likely not run into a solid reproduction; or cases where players are able to reproduce the issue but our teams are not able to do so on our internal builds. Localized problems can even happen internally, this is why we always have more than one QA go through the same written steps to eliminate every possible difference to make sure the cause can be narrowed down to hardware.
Looking at our In-Game Bug Reports
- QA will go through the list of Bugs and either filter by day or a specific problem. For example if we are looking for skill related issues, we might look for issues where players called out skills like “warpath” to help narrow down the list.
- In Game Bug Reports, which are sent from the community, have all sorts of valuable info attached to them as well as any details the player offers describing what the problem is. The more detailed the report, the better everything goes for us. Some of the information automatically added to these reports includes: game graphics settings, server logs, player logs, zone name of the place they are in, a screenshot of the location, etc.
- A good source of information outside of what the player is describing in the issue, is looking at their player logs which include client specific errors (errors related to any visual loaded into the scene) and server logs (anything loaded in on the server). If there are clues to the problem here, which most of the time there are, we use this to update the internal report to point to a specific problem that is called out.
Replication and Recording
It’s very scientific
The next step is reproduction of the problem. This is a complex step which takes a lot of trial and error using context clues from the error messages, screenshot, and what the player has mentioned in the report. From there we try to piece together what happened and try to recreate the problem. If we can, we record the reproduction steps in the report, before it gets passed to our internal tracker of issues ready for devs to take a crack at. What I want to stress is just knowing there is a problem is not enough to fix it, being able to reproduce the problem is what helps development handle Bugs quickly; reproductions are extremely important. If we cannot reproduce an issue, two things can happen:
- If it’s serious enough it might go to a dev to investigate what a possible source might be. This can be a very long and taxing process when we have very limited information to guide us. A developer could spend that time to fix Bugs we have reproduced reliably.
- It gets placed on-hold until we get reports of it again and reassess the details of the report to see if we can make a reproduction case with the new information. No bugs are ever forgotten and we do occasionally go through our On-Hold list to see if we can make any headway.
Fixing the Problem
Calling in the Cavalry
From there the issue is tagged by management with appropriate labels such as “skills”, “error” or “UI” and filtered to a specific team that would handle those problems. Managers task out these to individual devs when they get dropped off. The developer then fixes the problem.
The Steps:
The steps and time it takes to solve the issue varies dramatically, based on the complexity of the issue and how many other objects the Bug is affected by.
Once a developer has resolved the problem it is sent back into our QA queue for testing. This could take multiple iterations where we test if the Bug is no longer present using our reproduction steps. It might be sent back multiple times depending on if there are side effects to the fix or if it’s not fully resolved.
Once we give a Bugfix the green light, it is processed and patched to our main branch. From there it is tagged for Production, to be a candidate for a hotfix, and cycled into patch releases.
***
We'd like to thank Anna for taking the time out of her day to show us what it takes to bring down the pesky hoard of bugs that threaten our time in Eterra. We hope you've enjoyed this look into Making Last Epoch, and we will see you all again very soon!
