Read First!

Frostion

If there is a problem in regards to bugtracking from the developers side, where developers do not get informed enough about existing bugs, and therefore are out of touch with the game, the players and what's bugging, then it is of great importance to smooth the bug reporting process.

Any decisions and efforts should though be based on the proven fact, that many players/bug-reporters are not and will not be willing/motivated/capable of engaging themselves in GitHub or other more advanced bug reporting solutions. Some players might not want to do anything else, when reporting, than give a one time comment, ingame, via the forum or some other easy and simple method.

Ofcours there is the other side of the spectrum, with players reporting, describing, discussing and helping to test and fix the bug, but we can't expect this from the general player.

LaFayette

@Frostion said in Read First!:

Thanks for the feedback, I'm not sure I agree with this:

Many players/bug-reporters are not and will not be willing/motivated/capable of engaging themselves in GitHub or other more advanced bug reporting solutions.

(1) The github bug report form has a questionnaire, it is more 'guided' compared to a free-form field presented on the forum. In some ways it's less sophisticated, "answer these questions and you're done" compared to figure out how and what to put in a bug report (which leads to bug reports of varying quality and increased question/answer round trips which takes time).

(2) The non-willingness to use github issues I think is mostly, at the risk of causing some offense, established and very loud users. The casual user who has not yet even heard of the forums, let alone created an account here, is probably going to be equal to reporting bugs in either location.

What I do think has been proven is that the bug reporting "flow" to forums is well established, we advertise this in the forum lobby message and maybe that is why we get many bug reports here.

If there is a problem in regards to bugtracking from the developers side, where developers do not get informed enough about existing bugs, and therefore are out of touch with the game, the players and what's bugging, then it is of great importance to smooth the bug reporting process.

Agree on the conclusion. Dev's are split looking at multiple places, this creates some inefficiency, but also of note the forum currently really suffers from not being bug tracking software. For example, if I come back in 2 weeks, will I be able to find forum bug reports that are not solved? Will they have all needed information, perhaps well after the original author has posted and is less likely to engage. For example, we often want/need save-games, they can be really critical, 2 weeks after the fact, it'll be a burden to get that save game if it even still exists.

The biggest benefit I see to forums though is that a wider community is able to help with bug reports. IMO title-hacking to mark which reports as resolved will be a nightmare for tracking bugs that become months old. Simply being able to filter and categorize the bug reports is really valuable, as is being able to assign them to people, and hence I don't think we can simply replace proper bug tracking software with forum posts.

then it is of great importance to smooth the bug reporting process.

So, I really agree with this. The effort to move all bug reports to forums was once attempted and was not successful.

I think then the answer is to formalize how we handle bug reports:

official/unresolved problems opened to github
bug ticket number noted in forum post
forum post title is updated to indicate it is 'handled' or that a bug report issue has been opened
guidelines/training on what information should be included with a bug report so we do not have to constantly ask the same questions and to also make it easier to know how to submit a report and what to include. A problem with free-form text is that can be an impediment in itself. The answer to "report your bug" is much more difficult compared to: "which version of triplea are you using?" "what's your java version?" "Does this problem happen on a specific map" "Can you describe the game play sequence to trigger the bug? Can you describe where the bug occurred?"

I think we can probably achieve a lot of that by having a good "read this first" post that both explains what is needed and how bug reports are handled.

FWIW, the in-game bug reporting feature of TripleA should be live when 1.10 is launched, meaning any in-game crash can be reported from in-game without needing to create any accounts or go to any websites. In that case the crash stack-trace and system info will automatically be included, leaving just an optional text field for users to describe how they hit the problem if they choose to enter that. With luck that will really help for the cases of crashes, but there are still many problems that are outside of crashes, and hence where I strongly agree we need to streamline the process to make it clear and easy so that we can get the problem reports in the first place.

Frostion

@LaFayette said "the in-game bug reporting feature of TripleA should be live when 1.10 is launched"

That is great news. I really think that you can expect a ton of crash reports when this feature is introduced. An in game bug reporting feature should be very convenient for players to use, also it would be great if it leaves it up to the players to decide how much and how advanced the bug reporting input is to be. If the input can be simple from the players side, my guess is that more players would contribute with some bug descriptions. I think it would be equally important to give the players the option to fill out advanced and detailed bug reporting forms when giving their report.

Right now, when the pre-release game engine crashes, a bug report window pops up. So is it this window that will begin to work? I hope that this bug reporting can be supplemented by an ingame menu reporting link, so that the game does not need to crash for players to report bugs.

Last but not least, I can not imagine that in game bug reporting features will make forum bug reporting redundant. There should be an easy accessible place where perceived bugs can be debated with fellow TripleA players.

prastle

@LaFayette Then our priority needs to be 1.10 release Because in game reporting would solve all GREAT STUFF!

prastle

@LaFayette Also I have been afk for a week. Thus I have not read everything. We need a hot link from the forum to the git hub error reporting page that auto creates a tracking number that is added to the forum page. If this can be done without requiring the reporter to have a git hub account that would be great! Unless this has already been done.

LaFayette

I want to say there was some debate on whether to allow for a "free-form" error report. I think I gave it a try and abandoned. It is actually a bit more difficult.

One needs to describe in much more detail the information you would want. There is concern that these reports would not contain everything needed to be useful and actionable. Designing the UI quickly become really questionable/difficult.
The github issue is created by the bot account, responses to that thread are not going to go back to the user unless they proactively follow the thread.

I think the question is which of three options do we want:

(1) allow an in-game error report with the above problems
(2) keep current behavior where in-game error report opens github issues (bug tacker)
(3) change game so that the "report a bug" opens instead the "bug report" section in forums

@prastle

We need a hot link from the forum to the git hub error reporting page that auto creates a tracking number that is added to the forum page.

That is done, when an error report is created, the server responds with a link to the new github issue and the clickable link is displayed to the user.

Captain Crunch

@LaFayette I basically thought a person would play this game and if they found a bug in the map they are playing that they would come to this forum (or the other forum?) and then be told to post a save game and then hopefully if the bug is important enough then some admin or dev would post the bug at Github for them?

I am one who totally wouldn't want to give my email address to another forum or site like Github so my above assessment is the minimum I think any game would need. Anyways, yous got this covered and the 1.10 release sounds good so is everyone happy?

ubernaut

@LaFayette maybe make an option to choose between current behavior and rudimentary auto report?

what happens now if someone tries to report a bug and doesn't have a github account?

LaFayette

Thanks for the feedback @Captain-Crunch , to reply:

@LaFayette I basically thought a person would play this game and if they found a bug in the map they are playing that they would come to this forum

That was the old way we attempted it and made a lot of sense when the bug tracker was the SVN bug tracker and was really hard to find and/or even use. At that time we had multiple forums, and the bug tracker itself was scattered, the effort to look at 'n' places was pretty sizeable. For about the last 3 years the hope/goal was for bug reports to land directly in the bug tracker. Old habits are hard to shake and I'm not sure if it's realistic to have bug reports to always land exactly where they are supposed to go.

Second, map bugs are treated differently from game engine bugs, the two can be hard to distinguish. The game dev's are not going to be in the best position or necessarily able/wanting to fix problems with any/all maps that are pushed.

bug is important enough then some admin or dev would post the bug at Github for them?

I don't think the bar is 'important enough', if something is a problem it should be tracked. One nice thing about being in Github is any dev in the world can jump in. Having a queue of low-hitting, not too difficult to fix bugs is a great thing to have. New devs could consider those to be "trainer" bugs to get their feet wet working on TripleA. A new dev would not want to come in, ramp up on how to develolp and test in TripleA to then do major surgery for a risky change that could be rejected because it is too risky (TripleA is a 'brittle' code base, meaning the most innocuous changes can often cause catastrophic bugs in places you would not think to look; this basically makes it a PITA to work on the TripleA code and you spend 90% of time verifying and testing and not coding, which most dev's really do not like (a ratio of 40% verifying and testing is more reasonable and typical, over that and it speaks to problems in the code base and lack of automated testing.. It's a long effort to reduce the overhead/brittleness and get the amount of automated testing in so we can spend more time code and less verifying)

I am one who totally wouldn't want to give my email address to another forum or site like Github

FWIW, they'll securely store your email address 1000% more than TripleA forums would. Compromising the forums could happen, a site like github where many millions of dollars are involved, payed security experts, and many attackers to fin vulnerabilities, github.com is far more secure to store your email than forums.

Anyways, yous got this covered and the 1.10 release sounds good so is everyone happy?

We're still left in a state where the game menus tell you to post bugs in Github, the lobby tells you to post bugs here. Second, looking at the bugs here, I've no clue what has been addressed vs not. If the idea is we can have the community handle bugs, but if you go away for a week or two and then have to re-read all the bug posts anyways (and then are not sure if someone else has picked them up), it's not terribly effective for either dev's or for people reporting bugs.

My earlier post of re-writing this 'read-this-first' and adding a more clear protocol for forum bugs I think still holds.

LaFayette

@ubernaut said in Read First!:

@LaFayette maybe make an option to choose between current behavior and rudimentary auto report?
what happens now if someone tries to report a bug and doesn't have a github account?

Current behavior in 1.9 is typically "here is a link for reporting bugs".
The auto-report uses the triplea 'bot' account from the backend and does not rely on a personal github account.

You can see some examples of what an auto-report looks like when it lands in github issues here:

ubernaut

@LaFayette so currently there is an auto report option for people without github accounts? if so im not sure i understand how 1 is different from 2.

LaFayette

@ubernaut The auto-report sends data to the TripleA backend which uses its own github account to create the ticket. So yes, no github account required.

Re: 1vs2 - Option 2, currently you have to have a java exception for the auto-report window to show up. Option 1 is in addition to option 2 and would replace the "report a bug" menu item to instead show the auto-report window. Today and with option 2, the 'report a bug' menu item just gives you a link.

ubernaut

@LaFayette gotcha thanks seems like 1 would be better unless you figure more people would opt for the auto report even though they have github accounts.

LaFayette

@ubernaut I don't think creating a github account is actually a barrier for most people. Those participating on the forum here, I respect that the half dozen do not want to divide their attention just as I don't want to divide my attention between most locations.

The real problem with a free-form auto-report feature is ensuring there is good quality to the bug report. The github bug report gives you a series of optional questions that make it easy to know what to include. Without that, a free-form field becomes very daunting to know what to include. If not daunting, there is no way to track it back to someone, so we can't ask for a save-game, or maybe even which map, which phase of game play.

I tried to create a UI that would help guide a user through that series of questions, it started to seem like a lot. In the end the effort is duplicated some, if creating an account on github is too much, there likely is not going to be enough follow-up for a free-form bug report.

LaFayette

Just to contrast, on a game-crash, there is a 'java stack trace' which developers can look at without anything else and often create a fix. Hence the current auto-report feature in prerelease can submit that info to the backend without providing any additional information. The extra info is really gravy and helps the dev team know where/how the problem occurred (sometimes necessary, some traces are really hard to reproduce)

ubernaut

@LaFayette makes sense to me

ubernaut

i think there is something wrong with the battle calculator currently the one land unit must survive checkbox doesn't seem to have any effect on the results.

prastle

@ubernaut and all please read first post!

prastle

@prastle bump link fixed in first post