Read First!
-
The Developers would like to have a moratorium on New! forum related bug posts for a bit! They have developed a new bug reporting system to post directly to GitHub which would make things easier for them. (Thus they are not having to check multiple posts and sites) Can we please all attempt to use this new system.
Less work for the devs =s more time fixing issues
Bugs can now be reported at this link!Thanks Pras!
Please post any perceived bugs here from older maps/engines here. For any map that is in development please post the perceived bug in that map makers thread. A Developer can then look in one spot and answer or close the issue with a confirmed repair for older issues. (Feel free to edit the title etc Devs)
-
@prastle @Cernel Is it time we merged "Bug Reports" category into 'Help & Questions'?
(1) The game engine is contradictory to this category and directs users to submit bug reports directly to the bug tracker.
(2) We've agreed it's most efficient to direct users to bug tracker first and not have a meta-task of moving reports
(3) Help & Questions i basically the same thing, why do we need 2 categories?
(4) There are a number of issues with how we track bugs if we have a forum category to track bugs, and a bug tracker. Sorry to be redundant, in softare development, bug tracking is specific software to track tasks. 'Forum' software is not a bug tracker. There are a few key apps for any software project, bug tracking is one.
(5) The split of bug report and help & questions does not help. If you have questions, your first question is perhaps if you have a question or if you have a bug. Then there is the 'read this first' message to understand and then compare with the ''help & questions" category.
-
@LaFayette Help and Questions would not make any sense in the moment I surely spot a map bug and want to tell the mapmaker about it. In this case, I would rather tell the mapmaker in an open thread in Maps & Mods, or open a new one myself.
Often with bugs you don't need help (because you can't be helped) and you are not asking questions, you just notice something that ain't right (it's rather the opposite: you are helping the mapmakers or the developers to fix stuff).
I also disagreed regarding splitting the process between maps in development or not, as per current (and I recall @Frostion did too).
-
@LaFayette NP here as long as normal forum users can find a way to post when they do not use git.
-
If a bug report does not make it to the developers, it's not going to be helpful. On the other hand answering questions from users and resolving those problems is very useful.
With that I have two concerns:
- (1) Spreading developer attention thin. Time is limited and there is much to do, checking many places for otherwise the same information is inefficient.
- (2) Non-resolution of issues. It's very easy for 'work-arounds' to become the general practice instead of actually fixing issues. For example if there is a persistent bug due to old java versions and the advice is to keep upgrading java, it's a work-around and real fix is to see about bundling a latest java (or something else). We need to be sure we do not keep getting the same error reports over and over again and there is a long-term fix.
With all that mentioned, our bug tracker software is 'Github issues'. I've thought it's most efficient then and in-line with the above two concerns if we can funnel all bug reports there.
For various reasons that does not seem to have worked out:
- reluctance to create new accounts
- forum community reluctant to engage on official bug tracker (and they themselves suffer from the problem of spreading attention thin as they are focused on forums)
- mixed messaging, for example the lobby welcome message says to post here but the menu links and official documentation for posting bugs says to go to github issues.
It's also been somewhat long standing practice to have bug reports first go to forums and then 'official' reports go to the bug tracker. I think the historical roots were due to the SVN source forge bug tracker being hard to find and use. Regardless it seems like that is the prevailing current and I don't think fighting against it is working.
So, I think we need to do a few things (cc: @prastle ) :
(1) Update this post and re-write it a bit. We should include more information on what should be posted in a bug report.
(2) Standardize how we handle the bug reports and create some guidelines. For example the people that can help with bug reports should know that real bugs and/or problems should get posted in the bug tracker. I'd also like for us to start updating the title of bug reports with "[RESOLVED]" to indicate which problems have been fixed.
(3) Update our official documentation to be more clear about what to post in github bug tracker vs a forum questioncc:/ @redrum , curious your thoughts on how this process should look. Ultimately my desire is that we optimize for making it easy to report problems so that we can become aware of them, and secondly optimize for maintainer efficiency so our efforts can go as far as possible. I do very much like that there is a larger community base on forums that can often resolve problems without needing maintainer efforts.
-
If there is a problem in regards to bugtracking from the developers side, where developers do not get informed enough about existing bugs, and therefore are out of touch with the game, the players and what's bugging, then it is of great importance to smooth the bug reporting process.
Any decisions and efforts should though be based on the proven fact, that many players/bug-reporters are not and will not be willing/motivated/capable of engaging themselves in GitHub or other more advanced bug reporting solutions. Some players might not want to do anything else, when reporting, than give a one time comment, ingame, via the forum or some other easy and simple method.
Ofcours there is the other side of the spectrum, with players reporting, describing, discussing and helping to test and fix the bug, but we can't expect this from the general player.
-
@Frostion said in Read First!:
Thanks for the feedback, I'm not sure I agree with this:
Many players/bug-reporters are not and will not be willing/motivated/capable of engaging themselves in GitHub or other more advanced bug reporting solutions.
(1) The github bug report form has a questionnaire, it is more 'guided' compared to a free-form field presented on the forum. In some ways it's less sophisticated, "answer these questions and you're done" compared to figure out how and what to put in a bug report (which leads to bug reports of varying quality and increased question/answer round trips which takes time).
(2) The non-willingness to use github issues I think is mostly, at the risk of causing some offense, established and very loud users. The casual user who has not yet even heard of the forums, let alone created an account here, is probably going to be equal to reporting bugs in either location.
What I do think has been proven is that the bug reporting "flow" to forums is well established, we advertise this in the forum lobby message and maybe that is why we get many bug reports here.
If there is a problem in regards to bugtracking from the developers side, where developers do not get informed enough about existing bugs, and therefore are out of touch with the game, the players and what's bugging, then it is of great importance to smooth the bug reporting process.
Agree on the conclusion. Dev's are split looking at multiple places, this creates some inefficiency, but also of note the forum currently really suffers from not being bug tracking software. For example, if I come back in 2 weeks, will I be able to find forum bug reports that are not solved? Will they have all needed information, perhaps well after the original author has posted and is less likely to engage. For example, we often want/need save-games, they can be really critical, 2 weeks after the fact, it'll be a burden to get that save game if it even still exists.
The biggest benefit I see to forums though is that a wider community is able to help with bug reports. IMO title-hacking to mark which reports as resolved will be a nightmare for tracking bugs that become months old. Simply being able to filter and categorize the bug reports is really valuable, as is being able to assign them to people, and hence I don't think we can simply replace proper bug tracking software with forum posts.
then it is of great importance to smooth the bug reporting process.
So, I really agree with this. The effort to move all bug reports to forums was once attempted and was not successful.
I think then the answer is to formalize how we handle bug reports:
- official/unresolved problems opened to github
- bug ticket number noted in forum post
- forum post title is updated to indicate it is 'handled' or that a bug report issue has been opened
- guidelines/training on what information should be included with a bug report so we do not have to constantly ask the same questions and to also make it easier to know how to submit a report and what to include. A problem with free-form text is that can be an impediment in itself. The answer to "report your bug" is much more difficult compared to: "which version of triplea are you using?" "what's your java version?" "Does this problem happen on a specific map" "Can you describe the game play sequence to trigger the bug? Can you describe where the bug occurred?"
I think we can probably achieve a lot of that by having a good "read this first" post that both explains what is needed and how bug reports are handled.
FWIW, the in-game bug reporting feature of TripleA should be live when 1.10 is launched, meaning any in-game crash can be reported from in-game without needing to create any accounts or go to any websites. In that case the crash stack-trace and system info will automatically be included, leaving just an optional text field for users to describe how they hit the problem if they choose to enter that. With luck that will really help for the cases of crashes, but there are still many problems that are outside of crashes, and hence where I strongly agree we need to streamline the process to make it clear and easy so that we can get the problem reports in the first place.
-
@LaFayette said "the in-game bug reporting feature of TripleA should be live when 1.10 is launched"
That is great news. I really think that you can expect a ton of crash reports when this feature is introduced. An in game bug reporting feature should be very convenient for players to use, also it would be great if it leaves it up to the players to decide how much and how advanced the bug reporting input is to be. If the input can be simple from the players side, my guess is that more players would contribute with some bug descriptions. I think it would be equally important to give the players the option to fill out advanced and detailed bug reporting forms when giving their report.
Right now, when the pre-release game engine crashes, a bug report window pops up. So is it this window that will begin to work? I hope that this bug reporting can be supplemented by an ingame menu reporting link, so that the game does not need to crash for players to report bugs.
Last but not least, I can not imagine that in game bug reporting features will make forum bug reporting redundant. There should be an easy accessible place where perceived bugs can be debated with fellow TripleA players.
-
@LaFayette Then our priority needs to be 1.10 release
Because in game reporting would solve all GREAT STUFF! -
@LaFayette Also I have been afk for a week. Thus I have not read everything. We need a hot link from the forum to the git hub error reporting page that auto creates a tracking number that is added to the forum page. If this can be done without requiring the reporter to have a git hub account that would be great! Unless this has already been done.
-
I want to say there was some debate on whether to allow for a "free-form" error report. I think I gave it a try and abandoned. It is actually a bit more difficult.
- One needs to describe in much more detail the information you would want. There is concern that these reports would not contain everything needed to be useful and actionable. Designing the UI quickly become really questionable/difficult.
- The github issue is created by the bot account, responses to that thread are not going to go back to the user unless they proactively follow the thread.
I think the question is which of three options do we want:
- (1) allow an in-game error report with the above problems
- (2) keep current behavior where in-game error report opens github issues (bug tacker)
- (3) change game so that the "report a bug" opens instead the "bug report" section in forums
We need a hot link from the forum to the git hub error reporting page that auto creates a tracking number that is added to the forum page.
That is done, when an error report is created, the server responds with a link to the new github issue and the clickable link is displayed to the user.
-
@LaFayette I basically thought a person would play this game and if they found a bug in the map they are playing that they would come to this forum (or the other forum?) and then be told to post a save game and then hopefully if the bug is important enough then some admin or dev would post the bug at Github for them?
I am one who totally wouldn't want to give my email address to another forum or site like Github so my above assessment is the minimum I think any game would need. Anyways, yous got this covered and the 1.10 release sounds good so is everyone happy?
-
@LaFayette maybe make an option to choose between current behavior and rudimentary auto report?
what happens now if someone tries to report a bug and doesn't have a github account?
-
Thanks for the feedback @Captain-Crunch , to reply:
@LaFayette I basically thought a person would play this game and if they found a bug in the map they are playing that they would come to this forum
That was the old way we attempted it and made a lot of sense when the bug tracker was the SVN bug tracker and was really hard to find and/or even use. At that time we had multiple forums, and the bug tracker itself was scattered, the effort to look at 'n' places was pretty sizeable. For about the last 3 years the hope/goal was for bug reports to land directly in the bug tracker. Old habits are hard to shake and I'm not sure if it's realistic to have bug reports to always land exactly where they are supposed to go.
Second, map bugs are treated differently from game engine bugs, the two can be hard to distinguish. The game dev's are not going to be in the best position or necessarily able/wanting to fix problems with any/all maps that are pushed.
bug is important enough then some admin or dev would post the bug at Github for them?
I don't think the bar is 'important enough', if something is a problem it should be tracked. One nice thing about being in Github is any dev in the world can jump in. Having a queue of low-hitting, not too difficult to fix bugs is a great thing to have. New devs could consider those to be "trainer" bugs to get their feet wet working on TripleA. A new dev would not want to come in, ramp up on how to develolp and test in TripleA to then do major surgery for a risky change that could be rejected because it is too risky (TripleA is a 'brittle' code base, meaning the most innocuous changes can often cause catastrophic bugs in places you would not think to look; this basically makes it a PITA to work on the TripleA code and you spend 90% of time verifying and testing and not coding, which most dev's really do not like (a ratio of 40% verifying and testing is more reasonable and typical, over that and it speaks to problems in the code base and lack of automated testing.. It's a long effort to reduce the overhead/brittleness and get the amount of automated testing in so we can spend more time code and less verifying)
I am one who totally wouldn't want to give my email address to another forum or site like Github
FWIW, they'll securely store your email address 1000% more than TripleA forums would. Compromising the forums could happen, a site like github where many millions of dollars are involved, payed security experts, and many attackers to fin vulnerabilities, github.com is far more secure to store your email than forums.
Anyways, yous got this covered and the 1.10 release sounds good so is everyone happy?
We're still left in a state where the game menus tell you to post bugs in Github, the lobby tells you to post bugs here. Second, looking at the bugs here, I've no clue what has been addressed vs not. If the idea is we can have the community handle bugs, but if you go away for a week or two and then have to re-read all the bug posts anyways (and then are not sure if someone else has picked them up), it's not terribly effective for either dev's or for people reporting bugs.
My earlier post of re-writing this 'read-this-first' and adding a more clear protocol for forum bugs I think still holds.
-
@ubernaut said in Read First!:
@LaFayette maybe make an option to choose between current behavior and rudimentary auto report?
what happens now if someone tries to report a bug and doesn't have a github account?Current behavior in 1.9 is typically "here is a link for reporting bugs".
The auto-report uses the triplea 'bot' account from the backend and does not rely on a personal github account.You can see some examples of what an auto-report looks like when it lands in github issues here:
-
@LaFayette so currently there is an auto report option for people without github accounts? if so im not sure i understand how 1 is different from 2.
-
@ubernaut The auto-report sends data to the TripleA backend which uses its own github account to create the ticket. So yes, no github account required.
Re: 1vs2 - Option 2, currently you have to have a java exception for the auto-report window to show up. Option 1 is in addition to option 2 and would replace the "report a bug" menu item to instead show the auto-report window. Today and with option 2, the 'report a bug' menu item just gives you a link.
-
@LaFayette gotcha thanks seems like 1 would be better unless you figure more people would opt for the auto report even though they have github accounts.
-
@ubernaut I don't think creating a github account is actually a barrier for most people. Those participating on the forum here, I respect that the half dozen do not want to divide their attention just as I don't want to divide my attention between most locations.
The real problem with a free-form auto-report feature is ensuring there is good quality to the bug report. The github bug report gives you a series of optional questions that make it easy to know what to include. Without that, a free-form field becomes very daunting to know what to include. If not daunting, there is no way to track it back to someone, so we can't ask for a save-game, or maybe even which map, which phase of game play.
I tried to create a UI that would help guide a user through that series of questions, it started to seem like a lot. In the end the effort is duplicated some, if creating an account on github is too much, there likely is not going to be enough follow-up for a free-form bug report.
-
Just to contrast, on a game-crash, there is a 'java stack trace' which developers can look at without anything else and often create a fix. Hence the current auto-report feature in prerelease can submit that info to the backend without providing any additional information. The extra info is really gravy and helps the dev team know where/how the problem occurred (sometimes necessary, some traces are really hard to reproduce)
