Is this the official place to report bugs?

Frostion

I will speak my mind freely, and I hope that I am not misunderstood. I don't want to create a split, but I will point out a split that I all ready see.... not necessarily a negative one, but one I think we have to recognize and live with.

I have always felt that the TripleA community is made up of very different people, ranging from people who hang out and do work at Github, who have programming skills, interests and expertise enough to work on the engine, some of them rarely visiting the forum and engaging with that part of the community ... and ranging to the other end of the spectrum; TripleA pure gamers who have little or very limited technical knowledge and no interest either in regards to Github, game engine and even XMLs and game files. Still this group can be enthusiastic fans, playing for fun, contributing to the donation drive, giving great user based feedback, and produce ideas that fuel game development etc. (Of course some people are a mix of the two caricatures and some might not fit at all.)

It was envisioned at the old forum to have the new forum bring people more together, but as I see it, after looking back, people are different and have different interests regarding TripleA. We cannot force people together or expect them to be of equal mind. We cannot look at community members and say, that a new member is the one who has just started to play the game, when he gets more experienced he will engage himself in the technical side of the game, after that he will go and engage himself in Github, and finally when his motivated enough, he will help out in regards to development. That's is only a description of like 1% of TripleA fans. The community as a whole is and will remain a bunch of very different people with different interests .... many of them will NEVER go register at Github just to report a bug, and if they do, not all developers will be satisfied with their maybe "amateurish", un-detailed, lacking bug report anyway, and be irritated about the reporter not conforming to the standard report-form or understanding what people are saying at Github. Finally, if a developer can be absent from the forum for months and if many players (most I would guess) never think about Github or would rather stay away, I see it as the final proof of community members being very different people with very different interests.

My main point is, if TripleA development was to force community members to register at Github because it would be "convenient" for development, I must say that I see a lack of understanding or recognition about how different community members are and what kind of involvement members might want to engage in. A proposal that would demand the casual player to register at Github just to place a bug report (one that he might not even be sure is a bug, or he is "certain", but in reality it isn't) would be a step that excluded voices... a step in the wrong direction if you ask me.

If I should be more constructive, and say something else than "Bug reporting at forum seems to be great", I would propose a new in-game/launcher based bug report system. Maybe it could even auto upload crashes. But then again, the forum's bug report category brings folk into the forum and lets them see the whole thing, maybe motivating them to socialise, engage in discussions, try out map development, maybe even offer to help out with engine development.

RogerCooper

@frostion I would say report your bugs here first, as in many cases what looks like a bug is a misunderstood feature. Then if it can't be resolved, make a formal bug report on GitHub.

Cernel

@lafayette My suggestion is tracking map bugs in forum and engine bugs in the official TripleA issues tracker. This would need to be made clear to the users by updating the @prastle post I linked and changing the name of the subforum to "Map Bug Reports". We can, then, lock all threads that aren't pending map-only issues. Of course, this would require the developers to keep an eye on the forum (probably good anyways), to avoid that engine bugs remain forgotten in here (locking them and telling the users they need to go in GitHub); so, it's your call if you like this idea and want to roll with it. You'll have, then, to decide what to do with the map bug reports in your tracker. Also turning off the issue queue in all map repositories.

Cernel

Also, minor note, it's not really true that currently we have duplicate places for bug reports.

We rather have triplicate places, since there is also the bug reporting in the issue queque of each map.

For example, you can find bug reports about WWII Global in Forum, in the main GitHub tracker and in the issue queque for the map.
Like here you have a few bug reports too:
https://github.com/triplea-maps/world_war_ii_global/issues

Panther

Also we have bug reporters that either don't know about this forum or if they knew, would not create another account here, but would report their issues on their platform.
So we have axisandallies.org as another place for bug reports. Most of those can be resolved easily and quickly as many of them are issues on the user's side. However we had issues in the past that were valid , could not be resolved instantly and thus needed to be transferred to a (whatever) appropriate place (at GitHub in the past).
Personally I would not think of transferring an issue from one forum to another one, if Github appeared to be the appropriate place to note it.

redrum

@panther Yep. Generally, if there is a true engine/map bug not just user error we should look to get it into a github issue as that is the actual queue of bugs to fix.

LaFayette

Good feedback, we've had a number of historical problems to solve, all of these predating this lobby:

users have had trouble finding where to report bugs at all, let alone knowing about forums
admin attention being spread too thin with too much overhead to make much progress beyond just keeping up
issues getting lost/scattered, this is two fold; first the official SVN bug tracker did not have good visibility and its queue grew too long, secondly bugs were tracked outside of the bug tracker which lead to reports getting lost (specifically these are forum reports, if a flurry of reports come in, it's hard to make sure they all get addressed, and often some fall through the cracks particularly when a fix takes a few weeks)

The second item I've viewed as particularly important, we have few devs/admins in this project and the overhead cost for them historically has been really high (arguably the full time job of 1 to 3 people). That's honestly why we moved to github in one way, the efficiency boost for dev helped free us up to do more; maps were moved so that they could be largely self service as well.

Thinking about our goal and pipeline, we want to get bugs fixed and for each one we have to go through a discovery, reporting, (dev) reproduce and fix and then push the fix to prod and then users pick up the updated version. The two bottlenecks there are with reporting, any friction that prevents users from discovering and reporting bugs is something we want fixed, and also fixing bugs is a bottleneck.

To fix a process bottleneck, we can add labor, add automation, remove tasks. To that extent, TripleA is fortunate to have a large and knowledgeable volunteer base, we can remove burden from devs and add labor if we are able to effectively to get volunteers to help with incoming bugs.

At the same time, we don't want volunteers to be bottlenecks themselves, so we need to think of the bug intake process as checking in to the airport, the volunteers are not the people doing the check-ins, they are the people walking around making sure everyone is okay using the automated check-in machines. As part of that, there is key information needed when reporting bugs so devs know what to look for and fix. Part of the inefficiency of fixing bugs is repeated asking for information that is needed before starting, to help solve that questions/check lists and automated tools are good answers so it's much easier more natural.

So, 2~3 years ago it was really not clear where to report bugs, and we did not have this forum. There was a concerted effort to try and improve efficiency and increase the rate bug reports by making the bug reporting steps relatively clear and directing people to the bug system. Hence, my concern when I realize we have developed a parallel flow here for bugs, which in an unstructured process really backtracks and could re-invite many of the important problems that we have almost left behind us.

Sorry for not having a lot of time to be able to shorten that down, it's hard to answer in short.. Hopefully the problems and motivations make it more clear we are just looking at a different set of problems and the shared perspective I hope helps.

With a really long story short, I thnk the way this is helped is simply by merging/combining the bug report section with the help section forums. Looking at the types and content of threads between the two, they are really similar. Once they are clarified and combined, and it made a bit more clear that is the place for questions and official bugs are put in to a bug tracker, I think things would be more clear for the new players who will be joining us over the next few years (an easy bug report process is important for them, and I'm most concerned about attracting and retaining new players, growing community IMO is top importance)

Cernel

I believe the Bug Reports subsection was made because users would mostly not go reporting in GitHub, nor they would post in a specific place, nor the mods would keep traking the bug reports down and moving them to GitHub or even in a single place in the forum, but there were tons of bug reports lost spuriously in the various subsections, and diluted in between of all non-bugs posts in there.

If consolidating, I believe map bugs should better stay in "Maps & Mods", not in "Help & Questions", and preferably rather in a subsection of it.

LaFayette

@cernel said in Is this the official place to report bugs?:

If consolidating, I believe map bugs should better stay in "Maps & Mods", not in "Help & Questions", and preferably rather in a subsection of it.

Agree.

Thanks for background on why the 'bugs' category is needed - makes sense.

Is there agreement it looks to make sense to combine "Help & Questions" + "Bugs"? (please check out the thread posts between the two, they seem pretty similar topics). If that sounds right, here's a first list of potential titles for a new combined category I can suggest:

"Help, Bugs & Questions"
"Help, Bug Reports & Questions"
"Help, Problems & Questions"
"Bug Reports & Questions"
"Game Help & Bugs"

beelee

@lafayette fwiw "Game Help & Bugs" jumped out the most for me.

Frostion

I think it is a mistake to once again try to rearrange forum categories or remove entire categories in an effort to control how people report bugs and give feedback. The forum works fine. I can’t see the problems described in this thread as being related to bad categorization at this forum. People post about bugs here, and if they possess the know-how, they will post at GuitHub also or instead. If they don’t possess the know-how, somebody else will maybe post for them at GitHub if the bug they point out is actually a bug and/or if it is important enough for developers to tend to.

Some of the largest and hottest games in the world 2018 have both support and bug categories in their forums, to ask for help and to discuss (potential) bugs:

League of Legends has “Ask the Community” and “Report A Bug” as separate forum categories. (https://boards.na.leagueoflegends.com/en/)

Fortnite has “Technical support” and “Bug Reports” as separate forum categories. (https://www.epicgames.com/fortnite/forums)

PlayerUnknown's Battlegrounds have three categories: “General help”, “Troubleshooting” and “Bug Reports” (https://forums.playbattlegrounds.com/)

Warcraft has “Technical Support” and “Bug Report” as different categories (https://us.battle.net/forums/en/wow/)

And the list could go on … naturally … as it is a good idea to have both help and bug categories at a game forum, and serious games therefore have both. Meaning open doors for players to both discuss and report game related problems.

“Once they are clarified and combined, and it made a bit more clear that is the place for questions and official bugs are put in to a bug tracker, I think things would be more clear for the new players”

I think it is wishful thinking to believe that cutting down categories and mixing bug reporting with player support and inquiries (“Help & Questions” and “Bug Reports”) will do anything good or make bug reports more clear. On the contrary I think that it will make it even harder for devs to identify bug reports (as they will be mixed with a lot of other inquiries) and it would also be harder for players to find out where and what to post.

I am sorry if I am repeating myself, but I sense that we are maybe trying to fix something that isn’t broken, maybe even creating more problems in the process.

LaFayette

@frostion said in Is this the official place to report bugs?:

. People post about bugs here, and if they possess the know-how, they will post at GuitHub also or instead.

This is the thing, not having that know-how means we have failed to provide a reasonable way to report bugs. Having multiple ways to do that makes the problem worse. To boot, there is confusion and if you get help on how to post bugs, depending on who you ask, you'll get conflicted answers. Bugs on forums, if they never make it to bug tracker, will get lost, and if they do not get lost then they add a really excessive amount of overhead. Scenario, dev finds a bug report a week after the fact, asks for some important information and original poster is long gone. Then to boot, that thread goes to the top and there is no way to mark it's just waiting, so others devs are going to open that, read through it and lose time realizing it's in a waiting state.

@Frostion , that list of games, none are open source. The bug trackers for each of those would be closed source products not available to the general public. So in each case it seems they all have one way to report bugs. Surely those are not the official bug queues developers use. There is a reason bug tracker is a dedicated piece of software, whether that is github issues, JIRA, svn bugs, or what not - there are some key things a bug tracker needs to do, and forums does not satisfy those needs.

In the examples, I'm pretty sure the bug forums makes a lot of sense as it's the way for users to report bugs, it's split and different enough from other categories that the posts there share a common theme.

For us though, I'm seeing almost the same thread topics between Help and Bugs. I don't think our Help section is currently useful because of that, it's a toss-up to post in help or bugs. What makes it bad, is that we don't have an official policy for how we accept bugs anymore. Again, a bug report in forum that does not make it into tracker is a loss for everyone; so whatever we decide we need to be able to avoid that. Again, community volunteers should not be the ones taking bug reports, but they should be helping to make sure they are reported properly.

On the contrary I think that it will make it even harder for devs to identify bug reports (as they will be mixed with a lot of other inquiries) and it would also be harder for players to find out where and what to post.

I have trouble seeing the evidence for that. There are many wrongly placed items in the help:

slow AI turns after 5+rounds https://forums.triplea-game.org/topic/939/slow-ai-turns-after-5-rounds (this is a performance bug)
java error - https://forums.triplea-game.org/topic/926/java-error, this is a straight bug, should not be in help
https://forums.triplea-game.org/topic/890/show-code-for-dice-algorithm, this is a feature request, should not be in help
https://forums.triplea-game.org/topic/880/error-message, another bug report
registration email never received: https://forums.triplea-game.org/topic/862/registering-email-issue, bug

The above list is from the first two pages, I think that already shows it's hard to identify the bug reports that are posted, so I do not agree with " On the contrary I think that it will make it even harder for devs to identify bug reports", as is I see it is already 'harder' for devs. Devs essentially have to scan forums, all threads periodically to check and make sure that issues are not getting lost here (which happened in old forums, I see it happening here again too, as some of those bug reports in help are new to me and there are fixes that could be done to help those and are not done yet).

LaFayette

I think there is one point that is getting in the way, whether bugs need to get into the bug tracker or if it's sufficient to just list them on forums. As far as that goes, this project always has had a bug tracker and always needed bugs to get into the tracker.

The old bug tracker I think we collectively keep forgetting about, but it's always been around and still is even: https://sourceforge.net/p/triplea/bugs/?source=navbar

As far as it goes whether devs should use or need bug tracking software, it's a debate I never thought I would have necessarily. As is, forums is not set up to be a bug tracker, and that does not work for development.

There are lots of online resources about software project management and how a bug tracker fits into a software project (a bug tracker is one of a few central project tracking components, up there with source code management software):

Frostion

(some) people will always post about bugs in the wrong place, even if a category spelled out "NOT FOR BUG REPORTING!" I think we could see bug reports being posted. One of the forums listed above has a similar capital letter announcement in a category description, so I think what we see at our forum is not a unique phenomena. I think this is a good opportunity to remind members with moderator privileges to move all wrongly placed posts they see, of course only if the post can be determined as wrongly placed. I would say that many posts are actually a bit hard to categorise or define as bug reports, so no wonder bug reporters some times pick the wrong category.

I would say that the only way to have all players know exactly how and where to place bug reports would be to have a bug report button or bug report info at the TripleA program's main menu. The simple version could just be a pop up with text instructions and a link. The advanced maybe a fill out form and maybe a possibility to upload a file (screenshot or txt?). Just some sort of ingame bug reporting help, then devs could just stick to these reports without worrying about bugs being lost or misplaced. Maybe the bug tracker chosen for TripleA should be very user friendly? If the bug tracker is to complicated to use for the common player, I fear we can expect players to not report the bugs they encounter.

There must be some old experiences with the old bug tracker. Was it only the same few "power users" who posted about bugs? Or was the tracker friendly enough for everyone to use? I think that the format chosen will determine if the devs want everyone to be able to post, or if they feel fine with just the elite few players and their reports, just like Github feels now. Nothing wrong in my mind with the latter if the development runs fine without input from the masses, but it would maybe be a bit excluding. And in this case, the forum could just remove all involvement in regards to bugs, and everyone can just point towards the official tracker.

Cernel

I mostly agree with @frostion, but here it seems to me that we are going in circles a lot, also repeating stuff that has already been said several times in the many pages long Bunker topic I linked, and the more a topic keeps going in circles, the more the text added to it, the less the chance that all the topic will be fully read, thus the more and more the circling; this one might have already gone out of control, practically.

@frostion said in Is this the official place to report bugs?:

There must be some old experiences with the old bug tracker. Was it only the same few "power users" who posted about bugs? Or was the tracker friendly enough for everyone to use?

I've almost never seen known active people in the lobby posting in the old tracker, but really the matter is of little relevance, as for many years in sourceforge TripleA has remained very stable, so the addition of new bugs was near to non-existent, and veqryn pretty much knew all that was pending, that were all marginal or arguable matters. Really totally a different scenario from today, or the last years anyways.

Cernel

For several years when TripleA was really in icebox almost all new I was seeing in the sourceforge tracker was veqryn closing as invalid or duplicate rare bug reports from people whose names I had never seen before.

LaFayette

Thank you @Frostion , @Cernel for the dialog. We are talking in circles a bit, but I do think we are at least talking from a common point of view now. TripleA is pretty complex and it's interesting to think it is a half dozen people running this with a larger community.

The thing that I have a hang up over, just a problem I can't see how to solve, if we ask for bug reports as a bug report in forum, it will always fall on someone to move it to the official tracker. Maybe that is good, and if someone then sees that happen a few times and they themselves become a more active volunteer, then they'll cut bug tickets directly into the tracker. Really, it's how do we want this designed?

I like the mention that the old tracker and new are pretty different. That is a good important thing to keep in mind and the fact it's easier to see/add/update tickets in github issues is significant. The old tracker, I would have supported that being for absolutely just the devs, I still can't figure out how to mark tickets as fixed in it

I think we have two decisions to make:

Do we encourage users to go directly to bug tracker, or forums? There are in-game links, help menu, lobby message that direct users to report bugs directly to bug tracker. We could update those, but overall it's a process/flow question. I think we have two realistic answers here:
a) let's encourage bug reports to go directly to bug tracker
b) encourage bug reports to go to forums, encourage experienced users to user tracker (github issues) directly
Do we merge 'help' + 'bug' categories? If we go with 1a, then it probably makes sense to do this. If we go with 1b, it will make complete sense do keep the bug category.

If we can formalize a way that 1b works, that would be best from a labor perspective. People that are experienced and hanging around the forums can add their labor which saves from dev efforts (we want devs coding). At the same time, repeating myself here, we do not want the forum admins/moderator volunteers to be a bottleneck either, which makes me think this might be a risky idea. That has me leaning towards 1a, and then combining the help and bug report categories, which is effectively a "we'll help you here if you're not sure, but the official bug reports go elsewhere"

Sidenote: i'm heading out of town for a couple weeks and will not be able to follow up. It is just as well IMO, I think i've exhausted my thinking about this and have hit a brickwall. It's hard to balance getting new users onboarded, bug reports going where we want, using available talent and labor, and it being clear and simple.

I do really appreciate we are working through what has been a contentious issue : ) I think we'll get this solved for real. While migrating there was a lot of updates for old bug tracker to new, that was a dev only effort, and we had a big split before in the different groups within TripleA. We're now a more united community IMO, so it seems appropriate that we're now dealing with, okay, this was how 'devs' dealt with bugs, how should TripleA as a full holistic community deal with them?

Frostion

I think option 1b (Encourage common players to report bugs at forum, and advanced users to go directly to Github)

This is because I don't see a bottle neck. I see the forum as a filter where possible bugs are posted, evaluated by more competent people and identified as being an actual real bug or not. Also, at the forum, it can be determined if the reporter himself is capable of and competent enough to post at Github, describing and discussing his bug. If not, someone else could/should go to Github with the bug ... in best case the reporter has the drive and motivation to learn to do it himself.

I think this option ensures the most bug reporting (and broadest community inclusion ... something needed to recruit and create development and mapmaking interest ) and also it should ensure some level of quality when it comes to what is posted at Github, so Github does not become flooded with questionable and debatable reports. Hence the "filter".

I must admit that I am not at all competent to say what would or could lead to the best working environment for developers at Github, that is of course something the devs should know and determin. So I wish more devs would give some input here. But I do feel that the rest of my analysis is fairly reasonable

LaFayette

1b works if it is formalized. The 'bottleneck' comment should not be misconstrued please, it's more a matter of looking at
the process flow. One process being considered (1a) has a direct intake to bug tracker, while 1b has an intake to forum and output to bug tracker. The conversion from forum post to bug tracking ticket will not be immediate nor 1:1, by definition a bottleneck as we will be waiting for humans to do something (and no matter how diligant or fast, that will introduce some delay into the process). Whether it is a big impact or not is another and important matter. The impact will be inconsistent, we need to design this process so that this will not become a problem and we will not see a process break down.

How does 1b break down? what do we need to avoid/solve?

appropriate culture of 'always do something' not instilled. For example, in: https://forums.triplea-game.org/topic/920/some-help-me-please/3, the "your internet is not avaialble" warning message was misconstrued and reported as an error itself. A response of 'ya, that is normal, just close it and continue' is an example of where we do not have that culture. No UI notification should ever be routinely ignored, and secondly the warning message is clearly not helpful in its current form. So the action item is to either remove it or fix it, but we can't just say "yeah, that was your problem", because it was not the users problem, it was our poor design and lack of time/labor for code development (which is compounded by really high overhead processes and high overhead code).
potentially times of low capacity, people get busy; If we have only 3-5 people moving tickets, and 2 or 3 of them happen to coincide by being away and/or busy at work or what not, then we'll have pretty limited capacity and forum posts would build up. Dev's would jump in for these situations, but that is then reducing time available for devs to code, and it's hard to then justify having the extra reporting step.
we simply lose track if something has been moved and do not get around to it. This report: https://forums.triplea-game.org/topic/957/pbem-pbf-combat-move-check-box/8 is potentialy an example, the reported bug has not made it into tracker. With more time, the post will fall from the top and would be unlikely to ever be fixed.

For this process, the success and failure metrics I would choose would be (so these are numbers that we should work to optimize and track to know that we are doing well):

number of forum posts that are handled without needing a ticket. Sometimes questions just need to be asked. (We would need to be sure to count this metric very conservatively, most user reports should result in some action, even if the 'system is working as intended', if a user can't figure out what that intent is, it may as well not be working).
number of forum posts that result in a bug tracking ticket or are de-duped against an already open ticket. If the ratio of this number to the total number of posts start to drop, then we are falling behind
the amount of time between when a forum post is opened and when a bug report is created
number of bug reports (forum or github issue, any kind of report) that is filed in the wrong place
number of github bug tracking tickets missing important information

Cernel

Also, in past there used to be in lobby a link to where to post bugs. I guess you removed since you can click on Help (not so obvious, in my opinion), but just saying in case.

Maybe off topic but still talking about bugs, this is what happens when you are in the lobby and want to see how to report a bug:
You click on "Help"
Click on "Help Page"
Get redirected to this page:
https://triplea-game.org/help/
making you click to go to:
https://forums.triplea-game.org/category/10/help-questions
at this point, someone that is searching for bug reporting will hopefully click on:
https://forums.triplea-game.org/topic/526/how-to-submit-a-bug-report
where you find the link to opening a new issue in the tracker (need to register to GitHub, of course).
Side note, at this topic, the:
"Open Bug List"
and
"Lobby is on Fire"
links both have GitHub telling you:
"No results matched your search."
On the other hand, if the user would, instead, decide to click on:
https://forums.triplea-game.org/topic/13/faq-and-help
and then on:
To submit bug reports: http://www.triplea-game.org/bug_report/
he would get a:
404
File not found
(or at least this is what I'm getting)

So maybe in the moment you get a bug while you are in the lobby, knowing how to report it may be not very clear. Maybe it was better the old practice to have just the link to where to bug report at the start of lobby chatlog (your decision), for the users to copy-paste.
Anyways, not so important, as I believe lobby users very rarely report bugs anywhere (sadly), so having a link there won't do much to increase bug reporting, I guess.