Is this the official place to report bugs?

Frostion

(some) people will always post about bugs in the wrong place, even if a category spelled out "NOT FOR BUG REPORTING!" I think we could see bug reports being posted. One of the forums listed above has a similar capital letter announcement in a category description, so I think what we see at our forum is not a unique phenomena. I think this is a good opportunity to remind members with moderator privileges to move all wrongly placed posts they see, of course only if the post can be determined as wrongly placed. I would say that many posts are actually a bit hard to categorise or define as bug reports, so no wonder bug reporters some times pick the wrong category.

I would say that the only way to have all players know exactly how and where to place bug reports would be to have a bug report button or bug report info at the TripleA program's main menu. The simple version could just be a pop up with text instructions and a link. The advanced maybe a fill out form and maybe a possibility to upload a file (screenshot or txt?). Just some sort of ingame bug reporting help, then devs could just stick to these reports without worrying about bugs being lost or misplaced. Maybe the bug tracker chosen for TripleA should be very user friendly? If the bug tracker is to complicated to use for the common player, I fear we can expect players to not report the bugs they encounter.

There must be some old experiences with the old bug tracker. Was it only the same few "power users" who posted about bugs? Or was the tracker friendly enough for everyone to use? I think that the format chosen will determine if the devs want everyone to be able to post, or if they feel fine with just the elite few players and their reports, just like Github feels now. Nothing wrong in my mind with the latter if the development runs fine without input from the masses, but it would maybe be a bit excluding. And in this case, the forum could just remove all involvement in regards to bugs, and everyone can just point towards the official tracker.

Cernel

I mostly agree with @frostion, but here it seems to me that we are going in circles a lot, also repeating stuff that has already been said several times in the many pages long Bunker topic I linked, and the more a topic keeps going in circles, the more the text added to it, the less the chance that all the topic will be fully read, thus the more and more the circling; this one might have already gone out of control, practically.

@frostion said in Is this the official place to report bugs?:

There must be some old experiences with the old bug tracker. Was it only the same few "power users" who posted about bugs? Or was the tracker friendly enough for everyone to use?

I've almost never seen known active people in the lobby posting in the old tracker, but really the matter is of little relevance, as for many years in sourceforge TripleA has remained very stable, so the addition of new bugs was near to non-existent, and veqryn pretty much knew all that was pending, that were all marginal or arguable matters. Really totally a different scenario from today, or the last years anyways.

Cernel

For several years when TripleA was really in icebox almost all new I was seeing in the sourceforge tracker was veqryn closing as invalid or duplicate rare bug reports from people whose names I had never seen before.

LaFayette

Thank you @Frostion , @Cernel for the dialog. We are talking in circles a bit, but I do think we are at least talking from a common point of view now. TripleA is pretty complex and it's interesting to think it is a half dozen people running this with a larger community.

The thing that I have a hang up over, just a problem I can't see how to solve, if we ask for bug reports as a bug report in forum, it will always fall on someone to move it to the official tracker. Maybe that is good, and if someone then sees that happen a few times and they themselves become a more active volunteer, then they'll cut bug tickets directly into the tracker. Really, it's how do we want this designed?

I like the mention that the old tracker and new are pretty different. That is a good important thing to keep in mind and the fact it's easier to see/add/update tickets in github issues is significant. The old tracker, I would have supported that being for absolutely just the devs, I still can't figure out how to mark tickets as fixed in it

I think we have two decisions to make:

Do we encourage users to go directly to bug tracker, or forums? There are in-game links, help menu, lobby message that direct users to report bugs directly to bug tracker. We could update those, but overall it's a process/flow question. I think we have two realistic answers here:
a) let's encourage bug reports to go directly to bug tracker
b) encourage bug reports to go to forums, encourage experienced users to user tracker (github issues) directly
Do we merge 'help' + 'bug' categories? If we go with 1a, then it probably makes sense to do this. If we go with 1b, it will make complete sense do keep the bug category.

If we can formalize a way that 1b works, that would be best from a labor perspective. People that are experienced and hanging around the forums can add their labor which saves from dev efforts (we want devs coding). At the same time, repeating myself here, we do not want the forum admins/moderator volunteers to be a bottleneck either, which makes me think this might be a risky idea. That has me leaning towards 1a, and then combining the help and bug report categories, which is effectively a "we'll help you here if you're not sure, but the official bug reports go elsewhere"

Sidenote: i'm heading out of town for a couple weeks and will not be able to follow up. It is just as well IMO, I think i've exhausted my thinking about this and have hit a brickwall. It's hard to balance getting new users onboarded, bug reports going where we want, using available talent and labor, and it being clear and simple.

I do really appreciate we are working through what has been a contentious issue : ) I think we'll get this solved for real. While migrating there was a lot of updates for old bug tracker to new, that was a dev only effort, and we had a big split before in the different groups within TripleA. We're now a more united community IMO, so it seems appropriate that we're now dealing with, okay, this was how 'devs' dealt with bugs, how should TripleA as a full holistic community deal with them?

Frostion

I think option 1b (Encourage common players to report bugs at forum, and advanced users to go directly to Github)

This is because I don't see a bottle neck. I see the forum as a filter where possible bugs are posted, evaluated by more competent people and identified as being an actual real bug or not. Also, at the forum, it can be determined if the reporter himself is capable of and competent enough to post at Github, describing and discussing his bug. If not, someone else could/should go to Github with the bug ... in best case the reporter has the drive and motivation to learn to do it himself.

I think this option ensures the most bug reporting (and broadest community inclusion ... something needed to recruit and create development and mapmaking interest ) and also it should ensure some level of quality when it comes to what is posted at Github, so Github does not become flooded with questionable and debatable reports. Hence the "filter".

I must admit that I am not at all competent to say what would or could lead to the best working environment for developers at Github, that is of course something the devs should know and determin. So I wish more devs would give some input here. But I do feel that the rest of my analysis is fairly reasonable

LaFayette

1b works if it is formalized. The 'bottleneck' comment should not be misconstrued please, it's more a matter of looking at
the process flow. One process being considered (1a) has a direct intake to bug tracker, while 1b has an intake to forum and output to bug tracker. The conversion from forum post to bug tracking ticket will not be immediate nor 1:1, by definition a bottleneck as we will be waiting for humans to do something (and no matter how diligant or fast, that will introduce some delay into the process). Whether it is a big impact or not is another and important matter. The impact will be inconsistent, we need to design this process so that this will not become a problem and we will not see a process break down.

How does 1b break down? what do we need to avoid/solve?

appropriate culture of 'always do something' not instilled. For example, in: https://forums.triplea-game.org/topic/920/some-help-me-please/3, the "your internet is not avaialble" warning message was misconstrued and reported as an error itself. A response of 'ya, that is normal, just close it and continue' is an example of where we do not have that culture. No UI notification should ever be routinely ignored, and secondly the warning message is clearly not helpful in its current form. So the action item is to either remove it or fix it, but we can't just say "yeah, that was your problem", because it was not the users problem, it was our poor design and lack of time/labor for code development (which is compounded by really high overhead processes and high overhead code).
potentially times of low capacity, people get busy; If we have only 3-5 people moving tickets, and 2 or 3 of them happen to coincide by being away and/or busy at work or what not, then we'll have pretty limited capacity and forum posts would build up. Dev's would jump in for these situations, but that is then reducing time available for devs to code, and it's hard to then justify having the extra reporting step.
we simply lose track if something has been moved and do not get around to it. This report: https://forums.triplea-game.org/topic/957/pbem-pbf-combat-move-check-box/8 is potentialy an example, the reported bug has not made it into tracker. With more time, the post will fall from the top and would be unlikely to ever be fixed.

For this process, the success and failure metrics I would choose would be (so these are numbers that we should work to optimize and track to know that we are doing well):

number of forum posts that are handled without needing a ticket. Sometimes questions just need to be asked. (We would need to be sure to count this metric very conservatively, most user reports should result in some action, even if the 'system is working as intended', if a user can't figure out what that intent is, it may as well not be working).
number of forum posts that result in a bug tracking ticket or are de-duped against an already open ticket. If the ratio of this number to the total number of posts start to drop, then we are falling behind
the amount of time between when a forum post is opened and when a bug report is created
number of bug reports (forum or github issue, any kind of report) that is filed in the wrong place
number of github bug tracking tickets missing important information

Cernel

Also, in past there used to be in lobby a link to where to post bugs. I guess you removed since you can click on Help (not so obvious, in my opinion), but just saying in case.

Maybe off topic but still talking about bugs, this is what happens when you are in the lobby and want to see how to report a bug:
You click on "Help"
Click on "Help Page"
Get redirected to this page:
https://triplea-game.org/help/
making you click to go to:
https://forums.triplea-game.org/category/10/help-questions
at this point, someone that is searching for bug reporting will hopefully click on:
https://forums.triplea-game.org/topic/526/how-to-submit-a-bug-report
where you find the link to opening a new issue in the tracker (need to register to GitHub, of course).
Side note, at this topic, the:
"Open Bug List"
and
"Lobby is on Fire"
links both have GitHub telling you:
"No results matched your search."
On the other hand, if the user would, instead, decide to click on:
https://forums.triplea-game.org/topic/13/faq-and-help
and then on:
To submit bug reports: http://www.triplea-game.org/bug_report/
he would get a:
404
File not found
(or at least this is what I'm getting)

So maybe in the moment you get a bug while you are in the lobby, knowing how to report it may be not very clear. Maybe it was better the old practice to have just the link to where to bug report at the start of lobby chatlog (your decision), for the users to copy-paste.
Anyways, not so important, as I believe lobby users very rarely report bugs anywhere (sadly), so having a link there won't do much to increase bug reporting, I guess.

LaFayette

@cernel Thanks for tracing that through, hard not to chuckle at the absurdity.

The bug report link had the virtue it was simple. The above flow is broken, I don't think we solve that by creating a 'bug reports' section on forum though as it implies this is the place for bug reports and that becomes confusing. A formalized process of moving bug reports from one place to another I do not think will be sustained over the long-term (in the short it does not seem to be working, I note a number of bugs that are being 100% worked on here, and did not get proper tracking visibility).

If we really want to unify logins we can have forums to use github API for login, then that could be the one login for every TripleA system. There are some real benefits to that as we have less custom code to maintain and that would reduce our engineering costs.

So I think the bottom line is having a 'bug reports' forum section here is well-intended, but fundamentally confusing as this is not our flow/process for reporting and tracking bugs. Merging this category with 'help' section and fixing the bug reporting flow so it takes a person to the right place the first time would be good next steps.

LaFayette

Why those next steps?

Well, how should this get fixed? We need something, concretely how to instruct the computer what to do.

If we update the help menu to have a forum link for reporting bugs, then that will be where bugs go. Sadly, this is not bug tracking software and issues are going to get lost/missed (recall when map updates were driven by forums, I counted that a good percentage were just dropped and never actioned, that was largely in part that forums track active conversations, they are not designed for priority and task tracking).

So the answer seems to be to have the help links be the single "submit a new bug report" link that we once had. I'm wondering how to work in "see the forums if you are unsure of what to do and if you are not sure this is a bug", but that confuses everything and gets back to the problem that eventaully we will not move everything to bug tracking and that will cause items to be lost (and if lost, then reporting here is just not going to be as successful as it could be).

This is for sure an open conversation, I can be pursuaded, but my current thinking is:

it will not be successful to do task tracking in forums, we need a bug tracker for it.
we can route tickets through forum if we have a real and sustainable and formal process to move those tickets into bug tracking
over the long run, even short term, getting such a formalized definition and sustaining it for the life of the project (hopefully a few more decades at least) will eventually fail
splitting the instructions to route things to forums makes those instructions more difficult to follow and maintain, it adds to the confusion and the 'trust no documentation' because we have 5 copies of it all and all are slightly out of date

I think that leaves trying to make our bug tracking queue very obvious and the one place where we do that kind of reporting /tracking. Just not necessarily seeing how we can solve it otherwise without running into additional problems (and thinking historically to how tracking failed with the SVN bug tracking, and noting how things ran when we had no active forum and only github issues; things work better when there is process clarity and transparency)

LaFayette

Good news and a bit of a conversation pivot. How bug reports make it into the tracker will soon be a problem for just admin/dev.

We're introducing an http server to the mix and that let's us build a few things pretty quickly. The first feature to make use of this is a 'error report' feature for the client that is in progress.: https://github.com/triplea-game/triplea/projects/4

The original idea was to upload stack traces to the server, with a little bit more we can add a text field on to it to get a free-form input so we can submit bug reports directly from in the game. Hence, it'll be up to dev/admin how the reports are processed after they are entered, they will in a DB and we could potentially do things like automatically create bug tracker tickets when new items show up.

A logical add-on to this would be to add the capability to capture and upload a save-game while uploading the error-report. The http-server/client components are looking far enough along that we can potentially start thinking about how that process would look.