Is this the official place to report bugs?

LaFayette

I'm concerned that:

A. we are inviting many of the same historical problems we have already seen:

confusion on where to report, how to report, what to report
divided effort of devs to track and keep up with reports
duplicated and out of date documentation which lead to additional confusion

B. Silo'd responsibility. @redrum if you take a break then we'll have a problem with bug reports getting stale and languishing here. While it's great you're taking care of it, not having a team process get built up in a trial by fire to handle this I think in the long term does us more harm than good.

C. inviting confusion, new players asking the 'regulars' what to do, will be referred to here despite all other official documentation, links and buttons.

D. Doing things for the wrong reason. Github got a bad rap and is 5% understood. Even seeing terms like "your github repository" is evidence of this and is off-base.

I think part of what makes this hard is a lack of understanding of why dev's use bug tracking software, and why forum does not satisfy those needs (said another way, forum is not bug tracking software). To make it even harder, source control management is yet another piece of software. So the github repo has nothing to do with this, we could turn off the issue queue if we wanted. I think though we find that there are problems when we try to do project management, task tracking, bug tracking, research reporting, project proposals, general discussion all from forum software. You can do some of that in forums, but trying to do it all for something as complex as TripleA I think will lead again to many of the problems we have already seen.

I'm also suggesting a slippery slope here, and I think we are seeing it. As more and more users and infrastructure is dedicated to reporting bugs here, we'll expand on that and want more of the bug tracking and reporting to be done in the one place. Again, we've seen what happens going down that road, and we know it won't work fundamentally as project task tracking is too complex to effectively handle in forums.

Cernel

@lafayette Maybe you want to edit this:

https://forums.triplea-game.org/topic/685/read-first

prastle

@cernel Or delete it np here

redrum

@lafayette Well the way I see it is github issues are the actual bug tracker. Almost all games have a forum section that regular users can report bugs/issues/problems which are then reviewed and added to the actual bug tracker if deemed actual bugs. I personally don't really see a problem with that and that is how it generally worked with the old forum and SourceForge issues list. If you think you can get all users to just create an issue directly on github then you could try that as well but I think you still need something on the forum pointing to that.

Hepps

Besides... the entire thing is not just one guy... all of the Devs pop in here to scan the forums... and as a last chance desperate measure there are people like myself and @prastle that scan here daily and have at least a working knowledge of GIT to transfer issues there that are critical.

Cernel

@redrum said in Is this the official place to report bugs?:

If you think you can get all users to just create an issue directly on github then you could try that as well but I think you still need something on the forum pointing to that.

https://forums.triplea-game.org/topic/526/how-to-submit-a-bug-report

LaFayette

@prastle that scan here daily and have at least a working knowledge of GIT to transfer issues there that are critical

It would be more efficient to have that be in one place. You're describing what is already done for the official bug queue, it's a duplicated effort to do that here too.
Scanning forums does not work well:
A. not clear what needs work, need to read a full forum post to know status and even then the issue might be closed.
B. older issues potentially lost. Are they actually followed up on and fixed? (these are all problems that are solved by proper task tracking software)
C. there is more that could have been done on some reports. It would be hard to know without some additional technical background. Sometimes you need a dev to jump in and say "oh, that is actually this other thing, and it is easy to fix, you totally should not ignore that problem and I'll have it fixed in 5 minutes, OMG if i had only known sooner it would have been fixed."

And again, the original list of concerns I raised I don't see necessarily addressed or resolved.
Having a forum section titled "Bug Reports" I think is intrinsically the problem. Keeping that I think is difficult without repeating previous project mistakes/problems.

LaFayette

@hepps said in Is this the official place to report bugs?:

all of the Devs pop in here to scan the forums

Respectfully I'm not sure about that, and it's an important difference.

A. It's a problem to scan and track multiple places. Efficiency loss. We don't have so many devs we can afford inefficiency.
B. I don't think all devs do that scanning. There is not the time, we have a place to scan for bugs, and I did not scan here so clearly not all devs do that. I don't see devs keeping up consistently, and I suspect we'll see cases again where various reports get lost in the winds of forum paging.

Hepps

@lafayette All good. I know when I'm out of my league. I leave this to better minds than my own.

Cernel

Reference:

https://forums.triplea-game.org/topic/611/game-engine-feature-requests-ideas-categories/88

(the above topic is secreted in the moderators only section, so you need to be logged in and belonging to a group having the privilege to access Bunker)

LaFayette

Thank you everyone for chiming in, the feedback is welcome. In short, I'd like for us to consider options on this:

status quo - keep things as is
rename 'bug' category to something else
merge 'bug' category to another category like 'help & ideas'
formalize user bug reports to land in forums first, forums mod team will create bug reports in official github issues task queue as needed
devs bite the bullet and consolidate to forums, mitigate/solve any tracking problems we have in forums and just use forums for all bug reports and tracking
other direction, migrate all bug forum threads to bug tracking in github issues and focus on making it easy for bug reports to godirectly to github issues

All in all, it does seem there is a team here that is willing to take a first crack at helping users, and I can't see that as anything but as a good thing and I'm really happy that is available. I'd like the outcome of this thread and discussion to be a retrospective of how things have been working so far relative to bugs, and for us to identify what next steps we can take to be more organized and more effective. I'm curious how the group feels about our options, and if there is something else we could/should consider doing.

Frostion

I will speak my mind freely, and I hope that I am not misunderstood. I don't want to create a split, but I will point out a split that I all ready see.... not necessarily a negative one, but one I think we have to recognize and live with.

I have always felt that the TripleA community is made up of very different people, ranging from people who hang out and do work at Github, who have programming skills, interests and expertise enough to work on the engine, some of them rarely visiting the forum and engaging with that part of the community ... and ranging to the other end of the spectrum; TripleA pure gamers who have little or very limited technical knowledge and no interest either in regards to Github, game engine and even XMLs and game files. Still this group can be enthusiastic fans, playing for fun, contributing to the donation drive, giving great user based feedback, and produce ideas that fuel game development etc. (Of course some people are a mix of the two caricatures and some might not fit at all.)

It was envisioned at the old forum to have the new forum bring people more together, but as I see it, after looking back, people are different and have different interests regarding TripleA. We cannot force people together or expect them to be of equal mind. We cannot look at community members and say, that a new member is the one who has just started to play the game, when he gets more experienced he will engage himself in the technical side of the game, after that he will go and engage himself in Github, and finally when his motivated enough, he will help out in regards to development. That's is only a description of like 1% of TripleA fans. The community as a whole is and will remain a bunch of very different people with different interests .... many of them will NEVER go register at Github just to report a bug, and if they do, not all developers will be satisfied with their maybe "amateurish", un-detailed, lacking bug report anyway, and be irritated about the reporter not conforming to the standard report-form or understanding what people are saying at Github. Finally, if a developer can be absent from the forum for months and if many players (most I would guess) never think about Github or would rather stay away, I see it as the final proof of community members being very different people with very different interests.

My main point is, if TripleA development was to force community members to register at Github because it would be "convenient" for development, I must say that I see a lack of understanding or recognition about how different community members are and what kind of involvement members might want to engage in. A proposal that would demand the casual player to register at Github just to place a bug report (one that he might not even be sure is a bug, or he is "certain", but in reality it isn't) would be a step that excluded voices... a step in the wrong direction if you ask me.

If I should be more constructive, and say something else than "Bug reporting at forum seems to be great", I would propose a new in-game/launcher based bug report system. Maybe it could even auto upload crashes. But then again, the forum's bug report category brings folk into the forum and lets them see the whole thing, maybe motivating them to socialise, engage in discussions, try out map development, maybe even offer to help out with engine development.

RogerCooper

@frostion I would say report your bugs here first, as in many cases what looks like a bug is a misunderstood feature. Then if it can't be resolved, make a formal bug report on GitHub.

Cernel

@lafayette My suggestion is tracking map bugs in forum and engine bugs in the official TripleA issues tracker. This would need to be made clear to the users by updating the @prastle post I linked and changing the name of the subforum to "Map Bug Reports". We can, then, lock all threads that aren't pending map-only issues. Of course, this would require the developers to keep an eye on the forum (probably good anyways), to avoid that engine bugs remain forgotten in here (locking them and telling the users they need to go in GitHub); so, it's your call if you like this idea and want to roll with it. You'll have, then, to decide what to do with the map bug reports in your tracker. Also turning off the issue queue in all map repositories.

Cernel

Also, minor note, it's not really true that currently we have duplicate places for bug reports.

We rather have triplicate places, since there is also the bug reporting in the issue queque of each map.

For example, you can find bug reports about WWII Global in Forum, in the main GitHub tracker and in the issue queque for the map.
Like here you have a few bug reports too:
https://github.com/triplea-maps/world_war_ii_global/issues

Panther

Also we have bug reporters that either don't know about this forum or if they knew, would not create another account here, but would report their issues on their platform.
So we have axisandallies.org as another place for bug reports. Most of those can be resolved easily and quickly as many of them are issues on the user's side. However we had issues in the past that were valid , could not be resolved instantly and thus needed to be transferred to a (whatever) appropriate place (at GitHub in the past).
Personally I would not think of transferring an issue from one forum to another one, if Github appeared to be the appropriate place to note it.

redrum

@panther Yep. Generally, if there is a true engine/map bug not just user error we should look to get it into a github issue as that is the actual queue of bugs to fix.

LaFayette

Good feedback, we've had a number of historical problems to solve, all of these predating this lobby:

users have had trouble finding where to report bugs at all, let alone knowing about forums
admin attention being spread too thin with too much overhead to make much progress beyond just keeping up
issues getting lost/scattered, this is two fold; first the official SVN bug tracker did not have good visibility and its queue grew too long, secondly bugs were tracked outside of the bug tracker which lead to reports getting lost (specifically these are forum reports, if a flurry of reports come in, it's hard to make sure they all get addressed, and often some fall through the cracks particularly when a fix takes a few weeks)

The second item I've viewed as particularly important, we have few devs/admins in this project and the overhead cost for them historically has been really high (arguably the full time job of 1 to 3 people). That's honestly why we moved to github in one way, the efficiency boost for dev helped free us up to do more; maps were moved so that they could be largely self service as well.

Thinking about our goal and pipeline, we want to get bugs fixed and for each one we have to go through a discovery, reporting, (dev) reproduce and fix and then push the fix to prod and then users pick up the updated version. The two bottlenecks there are with reporting, any friction that prevents users from discovering and reporting bugs is something we want fixed, and also fixing bugs is a bottleneck.

To fix a process bottleneck, we can add labor, add automation, remove tasks. To that extent, TripleA is fortunate to have a large and knowledgeable volunteer base, we can remove burden from devs and add labor if we are able to effectively to get volunteers to help with incoming bugs.

At the same time, we don't want volunteers to be bottlenecks themselves, so we need to think of the bug intake process as checking in to the airport, the volunteers are not the people doing the check-ins, they are the people walking around making sure everyone is okay using the automated check-in machines. As part of that, there is key information needed when reporting bugs so devs know what to look for and fix. Part of the inefficiency of fixing bugs is repeated asking for information that is needed before starting, to help solve that questions/check lists and automated tools are good answers so it's much easier more natural.

So, 2~3 years ago it was really not clear where to report bugs, and we did not have this forum. There was a concerted effort to try and improve efficiency and increase the rate bug reports by making the bug reporting steps relatively clear and directing people to the bug system. Hence, my concern when I realize we have developed a parallel flow here for bugs, which in an unstructured process really backtracks and could re-invite many of the important problems that we have almost left behind us.

Sorry for not having a lot of time to be able to shorten that down, it's hard to answer in short.. Hopefully the problems and motivations make it more clear we are just looking at a different set of problems and the shared perspective I hope helps.

With a really long story short, I thnk the way this is helped is simply by merging/combining the bug report section with the help section forums. Looking at the types and content of threads between the two, they are really similar. Once they are clarified and combined, and it made a bit more clear that is the place for questions and official bugs are put in to a bug tracker, I think things would be more clear for the new players who will be joining us over the next few years (an easy bug report process is important for them, and I'm most concerned about attracting and retaining new players, growing community IMO is top importance)

Cernel

I believe the Bug Reports subsection was made because users would mostly not go reporting in GitHub, nor they would post in a specific place, nor the mods would keep traking the bug reports down and moving them to GitHub or even in a single place in the forum, but there were tons of bug reports lost spuriously in the various subsections, and diluted in between of all non-bugs posts in there.

If consolidating, I believe map bugs should better stay in "Maps & Mods", not in "Help & Questions", and preferably rather in a subsection of it.

LaFayette

@cernel said in Is this the official place to report bugs?:

If consolidating, I believe map bugs should better stay in "Maps & Mods", not in "Help & Questions", and preferably rather in a subsection of it.

Agree.

Thanks for background on why the 'bugs' category is needed - makes sense.

Is there agreement it looks to make sense to combine "Help & Questions" + "Bugs"? (please check out the thread posts between the two, they seem pretty similar topics). If that sounds right, here's a first list of potential titles for a new combined category I can suggest:

"Help, Bugs & Questions"
"Help, Bug Reports & Questions"
"Help, Problems & Questions"
"Bug Reports & Questions"
"Game Help & Bugs"