How test UI elements / get ServiceLoader working?

[RaiNova]

@lafayette WDYT?

PresenterModel looks like an interesting design principle. If it would be a general design principle in triplea
and if I would write a new dialog, then I would probably implement a PresenterModel.

What I am trying to do for triplea is much more limited:

• correcting the assignment of damage to air units with different movement points
• differentiating between units than can vs. can't withdraw in the casualty selecting dialog
• annotating the respective units in the dialog with a non-withdrawable icon
• making sure the casualty selection dialog shows therm in the right order

The diagram shows the most important classes in this context.

Redesigning CasualtySelection/UnitChooser would be a greater undertaking, which I try to avoid.
I‘m still writing automated test which include some aspects that - given the current design - can only be verified including the ui.

I’ve been an IT project manager for many years, switched to the business side a few years ago and set up a test team (since I thought it was a necessity nobody else cared about). The reluctance to automated ui testing always intrigued me - so I‘m having a go at it myself now. And I am having a lot of fun with it

I agree with the complications of automated ui testing you mentioned, taking the brittleness as the most serious one.
My developer colleagues argue that the brittleness leads to expensive adaptation of the test code.

In triplea, the greatest issue with brittleness is probably not efficiency but process: If another contributor changes
an implementation detail, I don’t want to cause her grief by a test that fails although functionality and ui are still correct.
This could be very confusing for her.

How about this: I take the scenario "my ui tests fail, because another developer has changed an implementation detail" into
account by

• writing tests that verify that my assumptions hold, so if they fail, the other developer will understand what happend
• try to address the right level of abstraction in my essential ui tests
• show you what I came up with, and we go on from there?

Cheers
Rai

PS: Don't worry: If you end up rejecting to include my ui tests, I had a great time writing them, anyway

[LaFayette]

@rainova The diagram you posted is interesting and thank you for creating that. I think it shows pretty well how UI and logic concerns are hopelessly intermingled and that has gone on to cause a plethora of other problems.

I don't think it's possible to solve that tests can't be headed when run in CI. That results in tests that need to be manually run, in which case developers may never know that they are broken without adding additional steps to verifying changes.

Generally UI tests are expensive compared to their benefit. They are often very slow and gets a team into a situation where test execution goes from seconds or minutes to many dozen minutes (and your computer is flashing UIs while this is happening, so you can't even do any other work while you are waiting for tests to run).

Yet another problem is that UI tests are very much end-to-end and that can lead to the integration testing trap. How do you account for all the various scenarios that you can come across, how do you test for error scenarios?

I think the place for UI tests would be for "given a UI model -> does the UI render correctly?" That would then be totally divorced from all logic testing. If we do test the logic of the UI as part of UI tests, then changes to the logic would then break the UI tests. That is not a good level of isolation. Further though, if the logic is already tested by a UI tests, then we have disincentive to test the logic on its own. This means more and more logic becomes tested through UI tests (and UI tests being end-to-end tests are impossible to write enough of them to sufficiently cover the majority of all logic branches)

We had an initiative in TripleA where we wanted to rewrite the UI to use JavaFX. That sputtered out largely because the UI is just too coupled with too much logic and doing that UI update would involved a rewrite of most everything. So we really would prefer to have a presentation model going forward, and it is a lot of work to refactor our UIs to have that.

[LaFayette]

FWIW, you may want to check out Fest @RaiNova: https://tuhrig.de/automated-ui-testing-with-swing-fest/

That does not solve the problem that you still needed a headed system to display the UI components, but fest is a pretty decent testing framework for UI components, it's quite functional.

[RaiNova]

@lafayette JBrains says isolated tests are hard to write on badly designed code. Forbidding automated integration tests encourages good design. We could start a long discussion about that, and it would lead us far away from the question, whether the automated ui tests I write to verify my code does what it should do should be integrated in the junit test run.

Nearer to that question are:
a) should I improve the design while applying my changes?
b) should someone else improve the design, before I apply my changes?
c) how great is the need to improve the design in triplea?
d) how can we get an overview of the most important design issues?
e) how could the respective issue be fixed?
f) what can / should we do to prevent the issue from manifesting again in an open source project?
g) how much effort is in e) and f)
h) what should we do next regarding triplea design?

a) I see more fun things I can do with triplea. If I would go into improving triplea design, I would not start middle out (introducing a particular design pattern in the specific piece of code I happened to work on for other reasons) but bottom up (fixing one particular issue I feel,like, e.g. replace the callback from the casualty selection code to disable the button that started it by making the casualty selection dialog behave like a normal modal dialog, or consolidating file access or one of the other potentials I happened to come across) or top down (first answering the questions d-h in order).

b) if someone volunteers, please let me know so I wait until she's done and do my code changes on top

c) I‘m cool with the current quality. I doesn't slow me down too much. But if someone sees the need to improve the quality, I might happen to volunteer

d) how about starting a thread in the developer forum?

e-h) when we have an overview, I am happy to work on these questions

Thanks for the link to the test framework.I'll have a look

Cheers
Rai

[LaFayette]

@rainova I had a different take-away from the JBrains talk. Notably that if you test end-to-end you cannot get complete coverage of all logic paths because there the logic paths combine in a combinatoral way. If you test each module individually, the number of logic path permutations becomes additive rather than combinatoral. He suggest though that end-to-end tests are still necessary, but are necessary to verify integration, not logic.

(A) I agree there are more fun things to do. On the other hand we have a track record of brittleness and needing to do a lot of manual testing.
(B) There is nobody else.
(C) In some places, pretty great. For example, you cannot change any of the the class variable name in any class that extends GameDataComponent. Another example, the battle calculator copies the entire game data on load and this copy represents the significant majority of time that the AI uses to do move computations.
(D) Good question
(E) Good question
(F) Another good question, we have more static analysis checks in place now that help, and we do code reviews.

(H) Have you read the TripleA 3.0 thread? https://forums.triplea-game.org/topic/2794/triplea-3-0-design-proposal-discussion/23

[RaiNova]

@LaFayette … the problem that you still needed a headed system to display the UI components

Does a linux server run the backend test? Can you equip it with a virtual screen as described in https://eclipsesource.com/blogs/2013/12/06/run-ui-tests-in-the-background-on-linux/ ?

[LaFayette]

@rainova Github actions (previously we used Travis) both run a within a container. The github actions are defined by each of the YML files in: https://github.com/triplea-game/triplea/tree/master/.github/workflows

I'm pretty sure, but not certain, that one can not connect a virtual screen.

[RaiNova]

@lafayette
Having no experience with build environments, I have to ask: Does this look promising
https://sick.codes/xfce-inside-docker-virtual-display-screen-inside-your-headless-container/
?

[LaFayette]

@rainova Maybe, that link is a bit skimpy on detail. FWIW you could verify some of that yourself by running that docker container. You would need to do more to have tests run from within the container or somehow to forward to the virtual env.

Though, if you did manage to get a headless container running, there are some drawbacks:

• setting up yet another docker and one more built step is just one more thing
• what happens if a test fails? If the UI is headless, how to tell what was incorrect? What if the results from the virtual desktop do not agree with running without docker?

[LaFayette]

@RaiNova I had an ulterior motive to want a well done view-model design, specifically as a good pattern for UI classes going forward and to retrofit the existing.

Check out this PR which adds one. Of note is how the UI-controller logic is tested: https://github.com/triplea-game/triplea/pull/9665/files#diff-73b2a24d2079dcf7d0b19c8777110b46bbe56a5e97fc9e4e41c875f7b98671cd

[RaiNova]

@lafayette Thanks for pointing me to your PR and giving me an impression how your idea regarding "Presentation Model" can be realized in TripleA! When you pointed me to "TripleA 3.0 - Design Proposal & Discussion", I read your idea to modernize TripleA by moving from Swing to JavaFX. That was a main driver to me writing "How does TripleA 3.0 have to be to attract such a community?" and

3. Technology

• How much is TripleA about the end user product and how much about playing with technology?
• Which technologies do we want to play with?
• Shall we go web?

I am a fan of Google trends, so I looked up some UI technologies:

I recommend a discussion about the questions I posted in "TripleA 3.0 - Design Proposal & Discussion" first, then a choice of candidate technologies and a selection based on how they support a developer community and how they support design patterns and best practices that could further our goals.

Lacking such a discussion, I will have a deeper look into React. I will have your "Presentation Model" in mind as well as UI testing. I am happy to let you know my impressions and hear your opinions about them. First I have to do some programming for work (Kotlin and JavaFX including JavaFX Robot btw). Then I'll finish my PR (I am already testing to make sure my changes don't affect other use cases of UnitChooser). Then I'll look into React. (Or not - depending on what else comes up in my programming interests or my life in general.)

[RaiNova]

@lafayette With all the little technical issues that held me up, I can‘t say I took a dive into react, but only a glance. These are my impressions, and I would be happy to learn your pov about it:
React is technologically so far away from TripleA, that I can‘t see a convincing migration path towards it. However, it taught me some design promising principles:

• There are not one but two languages for presentation models on two levels:
  ++ one level oriented towards the rendering engine (e.g. DOM for HTML or react-three-fiber for three.js); for simplicity I‘ll call it the DOM
  ++ a domain specific level
• the presentation model languages are embedded in the host programming language (typically jsx-Tags which are transpiled into JavaScript; I also found a Koltin-solution using annotations, see React Functional Components with Kotlin/JS). This embedding significantly reduces the amount of code necessary only because a programming language in itself doesn‘t support a particular concept well (do you call that scaffolds?)
• both presentation models are trees
• the render method of the root of the domain specific model updates the DOM
• the render method of each node of the domain specific model calls the render methods of its children and inserts any number of elements into the DOM
• the engine does not directly render the DOM, but compares it with the DOM of the previous render and renders only the differences

This two level presentation model should lend itself nicely to automated testing.

Do you think, these concepts could be introduced into the next generation of TripleA UI? Ontop of which UI technology? Can you imagine something more topical and more attractive to potential new programmers than JavaFx?

[LaFayette]

@rainova said in How test UI elements / get ServiceLoader working?:

Do you think, these concepts could be introduced into the next generation of TripleA UI? Ontop of which UI technology? Can you imagine something more topical and more attractive to potential new programmers than JavaFx?

All of those require huge efforts. 3.0 is in some ways a necessary stepping stone before we can do anything else. Our effort to migrate to JavaFX failed because the UI code is too coupled and too tangled to logic and is inseparable.

There is a lot of UI code. Re-writing it is a larger effort than the initial write by an order of magnitude.

As far as adopting concepts from React, IMO we should be looking to 'reactive UI' principles to help. FWIW the new UI event queue is a step squarely in that direction.

[RaiNova]

@lafayette, @ssoloff, @trevan, all other developers: What are your thoughts about attracting developers? What made developers join in the past? What made them leave? Would you see making Triple A attractive for developers as an objective? Would you consider this achievable? (In that case I might start a discussion thread.)