First, use wineboot instead of winecfg to set up the WINEPREFIX. It's
the right tool for it.
Second, when initialising WINEPREFIX, use output pipes instead of
CombinedOutput. The latter will wait for *all* output to be copied to a
buffer, including the output from grandchildren processes. Since wine
starts wineserver automatically, and wineserver lingers for three
seconds by default, this is bad. We would waste three seconds waiting
for wineserver doing nothing, and then the next wine call (to start the
app) would need to start wineserver all over.
Instead, with pipes we can get cmd.Run to only wait for the parent
process to finish. wineserver stays running but we don't care. And, when
we start the gio app, we very likely reuse the same wineserver process.
Third, disable wine-gecko and wine-mono. This ensures we don't get stuck
if they're not installed, and speeds up wineboot by avoiding work we
don't need.
The time to set up WINEPREFIX goes down form ~6s to ~1s, and the overall
subtest run-time goes down from ~10s to ~3s.
Finally, copiously document all of the precious data I've gathered above
after hours of debugging.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
Fix a long-standing TODO: instead of each sub-test handling its own
output separately, just make each expose its output via an io.Reader.
Then, the shared driverBase code can tell if any of the lines contain
the magic "gio frame ready" string.
Reduces the amount of code a bit, but most importantly, it keeps the "is
a frame ready?" logic in a single place.
In the future, this also enables us to do more with all the e2e test app
output consistently. For example, we might want to add a -debug flag to
always log output lines as they happen.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
A simple 'go test -modfile=<path>' won't work properly for the
end-to-end tests, since they run 'go build' under the hood as well.
To properly propagate the flag, we need to use GOFLAGS. Since -modfile
is always relative to the current directory, we can't use it to test
many packages at once, nor can we use it via 'go test ./gogio'.
While at it, document this distinction in go.local.mod to prevent others
getting confused like I did.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
This way, if the user has a custom winecfg, it can't possibly affect the
tests. I was encountering this as DXVK does not work on virtual Xorg
servers (which we use), and Gio thus failed to render on such a
combination.
>From the numbers below, it can be seen that setting up a new WINEPREFIX
takes roughly five seconds:
$ rm -rf ~/.cache/gio-e2e-wine
$ go test -run EndToEnd/Windows
PASS
ok gioui.org/cmd/gogio 16.369s
$ go test -run EndToEnd/Windows
PASS
ok gioui.org/cmd/gogio 11.810s
A repeated run still has a slow "wine winecfg /?", for some reason. Add
a TODO since I can see it taking a third of the time on my terminal. I
haven't been able to properly investigate why, unfortunately. As far as
I can tell, winecfg is just faster when run with a terminal instead of
an output buffer. They might use isatty on stdout/stderr.
The overall time to run the wine sub-test is increased from ~5s to ~11s,
but it's worth it to make it run everywhere. It looks like there is
plenty of room as per the TODO above, as winecfg seems to mostly do
nothing. We're also not too worried, as all e2e subtests run in
parallel.
Fixes#106.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
This way, a Gio app's logs can be filtered uniquely, which wasn't
possible before since the tag "gio" would be the same between gio apps.
Since the app ID is supplied at build time, inject it via a variable
with the linker's help. The variable is only used on Android for now,
but that's OK. It might be useful for other platforms or other internal
packages in the future.
Fixes#84.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
It's essentially a copy of go.mod, but with the gioui.org module replaed
with the parent directory. Useful when wanting to try a change in the
root module as part of a gogio invocation or end-to-end test, such as
either of the following:
$ cd cmd; go test -modfile=go.local.mod ./...
$ cd cmd/gogio; go build -modfile=../go.local.mod
Since go.sum can essentially be shared, and since it seems to work with
the go tool, use a symlink. The way the -modfile flag works, if given
foo.mod, it will use the sum file at foo.sum. The only caveat is that
'go mod tidy -modfile=go.mod.local' will try to remove gioui.org lines,
since that module is replaced. So tidy shouldn't be used with -modfile.
In the future, the only upkeep needed for go.local.mod is to update the
external dependencies to mirror changes in go.mod. This is not automated
nor checked by CI, but it could be in the future if it becomes
repetitive or error-prone.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
While launching an app on Android, the hard-coded theme is used for
the color. That color is by default black, and results in a jarring
transition to the actual app background.
The default Gio color is white, so use that for the theme background
as well.
This change will break for "dark mode" programs and similar. A future
improvement would be to reflect the actual app background in the theme.
Signed-off-by: Elias Naur <mail@eliasnaur.com>
Not strictly necessary, but makes embedding a Gio Android Activity
easier; adding
<activity android:name="org.gioui.GioActivity"
android:theme="@style/Theme.GioApp"
android:configChanges="orientation|keyboardHidden"
android:windowSoftInputMode="adjustResize">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
</activity>
should be enough.
Signed-off-by: Elias Naur <mail@eliasnaur.com>
Installing it on Debian was enough, with the only wrinkle that
propagating -race won't work when we're cross-compiling, since
cross-compilation disables CGo by default.
For now, just skip the test in that edge case. If we want to use the
race detector on Windows in the future, we need to get a Windows CI
builder somehow.
Tested on my fork; see https://builds.sr.ht/~mvdan/job/164899.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
Since Wine is heavily tied to X11, we build its end-to-end test driver
on top of X11's. We use the same mechanism to start an X server, take
screenshots, and issue clicks.
Its only quirk is that it was difficult to get the screenshots to line
up with Gio's window. The comments cover what we ended up with. The
display dimensions are now part of driverBase, so that methods other
than Start can also use them - this is necessary for the wine driver to
crop screenshots.
We also use a sleep for now; a comment explains why, and a TODO is left
for future Dan to deal with. What we have now works, and I've spent
enough hours on this patch as it is.
Adding Wine to CI, and ensuring that the test passes there, is left for
a follow-up patch.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
We were using 'go run . <args>' before, which works fine, but does mean
re-linking a new binary and throwing it away at each invocation. Given
that the end-to-end tests don't do all that much work besides building
the tiny red.go app, this amount of extra work was noticeable.
We can obtain statistics for the JS sub-test, which used 'go run', via
the perflock and benchcmd tools:
$ go test -c
$ perflock -governorp% benchcmd EndToEnd/JS ./gogio.test -test.run=EndToEnd/JS
After capturing those numbers before and after the change, we can then
compare them with benchstat. The CPU cost of the subtest is halved:
name old time/op new time/op delta
EndToEnd/JS 1.42s ± 2% 1.07s ± 3% -25.04% (p=0.008 n=5+5)
name old user-time/op new user-time/op delta
EndToEnd/JS 1.46s ± 3% 0.75s ± 5% -48.34% (p=0.008 n=5+5)
name old sys-time/op new sys-time/op delta
EndToEnd/JS 366ms ±13% 224ms ± 7% -38.79% (p=0.008 n=5+5)
An alternative here would have been to refactor main.go to allow being
called directly. However, that would have required a non-trivial
refactor, since flag parsing is done via globals. Given that the
TestMain method is asy and keeps the main function simple, we've
decided to avoid a refactor.
While at it, remove the sleep in the Android driver to wait for the app
to come up on screen. Since we retry screenshots now, we no longer need
a static sleep. On average, we still need one retry for the initial
screenshot, but that's just 100ms versus the old 500ms. The maximum wait
time is also 2s here, which should scale better for slower devices.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
First, move from debian unstable to testing, since sway was promoted to
testing as of earlier this week.
Second, use the --sync option when using xdotool to move an X11 mouse.
This makes the command block until the mouse has finished moving to the
specified location, removing a potential race with the following
'xdotool click' command.
Third, deduplicate some logic into driverBase: tempDir to create a
temporary directory within a test, and needPrograms to skip a test if
the required programs aren't available.
Lastly, split the code that starts the X11 server into a method, so that
the future Wine e2e driver can reuse it. Since Wine is tightly coupled
with X11, we can reuse a good part of the code, including the X11 server
and the xdotool mechanisms.
We also add a TODO to perhaps improve the handling of the app's output
under each of the e2e test cases.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
Right now, this was badly needed for the wayland subtest, as it seems
like waiting for a frame to be ready wasn't enough for the screenshot to
show what we want. In practice, even if the machine was idle, it could
sometimes take a few extra milliseconds for the app to first appear on
the display.
This was worse when the machine is under stress, which is often the case
with CI. For example, the command below showed a ~20% failure rate on my
laptop with four cores:
go test -c -o test && stress ./test -test.run EndToEnd/Wayland
Add a generic withRetries helper function, which allows us to keep
trying some action up to a timeout, with sleeps in between that start at
100ms and keep doubling until 2s. The function also logs before each
sleep, in case the user is confused why their test is stuck for
potentially may seconds at once.
Refactor the wantColors function into a separate function that returns
an error, as we can no longer directly report errors via *testing.T. It
still reports all the mismatches at once, which is useful. It can now be
used on to pof withRetries with a thin wrapper.
While at it, make the X11 subtest use withRetries to wait for the X
server to be ready. It was using a simpler method with a fixed number of
static sleeps. It's now more consistent, and a bit better overall.
With the changes above, the 'stress' command from earlier can get past
100 runs on my laptop with no failures at all.
Finally, fix a rogue log.Fatal call I had somehow missed.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
This type contains all the common bits, such as *testing.T, as well as
the channel and method used to wait for blocking until a frame is ready.
It also allows us to initialise this base separately from Start, which
keeps the exported method simpler to understand.
The base type is embedded into the specific driver types, so that the
code remains simple. While at it, start embedding *testing.T too, so
that we can write d.Fatalf instead of d.t.Fatalf. The drivers will only
have a small number of exported methods as per the interface, so it's
easy to keep those from colliding with the method set on T.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
It passes the whole e2e test flow on my real device, a OnePlus 5 running
LineageOS 16.0 (Android 9).
I was also successful at running it against an x86-64 Android 8.0
emulator, but I'm not including any of that just yet. A patch later this
week will include a piece of code to set up and start an emulator, which
CI can then use to run the test.
Also stop requiring the screen dimensions to be enforced when running in
non-headless mode. An Android emulator can run at an arbitrary
resolution, and even in headless mode, but a real Android device will
have its own predefined resolution. Forcing the test user to set the
-headless=false flag to not get annoying "unexpected dimensions" errors
would be annoying.
That check doesn't really mean much, as our test app doesn't care about
the screen resolution. And we were only doing the check sometimes. Drop
it entirely, making the resolution parameters merely a hint so that we
can keep the drivers a bit more consistent.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
I've seen a couple of "timed out waiting for a frame to be ready" errors
on CI in the past week. I think two seconds is a bit low, if the machine
in question is not very powerful or under stress - which is common for
CI environments.
Raise the timeout to 5s.
While at it, add some log lines to each e2e test, and mark waitForFrame
as a helper func, so that its errors show up at the caller's location.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
Newer Windows NDKs add the "x86_64" platform suffix like other
OS'es. Remove the special case.
Newer NDKs are also installed in "ndk/x.y.z" versioned directories
instead of in "ndk-bundle". Support that.
Signed-off-by: Elias Naur <mail@eliasnaur.com>
For example, if the test app fails to start on wayland, we'd block
~forever (ten minutes) waiting for it to render its first frame.
We don't have a good solution right now. But at least we can use a
relatively short timeout, to help out the human who rightfully expects
a result within ten seconds.
While at it, remove a sway "get_seats" command, which was a leftover
from my debugging of what input devices are available when running
headless.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
We already wait for the app to render itself via the "frame ready"
stdout prints. Waiting for the canvas to be ready isn't really necessary
anymore. It helped us while we had to rely on sleeps, but that's no
longer the case.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
The js/wasm bug meant that printing to stdout in the e2e app would crash
the runtime, so we couldn't synchronize with the app like in the other
drivers.
Now that the bug is fixed, we can do the same as the other drivers just
fine.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
Elias Naur