Fix a long-standing TODO: instead of each sub-test handling its own
output separately, just make each expose its output via an io.Reader.
Then, the shared driverBase code can tell if any of the lines contain
the magic "gio frame ready" string.
Reduces the amount of code a bit, but most importantly, it keeps the "is
a frame ready?" logic in a single place.
In the future, this also enables us to do more with all the e2e test app
output consistently. For example, we might want to add a -debug flag to
always log output lines as they happen.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
This way, a Gio app's logs can be filtered uniquely, which wasn't
possible before since the tag "gio" would be the same between gio apps.
Since the app ID is supplied at build time, inject it via a variable
with the linker's help. The variable is only used on Android for now,
but that's OK. It might be useful for other platforms or other internal
packages in the future.
Fixes#84.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
Since Wine is heavily tied to X11, we build its end-to-end test driver
on top of X11's. We use the same mechanism to start an X server, take
screenshots, and issue clicks.
Its only quirk is that it was difficult to get the screenshots to line
up with Gio's window. The comments cover what we ended up with. The
display dimensions are now part of driverBase, so that methods other
than Start can also use them - this is necessary for the wine driver to
crop screenshots.
We also use a sleep for now; a comment explains why, and a TODO is left
for future Dan to deal with. What we have now works, and I've spent
enough hours on this patch as it is.
Adding Wine to CI, and ensuring that the test passes there, is left for
a follow-up patch.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
We were using 'go run . <args>' before, which works fine, but does mean
re-linking a new binary and throwing it away at each invocation. Given
that the end-to-end tests don't do all that much work besides building
the tiny red.go app, this amount of extra work was noticeable.
We can obtain statistics for the JS sub-test, which used 'go run', via
the perflock and benchcmd tools:
$ go test -c
$ perflock -governorp% benchcmd EndToEnd/JS ./gogio.test -test.run=EndToEnd/JS
After capturing those numbers before and after the change, we can then
compare them with benchstat. The CPU cost of the subtest is halved:
name old time/op new time/op delta
EndToEnd/JS 1.42s ± 2% 1.07s ± 3% -25.04% (p=0.008 n=5+5)
name old user-time/op new user-time/op delta
EndToEnd/JS 1.46s ± 3% 0.75s ± 5% -48.34% (p=0.008 n=5+5)
name old sys-time/op new sys-time/op delta
EndToEnd/JS 366ms ±13% 224ms ± 7% -38.79% (p=0.008 n=5+5)
An alternative here would have been to refactor main.go to allow being
called directly. However, that would have required a non-trivial
refactor, since flag parsing is done via globals. Given that the
TestMain method is asy and keeps the main function simple, we've
decided to avoid a refactor.
While at it, remove the sleep in the Android driver to wait for the app
to come up on screen. Since we retry screenshots now, we no longer need
a static sleep. On average, we still need one retry for the initial
screenshot, but that's just 100ms versus the old 500ms. The maximum wait
time is also 2s here, which should scale better for slower devices.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
First, move from debian unstable to testing, since sway was promoted to
testing as of earlier this week.
Second, use the --sync option when using xdotool to move an X11 mouse.
This makes the command block until the mouse has finished moving to the
specified location, removing a potential race with the following
'xdotool click' command.
Third, deduplicate some logic into driverBase: tempDir to create a
temporary directory within a test, and needPrograms to skip a test if
the required programs aren't available.
Lastly, split the code that starts the X11 server into a method, so that
the future Wine e2e driver can reuse it. Since Wine is tightly coupled
with X11, we can reuse a good part of the code, including the X11 server
and the xdotool mechanisms.
We also add a TODO to perhaps improve the handling of the app's output
under each of the e2e test cases.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
This type contains all the common bits, such as *testing.T, as well as
the channel and method used to wait for blocking until a frame is ready.
It also allows us to initialise this base separately from Start, which
keeps the exported method simpler to understand.
The base type is embedded into the specific driver types, so that the
code remains simple. While at it, start embedding *testing.T too, so
that we can write d.Fatalf instead of d.t.Fatalf. The drivers will only
have a small number of exported methods as per the interface, so it's
easy to keep those from colliding with the method set on T.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
It passes the whole e2e test flow on my real device, a OnePlus 5 running
LineageOS 16.0 (Android 9).
I was also successful at running it against an x86-64 Android 8.0
emulator, but I'm not including any of that just yet. A patch later this
week will include a piece of code to set up and start an emulator, which
CI can then use to run the test.
Also stop requiring the screen dimensions to be enforced when running in
non-headless mode. An Android emulator can run at an arbitrary
resolution, and even in headless mode, but a real Android device will
have its own predefined resolution. Forcing the test user to set the
-headless=false flag to not get annoying "unexpected dimensions" errors
would be annoying.
That check doesn't really mean much, as our test app doesn't care about
the screen resolution. And we were only doing the check sometimes. Drop
it entirely, making the resolution parameters merely a hint so that we
can keep the drivers a bit more consistent.
Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
Daniel Martí