Calculations for pinch to zoom

In which I discover how to correct for things moving around when you zoom, using only elementary algebra.

Text Collector uses a pinch-pan-zoom view to let people preview how their messages will look in pdf format. Inexplicably, Android provides no pinch-pan-zoom view built-in, so a quick look online reveals implementations to fill that gap littered everywhere. Those that aren’t broken, however, can only handle ImageView content.

If you need pinch-to-zoom for something other than pictures, you need to reinvent it.

I struggled with this implementation for an embarrassing amount of time, and judging by the number of wonky zooms I’ve seen in Android games, I’m not alone in finding it tricky.

Android does give us ScaleGestureDetector to detect pinches; it reports a “scale factor” that is a ratio representing how far our fingers move apart or together. The obvious thing to do is to scale your content, using View.setScale(), something like setScale(getScale() * scaleFactor). That’s the right idea, but insufficient.

Scaling a view transforms it around its “pivot,” an arbitrary point somewhere in the view. What we really want is to scale it around the “focus” of the zoom, that is, the bit of content between our fingers. Focus and pivot don’t line up, so, as we zoom, the content we want to see rushes away offscreen.

Model

We have two different coordinate systems because we need a fixed-size touchable area to detect fingers and a changing-size area to display content. I call these the “window” and the “content,” respectively. As reported by Android, focus is in the window grid and pivot is in the content grid.

Misaligned pivot and focus cause scaling to shift the view content away from wherever it’s supposed to be after the zoom. To correct, we need to translate back by an amount t.

  • t: translation needed to correct for scaling, window units

Android gives us these measurements:

  • f: focal point of the zoom, window units
  • m: margin outside the content, window units
  • s: starting scale, window units per content unit
  • z: scale factor, that is, change in scale, unitless

Two measurements change during scaling. I will denote them with a tick mark meaning “prime:”

  • m': margin after scaling, window units
  • s': scale after scaling, window units per content unit

Scale factor is the ratio of scale before to scale after, so:

s' = zs

Actually, the scale factor and focus used here are approximations that work well, but could be refined in a more complete model.

We’ll use a couple measurements in the content grid as well:

  • P: pivot around which scaling happens, content units
  • D: content that aligns with the zoom focal point when zoom begins, content units

When scaling, measurements in the content grid do not change. Upon reflection, this should be obvious because the content can draw itself without knowing it’s been zoomed. So, even though it looks like P grows in this diagram, remember this diagram shows the window perspective. From the content perspective, P does not change.

Diagram of measurementsAndroid gives us P but we need to calculate D for ourselves. Since f and m are in different coordinates than D we cannot say that D = f - m.

This makes me wish for a language like Frink that attaches units to numbers. You actually can add measurements of different units together, but only if there’s a defined conversion. So, something, like D = f - m could do something sane.

In Java and all mainstream languages, numbers are unitless, so it’s easy to add numbers nonsensically.

For both grids, the origin is at the left side. To convert between coordinates on the window grid (subscript r) and the content grid (subscript c):

x_r = sx_c + m \newline s' = zs \newline x'_r =s'x_c + m'

So:

f = sD + m \newline \Rightarrow D = \frac{f-m}{s}

Given these things, we need to solve for t, the translation that will rescue the content we want to see from wherever it went during scaling.

It is important that even though we call a View function, setTranslation(), on the content to translate it, the number we pass that function is in window coordinates, not content coordinates.

Derivation

So far, the things we know, given by the Android api are f, m, s, z and P, from which we know how to calculate D and s'.

Next, we need m', the margin after scaling.

In software, you don’t actually have to calculate m' yourself. You can setScale() then getLocationOnScreen() to ask the view where it would place its corner, but that’s cheating.

To find m' in terms of things that we know, another variable helps to translate pivot from content to window:

  • w: position of the pivot, P, in the window grid, that is, w=Ps+m.

w = Ps + m \newline w' = Ps' + m'

Relationship of w to m and PThe t correction will move the pivot in window space, but only after scaling. By definition, the pivot does not move due to scaling alone, so w = w'. This implies:

Ps + m = Ps' + m' \newline \Rightarrow m' = Ps - Ps' + m \newline\Rightarrow m' = Ps(1 - z) + m

Next, we use m' to calculate the thing we really want, the translation t, to compensate for zoom. Define a variable translating the content position under the focus, D, to window coordinates:

  • h: position of the content at D after scaling, in the window grid, so h = Ds'+m'

Relationship of m, h, f and D to t

Because the translation t is in window coordinates, t = f - h. Recalling that s' = zs, we know:

t = f - h \newline t = f - Dzs - m'

Plugging in D:

t = f - \frac{f-m}{s}zs - m' \newline t = f - z(f-m) - m'

Since we previously derived m', we are now done:

t = f - z(f-m) - (Ps(1 - z) + m)

To make this Android-executable code, we just need to translate to Java and do the same for each axis. Source is on Bitbucket.

But wait, there’s more

An eager kid in the front row is waving his hand to tell me about affine transformations. We can simplify further, you see:

t = (f - Ps)(1 - z) + m(z - 1)

I didn’t go this far because it’s no cleaner in Java, but does look more symmetrical in Math: tantalizingly like a dot product. Unfortunately, I forgot most of my linear algebra long ago, so I have no idea why. Best go watch 3Blue1Brown.

Painless Android releases revisited

Previously, I described a Gradle script that handily generates release version codes for Android apps. The generated version codes take the form [date][number].

I finished that article with a litany of Gradle bugs. Today: fresh Google bugs!

In May, Google added automatic crash reporting to the Google Play developer console. Before auto-reporting, users had to explicitly send reports when apps crashed. So far so good, but if you’re testing on a physical device, you might notice something alarming: reports of bugs you already fixed, or crashes you only saw in development.

Apparently, Google forgot to filter out reports from debug-mode applications. Perhaps Google would claim this is a feature, but it means that you can’t tell which crashes are actually happening in the wild.

Google says crash reporting is “opt-in.” This is meant ironically, since the option to turn it off doesn’t actually exist on, for example, the Samsung S8. (There is a different option, “report diagnostic information.” As far I can tell, it’s a placebo.)

To work around this, we need to make crash reports from the debug version look somehow different from the production version. Crash reports include the version code, so remember that suffix? We can use that. Instead of using one number per release, use two: one for the release, one for the next development version:

// Version code updates when released to a date-based format. Even-numbered version codes are
// release builds, odd-numbered version codes are debug builds. MAX five releases per day.
def releaseVersionCode = null
def writeVersionCode(versionCode) {
    def releaser = project.plugins[net.researchgate.release.ReleasePlugin]
    def propsFile = releaser.findPropertiesFile()
    def props = new Properties()
    propsFile.withInputStream { props.load(it) }
    props.versionCode = versionCode.toString()
    propsFile.withOutputStream { props.store(it, null) }
}

task nextDebugVersionCode { doLast {
    // Even though this runs after the release build, project.versionCode is still the version
    // code *before* release. The Release plugin runs the release build in a separate Gradle
    // invocation, so the release package picks up version changes in gradle.properties. When
    // control returns here though, it's the original Gradle invocation, and has *not* reloaded
    // gradle.properties.
    writeVersionCode(releaseVersionCode + 1)
}}
updateVersion.dependsOn nextDebugVersionCode

task setReleaseVersionCode { doLast {
    def current = project.versionCode.toInteger()
    releaseVersionCode = new Date().format('YYMMdd0', TimeZone.getTimeZone('UTC')).toInteger()
    if (releaseVersionCode <= current) {
        // Should only happen when there is more than one release in a day
        releaseVersionCode = current + 1
    }
    writeVersionCode(releaseVersionCode)
}}
unSnapshotVersion.dependsOn setReleaseVersionCode

So, now the first release of the day gets suffix zero, the debug version that follows gets suffix one, and so on. I’m writing this on July 26, so if I cut two releases today, my version codes will be:

  • 1707260, production
  • 1707261, debug
  • 1707262, production
  • 1707263, debug

It’s subtle, but at least now we can tell which crashes actually happened to people using your app: they are even numbers.

Or are they?

It appears that Google stores the crash data on the phone and reports it only once per day. The version code it reports is the version running on the phone when it sends the report, not when the crash actually happened.

If the app updates in the interim, we can still get crash reports for bugs already fixed and they will seem to come from a version that includes the fix.

I don’t know of any workaround.

Painless Android releases

Android apps require not one, but two version numbers:

  • Version code: an integer that Android uses to check whether one version is more recent than another
  • Version name: a friendly version to display to the user, conventionally something like 1.2.3

This means that when you want to build a new release of your app, you have two things to manually update, and that is two things too many. You will make mistakes.

Luckily, it’s not too hard to automate this away in your Gradle build script.

Gradle inherited much of its design from Apache Maven. Maven defined a standard release feature that automatically handles typical pitfalls and mindless details of making a release: tagging in source control and incrementing your version number. For Gradle, there is a nice third-party implementation, the gradle-release plugin. So long as you don’t fight Maven-style version conventions, it can make cutting releases almost entirely automatic, modulo prompting you to confirm that it guessed correct version numbers.

If your project only has one version number, you just apply the release plugin and you’re done, but Android’s two-version-number system takes some customization.

I only discuss version numbers here, but the release plugin also does several other useful sanity checks.

First, move the versions out of your app/build.gradle into app/gradle.properties. They should look like so:

app/gradle.properties

version=1.0-SNAPSHOT
versionCode=1

app/build.gradle

android {
    // ...
    defaultConfig {
        versionCode project.versionCode.toInteger()
        versionName project.version
        // ...

“SNAPSHOT” is Maven’s convention for “between releases”. Version 1.0-SNAPSHOT means the code leading up to version 1.0. This convention is how the release plugin guesses what version number you are releasing: it just lops off the suffix.

When you run ./gradlew release, the release plugin updates the version thus:

  1. Edits gradle.properties, removing the “snapshot” part
    1.0-SNAPSHOT becomes 1.0
  2. Commits the change and tags this as version 1.0 in source control
  3. Builds the release
  4. Edits gradle.properties again, to next dev version
    1.0 becomes 1.1-SNAPSHOT
  5. Commits so you can immediately start working on version 1.1

Thus, out of the box, this handles the user-friendly version number, but not the “version code.”

Updating the version code

When Android installs an update to an app, it knows by version code whether the update is newer than what it currently has installed. 3 is newer than 2 and so on.

Thus, the obvious strategy for updating your version code is to add one on every release. If using the release plugin, you might do this as a manual step after it finishes a release. If you forget, you’ll accidentally build your next release with the same version code as you just used. If you have other branches, you need to remember to update them as well. Ouch.

There is a better way. Version codes need not be sequential, so instead of incrementing 1,2,3…, we can derive it from the date. A format like [2-digit year][month][day][0-9] works nicely. A release today gets version code 1704080, tomorrow, 1704090.

This format will cover you for 82 years at up to ten releases a day. If that’s not enough for you, use a four-digit year and a two-digit suffix, but watch out for integer overflow in 130 years or so.

The date-based strategy, however, means that you have to set your “version code” immediately before you release, instead of after. To do this, add a Gradle task right before updating version name.

app/build.gradle

task setVersionCode { doLast {
    // Add a task that updates version code
    def current = project.versionCode.toInteger()
    def releaseAs = new Date().format('YYMMdd0', TimeZone.getTimeZone('UTC'))
    if (releaseAs.toInteger() <= current) {
        // More than one release today
        releaseAs = current + 1
    }
    def releaser = project.plugins[net.researchgate.release.ReleasePlugin]
    def propsFile = releaser.findPropertiesFile()
    def props = new Properties()
    propsFile.withInputStream { props.load(it) }
    props.versionCode = releaseAs.toString()
    propsFile.withOutputStream { props.store(it, null) }
}}
// Execute our task before unSnapshotVersion, provided by the release plugin:
unSnapshotVersion.dependsOn setVersionCode

With this simple build script change (plus applying the release plugin), a single command updates both version numbers:

./gradlew release

The release plugin also runs the “build” task at the point of release, so this single command leaves you with both a release .apk and your working directory updated to the tip (snapshot) code ready to start work on the next release. There’s still a problem though: if you haven’t configured your build script to sign the build, you won’t be able to publish the release .apk.

Signing the build

To make Gradle sign a build, you need to add a “signingConfig”:

android {
    // ...
    signingConfigs {
        release {
            storeFile file('/home/myname/.javakeys/mykeys.jks')
            keyAlias 'myappsigningkey'
            // These two lines make gradle believe that the signingConfigs
            // section is complete. Without them, tasks like installRelease
            // will not be available! (see http://stackoverflow.com/a/19350401)
            storePassword "notYourRealPassword"
            keyPassword "notYourRealPassword"
        }
    }
    buildTypes {
        release {
            signingConfig signingConfigs.release
            // ...

This fails, so you put your real password in the “password” config place and get pwned. Your wife leaves you, and your dog dies. You didn’t do that, right?

So where should you put your password? The top-voted answer on Stack Overflow says ~/.gradle/gradle.properties, presumably protected by 600 permissions. I don’t see the point. If you’re relying on file system permissions to keep the password secure, why have the password at all? You could just protect the keystore with file system permissions.

What you need is a prompt for the password.

Thanks to bug 1251, Gradle running in daemon mode (the default) doesn’t let you use System.console().readPassword("Password:"). You can disable daemon mode, but then you run afoul of (orphaned?) bug 2357 because Android Studio generates a default gradle.properties that includes jvmargs. Once you remove that configuration, you find that prompts don’t display when you build not in daemon mode (bug 869). That’s a pain because you can’t see the version number confirmation prompts.

As a result of this epic adventure, you’ll eventually find that the only reliable way to prompt for password is via Swing. No, I’m not joking. It’s not as gruesome as it sounds, thanks to Groovy’s Swing builder, so pop over to where Tim Roes documented how to do it.

Update: there’s a new version of this build script

Build one to throw away

Most software projects inadvertently plan to build a throwaway product for delivery to customers, instead of heeding Fred Brooks:

In most projects, the first system built is barely usable…

Plan one to throw away; you will, anyhow.

Luckily, when building Text Collector, only my own management expectations burdened me, so I had an opportunity to build a real pilot and chuck it.

I wrote the pilot in Java and it included all the major features I knew I would need. Thanks to Steve Yegg popping up for the first time in a few years, I heard about Kotlin, so I switched to Kotlin for the real program.

Now that it’s in alpha, I can reasonably compare the size of the two programs:

Java Kotlin
Lines  1.9k  3.5k
Words 6k  13k
Bytes 71k  146k

By many standards, this is small, and smaller yet when you consider that I’ve counted using simply the “wc” command, so the numbers include whitespace and comments.

The pilot included all major functionality, but ignored most edge cases. It featured mms and sms collection with:

  • Date filtering
  • Message preview (small collections only)
  • Pdf creation
  • Pdf sharing
  • Inline image rendering (no scaling)
  • Zooming (without panning or clamping)

For the real program, I kept all the pilot features except contact filtering, added handling for many edge cases, and added features:

  • Organization by conversation
  • Zip creation
  • Reporting usage statistics and errors
  • Failure diagnostics and debug features
  • Option to cancel collections in progress
  • Pre-calculated layout (allows preview for large collection)

So, the Kotlin app has roughly twice the features, with roughly twice the amount of code. At first glance, it doesn’t seem like that significant a difference, but breaking it down this way lets me count up the lines associated with new features only: 1.3 thousand. Thus, the part roughly equivalent to the pilot weighs in at 2.2k lines of Kotlin to Java’s 1.9.

In other words, Kotlin, handling edge cases, is only 15% larger than the equivalent Java that ignores edge cases with wild abandon.

My commit log shows that the Java version took about one month, while the Kotlin version took three. That seems pretty dismal: two thousand lines per month in Java and closer to one thousand in Kotlin, but…

The Java pilot had no tests, the real, Kotlin version adds 6.6 thousand lines worth of test code.

Programming Android: first impressions

I suspended work on Comefrom0x10 for a little while to start my first attempt at a serious Android app. It is tentatively called “Text Collector” and essentially just makes a pdf of your text messages.

So, how is Android as a platform?

Well, first, it’s Java. This means that half my code is type declarations, the other half is keywords; we all saw that coming, move along…

The Android core api is unpleasant to use, but it could have been worse. Its main problem is severe under-documentation, apparently thanks to a bad case of “source code is the documentation” syndrome.

Though technically Java, for better or worse, it feels like an api designed by people who would rather write C. Integer constants and bitmasks are everywhere; there is even the occasional “out” parameter. On the bright side, there is a refreshing lack of abstract factory singletons. There is no xml standing in for “dependency injection” code.

There is plenty of xml for defining layouts, though. Layout xml is attribute-heavy, which means less verbose than it could have been, but also that you can’t put comments in many places where they ought to go:


<Frobnicator
  android:foo="bar" <!-- could use a comment here, but that's illegal -->
...

Thankfully, layouts and resource definitions appear to be the only places you have to use xml. In principle, you could define layouts entirely in Java, but frying pan, meet fire.

As far as I can tell, the entire Java standard library is available, but I’ve used only a few small parts of it. There are bizarro-world Android replacements of some parts. Methods that expect uris take android.net.Uri instead of java.net.URI. Bundle of Parcelable looks like it probably could just have been Map<String,Serializable>. I haven’t spent enough time with Android code to judge whether there are good reasons for this seeming duplication.

Like many apis, the core library is a mix of surprisingly easy juxtaposed with surprisingly difficult. There are some nice included layouts and widgets, like a date picker, but try hooking up a date picker to a TextView with inputType=date, and you are in for nasty surprises. Writing and displaying pdf is almost trivial, but if you want zoom and two-dimensional scrolling while you display it, expect pain.

Android List

Some time ago, I wrote a shopping list app as an exercise to learn Android programming. I do not plan to maintain this or enhance it, but the code is now available.

It is just a couple simple list views. One view adds items to the list. All items ever added remain in that view, the inventory. The other view is the actual shopping list. Tapping an item in the inventory moves it to the shopping list; tapping it in the shopping list moves it back to inventory.

A right or left swipe switches between the views, which brings up an annoying factoid: the Android api does not have built-in swipe detection. Creating your own gesture detector is easy; 30 seconds on Google finds as many slightly different implementations as you could care to see. Nevertheless, swiping is a simple, common need. A side-to-side swipe detector and a zoom detector would probably cover almost every application.

This illustrates another reason to like Python:

Fans of Python use the phrase “batteries included” to describe the standard library, which covers everything from asynchronous processing to zip files.