Secure email monitor

I segment my online accounts into two groups: valuable accounts and everything else. “Valuable” can vary a little by personal priorities, but for most of us, our most valuable accounts will be those with direct access to cash: banking and investments.

By transitivity, any account that can allow access to those accounts is also in the valuable and high-risk group. These include financial aggregators and any email address used for account recovery.

I would like to keep the only computer with access to the valuable accounts locked away in a dungeon guarded by trolls, but highly restricted access also makes it difficult to monitor activity. I want to be able to see notifications on all my devices while not actually allowing them access.

Enter Google Apps script.

Set up the script

You’ll need an everyday – normal – email address, one you access from anywhere. Then, you’ll need another that you only access from secured devices, the restricted – high risk – account.

Make a restricted Gmail account for yourself.

  • Don’t use an existing email address in for recovery
  • Do use a password manager and make sure you have a backup
  • Do set up two-factor auth

Go to https://script.google.com/

Untitled Google Apps script

Paste:

function digestAndArchive() {
  // CHANGE THIS TO YOUR NORMAL EMAIL ADDRESS:
  var monitor = "youreverydayemail@example.com"

  // Docs say that if you have many threads, for some unspecified value of "many", you
  // should use the paginated version of getInboxThreads, as the simple version will fail.
  //
  // It turns out that means "fail silently", returning at most some arbitrary number of
  // threads, and there is no obvious way to know there are more. I suspect the "correct"
  // way is to keep calling the paginated version with an increasing start index until
  // it returns nothing, but that seems ridiculous. For practical purposes, this function
  // returns more threads than you are likely to receive in a day.
  //
  // So, upon first installing this script on a long-ignored inbox, it might need to run
  // several times before it clears out the inbox, but that shouldn't hurt anyone.
  var threads = GmailApp.getInboxThreads()
  var bySender = {}
  for (var i = 0; i < threads.length; i++) {
    // I'm assuming this is a receive-only email address, so all messages in a thread
    // presumably have the same sender (or similar). Organizing by sender isn't
    // strictly necessary, but I think the final digest is more understandable.
    //
    // The docs don't say whether the first message is the most recent or not, but that
    // generally should not matter.
    var message = threads[i].getMessages()[0]
    var sender = message.getFrom()
    bySender[sender] = bySender[sender] || []
    bySender[sender].push(message.getSubject())
  }
  var body = ''
  var indent = '\n  - '
  for (var sender in bySender) {
    body += sender + indent + bySender[sender].join(indent) + '\n'
  }
  // Experimentally, it seems that GmailApp.sendEmail encodes the body as text/plain
  // so it should be safe to drop any old string in it. Would be nice to find
  // documentation to that effect. It munges astral plane characters, but for my
  // purposes here, I don't care.
  GmailApp.sendEmail(monitor, "Daily digest for " + new Date(), body)
  for (var i = 0; i < threads.length; i++) {
    // GmailApp.moveThreadsToArchive() can move multiple threads at once, but throws an
    // error and moves nothing for more than 100 threads. That's a pretty low limit when
    // you first run this on an inbox you haven't been regularly cleaning, so move one
    // by one.
    GmailApp.moveThreadToArchive(threads[i])
  }
  GmailApp.sendEmail(
    // Error detection. When a trigger-based app script fails, Google sends you an email
    // from apps-scripts-notifications@google.com, so it's a good idea to set up a
    // forwarding rule that emails from apps-scripts-notifications@google.com should go
    // to the email address you monitor.
    //
    // Sometimes, however, Google breaks app scripts silently. In 2020, they restricted
    // permissions and this script needed to be re-approved. Instead of failing,
    // ``GmailApp.getInboxThreads()`` returned an empty result set. The daily digests
    // continued to arrive. Each digest was empty, but it's easy not to notice that
    // digests have been empty for an unusually long time.
    //
    // So, this adds a canary email. If you receive a digest and it doesn't list this
    // status email, something has failed.
    Session.getActiveUser().getEmail(), "Daily digest OK", "")
}

Remember to change the email address in the script above to your normal, every day email. Save, as, for example “daily-digest”

To run by hand, go to the Run menu or just click the play button in the toolbar. At your normal address, you should see an email like this:

Subject: Daily digest for Fri Dec 02 2016 08:11:39 GMT-0500 (EST)
From: [your restricted address @gmail.com]

Andy from Google <andy-noreply@google.com>
– Josiah, welcome to your new Google Account

Run on a timer

To schedule, click Resources (menu) -> Current Project Triggers -> Click the link to add a trigger.

Set up to run daily on a timer. The time option is in your local time, as detected by Google.
Google apps time-driven trigger example

When you save the trigger, it will prompt you to authorize the trigger to run. Click “Review permissions”, which opens a new popup window, then allow.

Run the script by hand again after setting up the trigger. It should prompt for another required permission.

Daily summaries of your restricted account should now start to appear in your normal email.


If you’re paranoid – and if you’ve gotten this far, you probably are – delete the digest emails from time to time. Deleting the digests removes a possible attack vector on your high-value account because Google’s account recovery process asks the date you created an account as a form of identity verification.

Does anyone know how to configure xkb?

I usually prefer to leave my keyboard layout and shortcuts in default configuration. Partly, this makes switching between machines easier and partly it helps me learn what the defaults are, so I can avoid breaking them in programs I write.

Some customizations, however, are just too valuable to forgo: super+arrow to move windows between monitors and shift+space to type an underscore. Kde has global shortcut configuration for the first, but what about the second?

I’ve used xmodmap before, but had problems where it wouldn’t stick throughout a session, apparently forgetting my configuration from time to time. So I end up using dumb tricks like a shell script with an infinite loop. Nowadays, though, the Internet says that xkb is the new and shinier replacement.

So, xkb it is. The best documentation I can find is An Unreliable Guide to Xkb Configuration. I learn that configuration lives in /usr/share/X11/xkb. There’s also something about /etc/X11/xorg.conf.d. Bad start… where does user-configuration go? I have no idea. I think the concept is probably that I should define a custom layout for me specifically, maybe naming it something like kingdom_of_joe, then pick that as my keyboard layout somewhere else in my window manager or login script. Screw it. I’ll just edit the files in /usr/share and if some barbarian who maps shift+space to backspace shares my computer, we’ll have to go to war.

Now, to look at the configuration. There are 275 config files. It’s slightly less than obvious where I should start.

Lampson attributes the aphorism that started our exploration (all problems in computer science can be solved by another level of indirection) to David Wheeler, the inventor of the subroutine. Significantly, Wheeler completed his quote with another phrase: “But that usually will create another problem.”

From Beautiful Code

Back to xkb. What do these six directories represent?

First layer of indirection is translating a scancode (some bytes the keyboard firmware generates) to a mysterious all-caps alphanumeric identifier that looks like FROB or AE01. This symbol is supposedly a mnemonic for the key’s physical position, except when it isn’t. The mapping happens via files in the “keycodes” directory and I think I can ignore it.

I can also ignore the “geometry” directory; it apparently contains specs for how to draw keyboards.

Thus, I eliminate 50 config files from consideration. Only 225 to go.

The “rules” directory seems like a promising place to look. This is hopeless. The files look like an almost-but-not-quite scripting language that refer to other parts of the configuration. Maybe the docs will enlighten me.

The main advantage of rules over formerly used keymaps is a possibility to simply parameterize (once) fixed patterns of configurations… A sample rules file looks like this:

! model = keycodes
 macintosh_old = macintosh
 ...
 * = xorg

! model = symbols
 hp = +inet(%m)
 microsoftpro = +inet(%m)
 geniuscomfy = +inet(%m)

! model layout[1] = symbols
 macintosh us = macintosh/us%(v[1])
 * * = pc/pc(%m)+pc/%l[1]%(v[1])

! model layout[2] = symbols
 macintosh us = +macintosh/us[2]%(v[2]):2
 * * = +pc/%l[2]%(v[2]):2

! option = types
 caps:internal = +caps(internal)
 caps:internal_nocancel = +caps(internal_nocancel)

I think the writer has a different idea of “simple” than I. Having given up on rules files, I move on to “compat”, “symbols” and “types”.

The docs make it sound like these configurations all do just about the same thing:

  • Types “…describe how the produced key is changed by active modifiers…”
  • Compat “…defines internal behaviour of modifiers…”
  • Symbols “…defines what values (=symbols) are assigned to what keycodes [depending] on a key type and on modifiers state…”

I cross my fingers and hope I won’t need compat, so I look at types. The files are full of incantations like this:

type "TWO_LEVEL" {
    modifiers = Shift;
    map[Shift] = Level2;
    level_name[Level1] = "Base";
    level_name[Level2] = "Shift";
};

It appears that xkb abstracts the concept of a modifier key to something called a “level.” Level one means no modifiers, level two is with shift pressed, level three is alt or ctrl or super or something, and so on. I guess, maybe, if I wanted space to behave as shift, I might do that in the types (or compat?) folder, but since those files don’t appear to mention specific keys like space, they probably are not what I want today.

On to symbols…

There are a mere 183 config files in symbols. They have names that look mostly like country codes, but some are a little odd. I’ve never heard of a country called “capslock”, for example. How do I know which symbols file applies to me? I have no clue; guessing it is.

I discovered in keycodes that the four-letter word for space in xkb-language is SPCE, so I break out grep to find where it appears in symbols… it is all over, but I sense a pattern. Also, I notice an oddly-named country called “pc”.

$ grep SPCE symbols/pc
key &lt;SPCE&gt; { [ space ] };

In other countries other than the republic of pc, it looks a bit different:

$ grep SPCE symbols/fr 
    key  { [ space, nobreakspace, underscore, U202F ] };
    // ␣ (espace insécable) _ (espace insécable fin)
    ...

The French seem to get four mappings for space. Combined with my knowledge of levels, I finally put this together. It seems that the symbols files are tables of what character to produce, given an key and a particular modifier, if the modifier is level two (shift), you use column two, and so forth. I think.

So, I edit the pc symbols:

$ sudo vim /usr/share/X11/xkb/symbols/pc 
...
    key <SPCE> { [ space, underscore ] };

I log off, and back on again and I finally can type shift+space=__wtf__ in comfort.

Inheritance: is-a has-a

Lots of things we learn in school turn out to be naive simplifications of how the real world works, and sometimes we later learn, to our chagrin, that the way we thought about the world really isn’t true at all. Take that familiar organization of life into a giant tree: kingdom, phylum, genus, species. It seems neat enough, but in the grown-up world, people can spend lifetimes arguing about where things fit in this classification.

A related simplification that I learned in school was the rule of when to use inheritance versus composition. It went like so: in this assignment, you simulate a world full of monsters. Zombie is a type of monster, so zombie should inherit from monster. On the other hand, vampires have a coffin, so vampire should have a field that refers to coffin. Now make a UML diagram.

This makes sense as far as it goes, but there’s a major problem: it’s not usually a useful way to think about inheritance when building real programs.

Is-a v has-a perspective makes most sense when thinking about type systems. If a function takes an argument of type monster, it can also take any type of monster, either vampire or zombie. The trouble starts when you use the same reasoning to design a program and it comes back to our taxonomy problem.

You start designing a system by figuring out what your different things are: zombies, vampires, ghosts, coffins and so on. It’s easy enough: three types of monsters, each a class that inherits from monster, and coffin, its own thing. Naturally, you also need people; people need places to live and ghosts need places to haunt, so you have houses. But wait, people aren’t monsters, but they have a lot in common, so they need a base class, say living things. But that’s not quite right; the monsters aren’t technically alive, so maybe they are dead things. Also, houses and coffins seem to be of a non-living type, so that’s another base class. Should it be dead things? If the coffin is made of wood, it used be alive, so maybe that makes sense.

Most real-world characteristics of things are completely irrelevant to most programs. In our simulation, perhaps the only thing ghosts do is haunt, whereas vampires and zombies bite people but don’t haunt. It’s confusing and wasteful to worry about how they are all types of monsters, who are types of dead things and so forth.

Now, occasionally, it does make sense to think of inheritance as an is-a relationship. The cf0x10 parse tree, for example, is a pile of subclasses. When this type of design makes sense, however, it will be obvious; no need to shoehorn everything into it.

What about other metaphors? It’s common, for example, to say that instances of classes are receivers while method calls are messages to that receiver. That’s a useful perspective for language design and it’s useful to have a name for that bit before the dot – receiver.message() – but, again it’s not so helpful a metaphor when designing a program.

In real programs, metaphors like these just tend to cause trouble. Software isn’t made of physical things. A class, in reality, is just a way to group related bits of a program. I prefer not to start by creating any design for a class hierarchy; instead I write code that does the things I need it to do. A class hierarchy, if any, usually emerges from unifying the bits that make sense to put together.

Exception Rules IV: The Voyage Stack

Something I wrote seven years ago and I’m publishing now to see if learned anything since then.

This is part of a series where I review common wisdom about Java error handling. The series is in no particular order, but the first installment explains my categorization method.

Clean up with finally
Truth: high
Importance: high

Joshua Bloch explains the reasoning behind this in Effective Java as “strive for failure atomicity.” Whatever happens, clean up after yourself. Checked exceptions give us one of their rare benefits by just maybe reminding us to write finally blocks.

One way not to write a finally block, however, is like this:

try {
	connection = DriverManager.getConnection("stooges");
	// Snip other database code
} catch (SQLException e) {
	throw new RuntimeException(e);
} finally {
	try {
		// Bad. Do not do this.
		connection.close();
	} catch (SQLException e) {
		throw new RuntimeException(e);
	}
}

If opening the connection throws SQLException, closing the connection throws NullPointer and obscures the original cause. Usually, people suggest wrapping a conditional around the connection.close() call to check for null, but that gets ugly fast and is easy to forget. Instead, follow the next rule.

Make try blocks as small as possible
Truth: high
Importance: medium

Consider this code that obscures which file could not open:

try {
	// Smelly. Don't do this.
	curly = new FileReader("Curly");
	shemp = new FileReader("Shemp");
} catch (FileNotFoundException e) {
	throw new RuntimeException("Whoopwoopwoopwoop", e);
} finally {
	// The close method translates IOException
	// to RuntimeException
	if (curly != null) {
		close(curly);
	}
	if (shemp != null) {
		close(shemp);
	}
}

We programmers wrap multiple lines in the same handler because we are lazy, but like the hare napping during the race, that laziness hurts us in the end; it forces us to reason much more carefully about the application state during cleanup.

Instead, handle the exception as close to its cause as possible:

try {
	curly = new FileReader("Curly");
} catch (FileNotFoundException e) {
	throw new RuntimeException("Missing: Curly", e);
}
try {
	shemp = new FileReader("Shemp");
} catch (FileNotFoundException e) {
	throw new RuntimeException("Missing: Shemp", e);
} finally {
	close(curly);
}
close(shemp);

The benefits of the shrunken try block may not be obvious from this tiny example, but consider how you can now factor out the file opening blocks to a method that simply throws an unchecked exception for missing files. Once done, this code simplifies down to:

curly = open("Curly");
try {
	shemp = open("Shemp");
} finally {
	close(curly);
}
close(shemp);

Note that all these examples lose the original exception when the close method throws an exception. Most applications can afford that minimal risk, but if you believe it likely that your cleanup code will throw further exceptions, a log and re-throw might be appropriate.

Do not rely on getCause()
Truth: high
Importance: high

Peeking at an exception’s cause is equivalent to using something’s privates or parsing the string representation to find a field value. It makes code brittle; moreover, you can only test it by forcing errors in your collaborators.

If you find that some library forces you to do this, consider avoiding that function completely; if you still have no way around it, adorn your code liberally with “XXX” comments, and test as best you can.

Do not catch top-level exceptions
Truth: low
Importance: low

Top-level exception classes like Exception live close to the root of the exception hierarchy. The argument against catching these says that you should avoid it because the specific lower type, such as FileNotFound, traditionally conveys information necessary to handling the exception, so by catching the top-level exception, your handler is dealing with an unknown error, which it probably knows little about.

Actually, the advice should say “do not try to recover from top-level exceptions.” Catching top-level exceptions is not fundamentally wrong or even bad, but because you do not really know how severe the exception was, you should usually do no more than report and organize a crash.

Exception Rules III: The Search for Cause

Something I wrote seven years ago. Did I learn anything in that time?

This is part of a series where I review common wisdom about Java error handling. The series is in no particular order, but the first installment explains my categorization method.

Do not use empty catch blocks
Truth: high
Importance: high

This is the most obvious of the category of exception handling rules that address how to avoid losing error information. The usual example for when you might justifiably ignore an exception goes like so:

static void closeQuietly(Closeable closeable) {
  // Smelly. Do not do this.
  try {
    closeable.close();
  } catch (IOException e) {
    // I've already done what I needed, so I don't care
  }
}

Yes, the comment makes this better than the completely empty catch block, but that is like saying that heroin is fine because you wear long sleeves.

Very rarely, suppressing an exception actually is the right thing to do, but never unless you absolutely know why it happened. Do you know when close() throws IOException? I thought not.

Do not catch and return null
Truth: medium
Importance: medium

Catching and returning null is a minor variation on the exception-suppression theme. Consider the Stooges class, which contains this method:

public String poke(String stooge)
              throws StoogeNotFoundException {
  if (stooges.contains(stooge)) {
    return "Woopwoopwoopwoop";
  } else {
    throw new StoogeNotFoundException("Wise guy, eh");
  }
}

Suppose you want to write another method that checks a Stooge’s reaction to a poke, but Stooges gives you no isStooge method. Instead, it forces you to write this:

static String getStoogeReaction(Stooges stooges, String name) {
  try {
    return stooges.poke(name);
  } catch (StoogeNotFoundException e) {
    return null;
  }
}

If you have to use an API that uses exceptions for flow control, something like this might be your best option, but never write an API that makes your clients do it.

Log only once
Truth: medium
Importance: low

You can also state this rule as “Log or re-throw, not both.” Redundant logging is certainly impolite to those maintaining your application, but hardly the worst you could do. You might log and re-throw for legitimate reasons:

  • Your framework swallows exceptions you throw at it
  • Your application logs in a specific location, different from your container
  • This is part of a larger application, and you worry that your clients might ignore the exception

Do not log unless you need to, but if in doubt, log it.

Always keep the cause when chaining exceptions
Truth: high
Importance: high

Only the very naive intentionally do this, but it is easy to do accidentally, and a very easy way to lose the information about what went wrong.

In 2016, I still think exception chaining is a very important feature and I’ve been surprised by how many mainstream languages lack exception chaining.