javascript – Should Be Simple

Do you need a Model?

It’s common for graphical programs to work something like this:

Pull data from persistent storage: database, web server and so on
Tuck that data away in a “Model” object
Display the data
Receive updates from either the user interface or the persistent storage
Hope that the screen or the storage also updates, respectively
Goto 4

The role of “hope” in this scenario is played by something often called “Data Bindings.” The problem with this design is that it’s an obviously bad idea: by keeping a Model like this, you’ve created a cache and cache invalidation is known to be a hard problem.

I’ll assume you’re writing a webapp, because, who isn’t? I claim you don’t need any Model client-side at all. If you have a long memory, you might say that I’m just being ornery and yearning for a time when we didn’t need a megabyte of JavaScript to display a form with one field:

You’ll say you need these Models and Bindings because modern.

You’d be partly right. This does make me ornery, but let’s step back and look at what might be the number one killer of program maintainability: mutable state. A graphical interface is nothing if not a big pile of mutable state and the job of its programmers is to wrangle the ways it mutates.

In a sense, every useful program needs to grapple with this problem. Programs that lack persistent data or ways to display it are pretty useless, in the way that a tree might fall but nobody knows whether it makes noise and most people don’t care.

I advocate that you scrap the client-side Model, but I don’t claim this will make your webapp easy to write. No matter what, you’re dealing with a pile of mutable state and that’s bound to be tricky.

Why make it trickier than necessary? You really do need widgets on the screen: checkboxes that can be checked, or not, spans that could say one thing, or another. Do you need a shadow copy of those things as well?

Now, not all state duplication is called a Model and not everything called a Model is state duplication. Elm, for example, takes the view that Model is the application state but avoids state duplication by virtue of being purely functional.

Most programs aren’t written in purely functional languages, so the Model is state duplication, but users don’t see Models. They see widgets and to them, the state of the widgets is the state of the data. In a programmer’s ivory tower, you might argue that the Model is the source of truth, but your arguments matter not at all to actual people.

So don’t fight it. The path to enlightenment lies in realizing that once you’ve rendered the data, it doesn’t matter anymore. Throw it away.

Wait, that might work for your Web 2.0 site and its DHTML, but I have a Web APP. It’s modern. It has a span over here and a checkbox over there and the span says ‘frobbing’ or ‘unfrobbing’ depending on whether the checkbox is checked.

Ok, but I’m not seeing how Models and magic Bindings are simpler than this:

checkbox.onchange =
  span.textContent = checkbox.checked ? 'frobbing' : 'unfrobbing'

Well, it’s not just this one span, this blink tag over here needs to appear if you’re not frobbing. With your plan, I need to add another line of code to handle that.

True, but you’d have to add something when backing it with Models and Bindings as well. It wouldn’t be less code, just different code and more indirection.

There are more ways frobbingness can change though. I don’t want to copy this logic in every place.

And well you should not, but you already have the tool to solve that problem: a function. Move the interface update logic to a function and call it anytime frobbingness changes.

I see what you’re saying, but I don’t like it. The span and the blink are in logically different components. I’ll end up with a bunch of functions that update both, but don’t sensibly belong to either.

That’s a very good point.

Partly, the appeal of the client-side Model is that it seems we ought to be able to bind all the widgets to the same object, giving us a kind of fan-in-fan-out approach to updating them. In principle, you’d have just one Frobber and you’d change its isFrobbing property, then all the dependent widgets would get notified. People have been trying to implement this concept since the invention of the computer and it’s gone by many names. Recently, the buzzword is “reactive” programming.

This idea can work well in some systems. Spreadsheets and style sheets are examples of very successful reactive programming models, for example. It works well in limited domains with declarative or purely functional languages.

In non-functional practice, the same data structures aren’t convenient for all widgets. Imagine that Frobbers are parts of Whatzits, but in one place you display all your Frobbers and in another you display the content of a select Whatzit. Your Models probably look something like this:

frobbers:
  - id: 1
    isFrobbing: true
  - id: 2
    isFrobbing: false

selectedWhatzit:
  frobbers:
    - id: 1
      isFrobbing: true

Now which copy of Frobber number 1 is the true Frobber? Ideally, you’ll make them point to the same object so it doesn’t matter. Your Binding magic may or may not be able to make sense of that arrangement and the temptation to duplicate data is strong. Models and Bindings just moved the concrete problem of what appears in widgets to the abstract problem of what goes where in the invisible Model.

With or without magic Models, as the application grows, you’ll need to put things in sensible places and call them in sensible ways. That is the whole job of user interface programming.

If you keep doing this for all the data and all the widgets in your app, you’ll notice you keep doing the same kind of tedious things over and over. This thing over here updates some data. Need to show it over there. Where’s the right place to put that code? Here, there, somewhere else? Decide, repeat.

Naturally, as programmers, we think we can automate away the repetition, but some complexity is fundamental.

Evolution

In the beginning, there was html


<form method=post action=signup>
  <label>Username       <input name=user></label>
  <label>Password       <input name=pw type=password></label>
  <label>Password again <input name=pwv type=password></label>
  <button>Submit</button>
</form>

Then came JavaScript

<form id=signup method=post action=signup>
  <label>Username       <input name=user></label><br>
  <label>Password       <input name=pw type=password></label><br>
  <label>Password again <input name=pwv type=password></label><br>
  <button>Submit</button>
</form>

<script>
<!--
document.forms.signup.onsubmit = function () {
  if (this.pw.value != this.pwv.value) {
    alert("Passwords don't match")
    return false
  }
}
// -->
</script>

And jQuery

<script src="https://code.jquery.com/jquery-1.1.4.js"></script>

<form id="signup" method="post" action="signup">
    <label>Username       <input name="user"></label><br/>
    <label>Password       <input name="pw" type="password"></label><br/>
    <label>Password again <input name="pwv" type="password"></label><br/>
    <button>Submit</button>
</form>

<script>
// <![CDATA[
$('#signup').submit(function () {
  if (this.pw.value != this.pwv.value) {
    alert("Passwords don't match")
    return false
  }
})
// ]]>
</script>

And json

<script src="https://code.jquery.com/jquery-1.5.js"></script>

<form id="signup">
    <label>Username       <input name="user"></label><br/>
    <label>Password       <input name="pw" type="password"></label><br/>
    <label>Password again <input name="pwv" type="password"></label><br/>
    <button>Submit</button>
</form>

<script>
/*jslint browser: true */
/*global $, alert, window */

$("#signup").submit(function (event) {
    "use strict";

    var form = $(event.target);
    var formData = {};

    form.find("input").each(function (ignore, el) {
        var input = $(el);
        var name = input.attr("name");
        formData[name] = input.val();
    });

    if (formData.pw !== formData.pwv) {
        alert("Passwords don't match");
    } else {
        $.ajax("signup", {
            type: "POST",
            contentType: "application/json",
            data: JSON.stringify(formData)
        }).done(function () {
            window.location = "signup";
        }).fail(function () {
            alert("Error");
        });
    }

    return false;
});
</script>

And Ecma 6

<form id="signup">
    <label>Username       <input name="user"></label><br>
    <label>Password       <input name="pw" type="password"></label><br>
    <label>Password again <input name="pwv" type="password"></label><br>
    <button>Submit</button>
</form>

<script>
/*jshint esversion: 6 */
document.querySelector("#signup").addEventListener("submit", submitEvent => {
    "use strict";
    
    const form = submitEvent.target;
    const formData = {};

    submitEvent.preventDefault();

    for (const input of form.querySelectorAll("input")) {
        var name = input.getAttribute("name");
        formData[name] = input.value;
    }

    if (formData.pw !== formData.pwv) {
        alert("Passwords don't match");
    } else {
        const xhr = new XMLHttpRequest();
        xhr.addEventListener("load", () => {
            if (xhr.status === 200) {
                window.location = "signup";
            } else {
                alert("Error");
            }
        });
        xhr.addEventListener('error', _ => {
            alert('Error');
        });
        xhr.open("POST", "signup");
        xhr.setRequestHeader("Content-Type", "application/json");
        xhr.send(JSON.stringify(formData));
    }
});
</script>

Progress?

Linting JavaScript considered harmful

I am in a minority of programmers so small that I might be the only member. I can’t find anyone on the Internet advocating my position: you should not lint your JavaScript.

I don’t mean you shouldn’t use this or that flavor of lint. I mean that you shouldn’t use any JavaScript linter. At all.

First, the arguments for linting…

Lint catches bugs?

How many bugs does it really catch? Only a few. Rules against unused variables can be useful when you’ve renamed something in one place and forgotten another place, for example. Trouble is, I’ve only noticed lint catching bugs in code that wasn’t complete or tested anyway, and therefore already broken by definition.

The major category of bugs caught by lint can be caught instead by a simple statement:

use strict

JavaScript strict mode really does find bugs, almost always results of accidentally overwriting globals. It’s tragic that we need lint to tell us about missing strict mode declarations when browsers could warn us.

But is it worth bringing in lint for the few bugs it can find?

I think not; the better approach is simple: learn to use strict reflexively. Then spend the effort you were going to use typing semicolons on testing instead.

Lint enforces consistency?

So what? Consistency for its own sake is a pursuit of feeble minds.

In writing, particularly writing computer programs, consistency is a proxy for something much more important: readability. And readability is not something that computers yet understand well.

It is perfectly possible, common, in fact, to write incomprehensible code that passes a linter.

But, you say, “lint rules can help a little, so we should use them. We just need to pick a ruleset.”

Lint reduces bikeshedding?

In the beginning, lint was an inflexible representation of one man’s own preferences. Next, everyone bought into the idea they should lint their JavaScript, then ~~adopted Crockford Style and moved on~~ argued endlessly about which rules were important.

We went from the ultra-rigid linter, to the ultra-configurable, to the ultra-pluggable. At each step, we introduced more and more time-consuming opportunities to argue about picayune issues.

To paraphrase Tim Harford:

Lint is such a temping distraction because it feels like work, but it isn’t. When you’re arguing about lint rules or fixing lint errors, you’re editing code, but you’re not getting things done.

So should I never lint?

All that said, I recommend lint for one purpose: it can be a useful way to learn about the idioms and pitfalls of a language. Run your code through a linter and learn why it complains.

This use of lint as a teaching tool is actively discouraged by the way most people advocate using it. Imagine you set up a hook that says “all code must pass lint before commit.” Those who actually could benefit from the lint suggestions are blocked by them, thus encouraged to “fix” the “problem” as quickly as possible: obey the tool and never learn why.

Secure email monitor

I segment my online accounts into two groups: valuable accounts and everything else. “Valuable” can vary a little by personal priorities, but for most of us, our most valuable accounts will be those with direct access to cash: banking and investments.

By transitivity, any account that can allow access to those accounts is also in the valuable and high-risk group. These include financial aggregators and any email address used for account recovery.

I would like to keep the only computer with access to the valuable accounts locked away in a dungeon guarded by trolls, but highly restricted access also makes it difficult to monitor activity. I want to be able to see notifications on all my devices while not actually allowing them access.

Enter Google Apps script.

Set up the script

You’ll need an everyday – normal – email address, one you access from anywhere. Then, you’ll need another that you only access from secured devices, the restricted – high risk – account.

Make a restricted Gmail account for yourself.

Don’t use an existing email address in for recovery
Do use a password manager and make sure you have a backup
Do set up two-factor auth

Go to https://script.google.com/

Paste:

function digestAndArchive() {
  // CHANGE THIS TO YOUR NORMAL EMAIL ADDRESS:
  var monitor = "youreverydayemail@example.com"

  // Docs say that if you have many threads, for some unspecified value of "many", you
  // should use the paginated version of getInboxThreads, as the simple version will fail.
  //
  // It turns out that means "fail silently", returning at most some arbitrary number of
  // threads, and there is no obvious way to know there are more. I suspect the "correct"
  // way is to keep calling the paginated version with an increasing start index until
  // it returns nothing, but that seems ridiculous. For practical purposes, this function
  // returns more threads than you are likely to receive in a day.
  //
  // So, upon first installing this script on a long-ignored inbox, it might need to run
  // several times before it clears out the inbox, but that shouldn't hurt anyone.
  var threads = GmailApp.getInboxThreads()
  var bySender = {}
  for (var i = 0; i &amp;lt; threads.length; i++) {
    // I'm assuming this is a receive-only email address, so all messages in a thread
    // presumably have the same sender (or similar). Organizing by sender isn't
    // strictly necessary, but I think the final digest is more understandable.
    //
    // The docs don't say whether the first message is the most recent or not, but that
    // generally should not matter.
    var message = threads[i].getMessages()[0]
    var sender = message.getFrom()
    bySender[sender] = bySender[sender] || []
    bySender[sender].push(message.getSubject())
  }
  var body = ''
  var indent = '\n  - '
  for (var sender in bySender) {
    body += sender + indent + bySender[sender].join(indent) + '\n'
  }
  // Experimentally, it seems that GmailApp.sendEmail encodes the body as text/plain
  // so it should be safe to drop any old string in it. Would be nice to find
  // documentation to that effect. It munges astral plane characters, but for my
  // purposes here, I don't care.
  GmailApp.sendEmail(monitor, "Daily digest for " + new Date(), body)
  for (var i = 0; i &amp;lt; threads.length; i++) {
    // GmailApp.moveThreadsToArchive() can move multiple threads at once, but throws an
    // error and moves nothing for more than 100 threads. That's a pretty low limit when
    // you first run this on an inbox you haven't been regularly cleaning, so move one
    // by one.
    GmailApp.moveThreadToArchive(threads[i])
  }
  GmailApp.sendEmail(
    // Error detection. When a trigger-based app script fails, Google sends you an email
    // from apps-scripts-notifications@google.com, so it's a good idea to set up a
    // forwarding rule that emails from apps-scripts-notifications@google.com should go
    // to the email address you monitor.
    //
    // Sometimes, however, Google breaks app scripts silently. In 2020, they restricted
    // permissions and this script needed to be re-approved. Instead of failing,
    // ``GmailApp.getInboxThreads()`` returned an empty result set. The daily digests
    // continued to arrive. Each digest was empty, but it's easy not to notice that
    // digests have been empty for an unusually long time.
    //
    // So, this adds a canary email. If you receive a digest and it doesn't list this
    // status email, something has failed.
    Session.getActiveUser().getEmail(), "Daily digest OK", "")
}

Remember to change the email address in the script above to your normal, every day email. Save, as, for example “daily-digest”
To run by hand, go to the Run menu or just click the play button in the toolbar. At your normal address, you should see an email like this:
Subject: Daily digest for Fri Dec 02 2016 08:11:39 GMT-0500 (EST)

From: [your restricted address @gmail.com]
Andy from Google <andy-noreply@google.com>

– Josiah, welcome to your new Google Account
Run on a timer
To schedule, click Resources (menu) -> Current Project Triggers -> Click the link to add a trigger.
Set up to run daily on a timer. The time option is in your local time, as detected by Google.


When you save the trigger, it will prompt you to authorize the trigger to run. Click “Review permissions”, which opens a new popup window, then allow.
Run the script by hand again after setting up the trigger. It should prompt for another required permission.
Daily summaries of your restricted account should now start to appear in your normal email.

If you’re paranoid – and if you’ve gotten this far, you probably are – delete the digest emails from time to time. Deleting the digests removes a possible attack vector on your high-value account because Google’s account recovery process asks the date you created an account as a form of identity verification.

Qwerty War: days 2 and 3

For day 2 of the Qwerty War project, I wanted to get all the visuals roughed in, though without final sprites for the player and enemies. It turns out this actually took two days.

I considered doing the whole thing by moving divs about the screen, or even svg everywhere, but the obvious choice is canvas, so best not to overthink it.

User controls – word input, health meter and score – are still regular html elements overlaid on the canvas. At first, I thought I would do the same with the enemy words: reposition the table cells from day one’s text-only version to follow sprites around, but soon decided drawing the text with canvas api would be simpler.

More structural changes came from the rework necessary to allow for delay while bullets reached the targets. This may not have been strictly necessary: originally, I planned to keep reactions instantaneous. When the player shoots, for example, the enemy would explode immediately. I would then animate the bullet motion toward the enemy. Since the bullet would whiz over in less than a tenth of second, the delay would not be noticeable.

As I was starting, though, this felt wrong. The code made more sense if I associated the explosion with a bullet impact, rather than the shot. If I’m inclined in the future, this should allow features like slow motion.

Bullets are the only effect not built with sprites: they are just lines with gradients to fade out toward the shooter. The “collision detection”, if you can call it that, is easy. Since every moving element is arranged radially from the player, it’s a simple matter of checking whether the bullet traveled far enough to hit its target, which causes the target to explode.

Explosions, of course, are the fun part. The nifty http://explosiongenerator.com/ gives me the look I want. There are three levels of enemies in the game, from words length three up to five, so I just need an explosion per enemy type. Stopping the explosion a few frames from the end gives a nice debris and blood splatter to paint on the background. By randomizing the rotation of each explosion and its ending frame, just three explosions look pretty organic.

Explosion generator gives you a download full of .png images; I used ImageMagick to combine them into a single image suitable for animation:

montage images/explosion0{000..191..4}.png -geometry +0+0 -tile 12x4 -background none explosion1.png

That squirly brace stuff is a handy shell trick to pick every fourth image from a desired range. From there I just use JavaScript to animate these frames.

So far, the game on day 3 looks pretty good, but a few details bother me:

Enemies run over each other, sometimes obscuring the words you need to see. This also happens in the original Qwerty Warriors. It’s not too much of a problem – you could even say it makes you think a little more strategically about who to shoot – but it seems unpolished. Would be nice to put in collision detection so enemies don’t run each other over.
The end is abrupt. It might be a neat effect to finish animating all bullets in flight and explosions after the player dies.
Bullet animations are not entirely satisfying. They are functional, but not exciting.
As I was testing, I realized muzzle flashes are a nice feature to show you who is shooting at you. I put them in as just one extra sprite for each of the player and enemy types, flash on or off, but there probably should be a few frames to that animation.
The bullet trail extends through the shooter. I mentally justify that by saying it’s the stuff coming out of the rear of a rocket or something, but actually it just looks odd.

Nonetheless, it looks decent and is becoming fun to play. Day four will be for sound effects.

Qwerty War: day 1

“Qwerty War” is what I’m calling a small typing game inspired by Qwerty Warriors.

The concept is that you are a keyboard-slinging warrior surrounded by never-ending waves of enemies. There is no escape, but you’ll take as many of them down with you as you can. Qwerty Warriors is my favorite typing game; since it surrounds you with enemies, it has an intensity that other typing games lack. Its remake, Qwerty Warriors 2, by contrast, only sends enemies in from the top, typical for typing games, and boring.

The original is built in Flash and often syndicated on shady sites who make their money via ads for scareware and other garbage. So my own Qwerty War will be an open source JavaScript only game.

On day one of this project, my goal is to get a text-only version working with all components. In the text only version, a table of “word” and “range to enemy” replaces the visual of enemies encircling the player, but it contains all the actual gameplay components. There is no reason for a typing game not to be accessible to the blind, so the text only version will remain as a fallback to the final graphical version, build on html canvas.

The final version will include a server-side component for saving the leaderboard, but the game itself runs entirely in the front end. After day 1, it is playable, if a bland. Source is on Bitbucket.