Leaps

There’s a lesson I seem to repeatedly forget, applicable to both life and design, in the nature of leaps forward. By “leap” here, I refer to an evolutionary change in the quality of an experience or approach which simultaneously solves many problems at once. The big “ah-has” that transform entire problem spaces and change the ways we think about possibilities.

A leap typically has the ability to take things we knew would be good, but opens them up in ways we couldn’t imagine before. For example, the potential of smartphones was long known before they became widely available. Authors and researchers imagined the possibilities of portable computers and ubiquitous connectivity decades before they became everyday utilities. Sci-Fi predicted things like the internet, Google, widespread social networking, and cryptocurrency. However, what it didn’t predict is the things that come after. The results of the leap.

As designers and engineers, we are constantly asked to quickly sort through possibility spaces, finding the good or elegant options. When searching a possibility space, we often use heuristics, basing our perception of options on experience or existing data that models expectation. Leaps obscure the options that these models fail.

Sometimes you can’t measure an experience after a leap based on what has come before it.

Why did social networking take hold when it did? Why did YouTube and Netflix start to work after so many video streaming services failed? Why are peer-to-peer sharing economies like Uber or AirBnB working now, rather than 10 years ago? Many people felt certain all of these things would eventually work, but why now? It seems easier to look back and remember the reasons these things wouldn’t work, rather than notice the changes which allowed them to.

Perhaps the reason leaps are so difficult to pin down is that their effectiveness comes from a lot of little changes, rather than a few big ones that humans find easier to reason about. Many small reductions in frictions that touch our lives and the lives of others in little ways throughout our day.

I often make the mistake of forgetting leaps when I judge a new idea. While it is necessary to be able to rapidly weed out bad ideas when searching, I miss good ideas too because I didn’t see the leap. And there’s the crux of the problem: many of those little reductions in friction can’t be noticed until you try.

A leap of faith? Or rather intuition?

Fixing hanging deja-dup S3 uploads

After configuring deja-dup to back up to S3, I hit a snag: the process seemed to hang during the upload phase.

To obtain more information, I found that you can enable verbose output via an environment variable (why it isn’t a verbose command-line parameter is a mystery to me):

DEJA_DUP_DEBUG=1 deja-dup --backup

The first S3 upload would start and hang, eventually printing the error:

DUPLICITY: . Upload 's3+http://[...].vol1.difftar.gpg' failed (attempt #1, reason: error: [Errno 104] Connection reset by peer)

It turns out that this is a transient error for new S3 buckets while the DNS changes propagate through AWS (reference). Indeed, the full error contents of curling the bucket described a temporary redirect, which was probably not being handled properly by deja-dup/duplicity/python-boto. After waiting about an hour, the problem was resolved and my backup process went smoothly.

---

As a side note, after tinkering with the IAM profile a bit, this is the minimal set of permissions I could find for the duplicity account:

{
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:ListBucket"],
      "Resource": "arn:aws:s3:::BUCKETNAME"
    },
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject"],
      "Resource": "arn:aws:s3:::BUCKETNAME/*"
    }
  ]
}