Sam Gentle.com

Papercuts

A letter opener in a stone

The Ubuntu project once realised that the main problem with Desktop Linux adoption wasn't usually that there was one big issue, but lots and lots of smaller issues. They started a project that they called One Hundred Papercuts. A papercut was a minor issue that most users would encounter in everyday use of Ubuntu. They set out to find and fix these issues, and ended up discovering thousands of them.

Why was this necessary? I mean, it's not as if the developers didn't know about these bugs; they used Ubuntu more than anyone else and were usually intimately familiar with its issues. But, in a sense, that was the problem: when you do something often enough, the minor inconveniences tend to stop being noticable. Either you work around them, or you just ignore them and eventually forget about them. This might be fine if you're already a committed enthusiast, but for a new user this assault of minor incoveniences is a great reason to go back to whatever OS you came from.

The funny thing is, it's actually the people who use a system the most that are most affected by its papercuts. If it takes a few minutes extra to copy a file once per month, who cares? But if you're doing it several times an hour, those minutes add up very quickly. This means power users are in the perfect position to improve the system. However, instead of adapting the system to their needs, they often adapt themselves to the system. It takes someone new and maladapted to even realise there's something wrong.

Beyond computers, there are a lot of things in our lives that could be called papercuts, that are bad in minor enough ways that we ignore them rather than fix them. But as our actions become associations and associations become habits, it becomes less and less likely that we'll ever change them.

So I think it's worth following Ubuntu's example by taking inventory of these papercuts and setting out to fix some of them. You don't even need to fix everything, even hitting a few of your most frequent inconveniences should be enough to make your life easier.

Unix: An Expanding Explanation

A year ago, I wrote Expanding Explanations, an idea for a structured way to explain a concept that lets you drill down into it at your own pace and direction. I'd been meaning to actually explain something using this technique, but never got around to it. So, at last, here is an Expanding Explanation of Unix:

Unix is many things: an important historical operating system,
  • An important historical operating system
  • Unix was an operating systemoperating system, the low-level system software designed to operate computer hardware and allow user software to run. It was developed at Bell LabsBell Labs, primarily by Brian Kernighan and Dennis Ritchie (who also invented the C programming language). It was primarily developed in the 1970s1970s, which was a watershed decade in the history of computing. Research groups like Bell Labs and Xerox PARC invented a lot of what we recognise as a computer today, including Unixa lot of what we recognise as a computer today, including bitmap graphics, the mouse, WYSIWYG editing, C, object-oriented programming and, of course, Unix. Its design was a response to the complexity of operating systems at the timecomplexity of operating systems at the time, especially its predecessor Multics, which was innovative (many Unix ideas came from Multics) but slow and overcomplicated. Unix invented or popularised many important concepts in operating systemsmany important concepts in operating systems, including hierarchical filesystemshierarchical filesystems (the nested files-and-folders structure common today), the process modelprocess model (a particular way of managing multiple running programs), "everything is a file""everything is a file" (a powerful and generic way of representing resources), pipespipes (a system for connecting multiple programs together), and shellsshells (the idea of the user's control program being a user-controlled program itself). After leaving Bell Labs, Unix evolved from a single operating system into a family of operating systemsfamily of operating systems, either licensees of the trademark "UNIX", implementers of the Unix-derived POSIX standard, or spiritually similar "Unix-like" systems. Most modern operating systemsMost modern operating systems, with the notable exception of Microsoft Windows, are officiallyofficially (as in the case of Mac OS X) or unofficiallyunofficially (as in Linux) based on Unix.
the elegant system architecture that it pioneered,
  • The elegant system architecture that it pioneered
  • Much of Unix's success is due to its simplicity and power. There are three major concepts in Unix: the Kernel, The Kernel is trusted system-level codetrusted system-level code, which means it operates in a special unrestricted mode, usually implemented in hardware. It is responsible for low-level hardware accesslow-level hardware access (ie, drivers) and management of system resourcesmanagement of system resources, which includes coordinating simultaneous access to resources, and regulating sharing so that resources are used fairly. An important resource the kernel coordinates access to is the processor,processor, which can only perform a single task at a time, yet must appear to run multiple tasks at once. This is done via schedulingscheduling, where the kernel runs a task for a short period of time and then stops it so another one can run. While the kernel is the foundation of the operating system, most code runs outside of the kernelmost code runs outside of the kernel for reasons of flexibilityflexibility (the whole kernel has to be updated at once and requires a reboot) and securitysecurity (kernel code has unrestricted hardware access, so it's a juicy target for hackers). Code outside of the kernel is known as user-space. Processes, Processes are a representation of code running in user-space, along with everything needed to run iteverything needed to run it, including the code itself, processor stateprocessor state (data stored by the processor, like where in the code it's up to), memorymemory (each process gets its own virtual memory that maps to various parts of physical memory or disk space), open filesopen files (a list of which files the process has open), and scheduling informationscheduling information (like whether the process is waiting for something and how long it's been waiting). Processes are untrusteduntrusted, meaning they are not allowed to access the underlying hardware directly, so to perform trusted operations they use system callssystem calls, which are special commands that stop the current process to trigger code in the kernel. Using system calls, processes can create copies of themselvescreate copies of themselves using the "fork" system call and run new processesrun new processes using the "exec" system call (replacing the current process, which makes fork-exec a common pairing), among many other thingsmany other things. Almost any operation, from getting the time to reading from the keyboard, involves system calls. Many important system calls are used to deal with data, in the form of files. and Files. Files are Unix's representation of resources. Most commonly, files are static datastatic data like a document or a video, but in Unix files are also used for all sorts of thingsall sorts of things: audio, live system information, random numbers... even an infinite sequence of zeroes. This is a philosophy called "everything is a file", because (almost) all resources are represented this way(almost) all resources are represented this way, although modern Unix environments break from this by having separate systems for things like networking and graphics. Files are also used to represent a process's input and outputinput and output (known as stdin and stdout). Two processes can be joined together by connecting the input of one to the output of anotherconnecting the input of one to the output of another (known as piping). When connected in this way, programs form a pipeline, where each individual program is simple, but powerful in combination with others. This model came to be a central part of the Unix philosophy.
and a general philosophy for software development.
  • A general philosophy for software development
  • Unix, as a whole, is notable not so much for what it did do as how much it didn't do. Unix, as a whole, is notable not so much for what it did do as how much it didn't do. Comparable systems existed at the time and have been created since, but what sets Unix apart from them is its economy of ideas, its sense of simple and elegant power. You can see this in its historical origins, in its minimal architecture, and most of all in its use of pipes and the toolbox model that resulted. Instead of one enormous system to do everything, Unix was lots of tiny systems that could do anything. Instead of one enormous system to do everything, Unix was lots of tiny systems that could do anything. The Unix philosophy is said to be "do one thing and do it well". Perhaps it's better reflected in Sun's old motto, "the network is the computer". The number of combinations of all those tiny tools was far greater than any one piece of software could hope to be. In a time when many modern pieces of software run into the tens of millions of lines of code, it can be helpful to remember this. Unfortunately, in many ways, even modern Unix is no longer a good representative of the Unix philosophy. Unfortunately, in many ways, even modern Unix is no longer a good representative of the Unix philosophy. Successive versions have become more complex and monolithic, mostly in the pursuit of greater performance and control over the user experience. In that sense, the Unix philosophy is at odds with the modern product philosophy: you can't control and optimise every facet of an experience when that experience is actually made up of a hundred independent parts. Maybe the transformation of software from tool into product means the end of the Unix way. However, even if operating systems are no longer Unixy, there is still hope in the web. However, even if operating systems are no longer Unixy, there is still hope in the web. Everything is a link, human-readable text still reigns, and the value isn't any individual member, but the connections between them. Although programming for the web is often far from elegant, it does provide an enormous amount of power, flexibility and decentralisation. The implementation might be questionable, but the web's principles echo those that appeared in Unix many years ago.

First class

A ticket featuring several first-class things

In programming it's common to talk about things being first-class, in the sense of being fundamental to some system; the system is designed with them in mind, and they are part of its core model. To say a language has first-class functions is to say that you can pass functions around like you can any other data and they can be used anywhere.

Some languages do not have first-class functions, meaning functions operate according to some separate system and are usually not as powerful. This is an especially bad thing, because functions are the basis for the abstract unit of operation and being able to manipulate them directly and make them part of your data is immensely powerful. However, there are other non-first-class things too. For example, some languages (like C) do not even have first-class strings, meaning they have to be implemented in terms of something else (arrays).

The point is that everything that's first-class gets a seat at the design table. Having first-class functions means functions (and functional programming) are part of the core of what your language is. Having first-class strings means your language is intended to be used with strings, the other features in your language work well with strings, and strings are as easy to use as anything else. That's not to say you couldn't implement your own strings. Hell, you could implement your own programming language, but it would always be a little bit worse for not being first-class.

Going beyond programming, there are other examples of systems where things are first-class. The internet is one: on the internet, servers are first-class. Every website you visit, bank transfer you make, message you send is going to or coming from a server somewhere. There was a time when every computer was a server, and people ran their websites on their home computers. Those days are basically gone, though; these days you probably wouldn't have your own website, and if you did you'd probably run it on someone else's server. Your Wordpress or Medium or Tumblr blog is second-class on the internet the same way strings are second-class in C.

What's bad about this? Well, nothing, necessarily. Much like people use strings every day in C, people get by just fine on Wordpress. But the internet wasn't designed with you in mind, and that leads to certain consequences that can limit or surprise you. These services can go down (though it's admittedly far less likely than your own server going down) or remove your content if they find it objectionable. Worse still, if you want to do something that hasn't already been envisioned by an existing provider (like, ahem, a demo server), you can't do it. Being second-class on the internet means you can't be creative with it. You can be creative on it, but that's not the same thing. You're limited to ways of using the internet that someone has already thought of.

Another important system with thisn property is the economy. Capitalism is a fine system, as long as you are a first-class member, which means holding and investing capital. Beyond home ownership, not a lot of people do this. The main capital most people have access to is their labour, and instead of investing it by starting a business they rent it out via employment. But there's no chance of employment generating a substantial return; any extra value you generate is immediately captured. In employment, your membership in the economy is implemented on top of your employer; it's second-class.

That's not to say that everyone is able to become first-class in this sense; the reason it is rare is because there are various systematic barriers to membership. In many cases people don't have sufficient resources to pay their biological debts, and have no choice but to rent their labour. In other cases, the barriers are implicit, such as being discouraged from or not knowing about investment as an option. Sometimes, too, the barriers are self-imposed, such as being underconfident or intimidated.

I believe a lot of the disconnect between left- and right-wing politics comes from crossed assumptions in this regard. Every story about a plucky young libertarian who hauled themselves up by their bootstraps is someone who joined the first class. Every downtrodden single mother chewed up by the system is someone stuck in the second class. In both cases, the message is simple: becoming part of the economic first class is good, being in the second class is bad. The question is just about in what proportion we encourage people to get into first vs ease their time in second. Two points that don't make a lot of sense are to assume everyone is in the first class, or to aim for a permanent-yet-livable second class.

Ultimately, much like with programming or the internet, being a second-class citizen in the economy can work fine, but there's always some things that aren't available to you, and the inevitable feeling that the system isn't built with you in mind. That's because it isn't. This may be okay for strings or websites, but I don't think it's something you should be okay with when it comes to your life.

Timing update #2

Time-tracking donut chart for a month

In my original post about time tracking I said I'd give it a shot and report back after a week. I then said I'd follow up after a month, and here we are. I tracked roughly 1000 hours. What did I find out?

Not a lot that wasn't suggested by the weekly data, which is at least indicative that the weekly data was pretty accurate. The biggest chunk, the grey in the top right, is untracked time, so actual tracked time was only 820 hours. Of that, the top was sleep in dark blue with 240 hours (29%), misc in grey with 180 hours (20%), friends in orange with 113 hours (14%) and this website in dark blue with 73 hours (9%).

I'm surprised that friends was as high as it was. 14% is roughly 3.5 hours per day, but I guess I'm more social than I realised. I evidently didn't make that much of a dent in miscellaneous time, because it still took about 4-5 hours per day. One thing that's nice is that prototypes and fun were both up on the list from last time (4% and 5% respectively). I also saw a big chunk of time drop into Markov Technology, which is both accurate and gratifying.

I feel like to get things beyond this point, I'll have to do more than just look at the data. I'll need to monitor it as it changes over time, make theories and predictions, and observe the results. Probably, that's going to mean integrating it in some way with MeDB to do actual analytics with. On a related note, Toggl, which I was using, has a showstoppingly broken concurrency model, so I might need to migrate away to another tool if I want to use it on multiple devices.

Regardless, my overall conclusion is that this is useful data. I don't have terribly useful results yet, but at least one surprise insight (friend time) and at least one meaningful metric (misc time) to work on. Overall I'd call it a success, especially given that the overhead is fairly minimal once you get used to it.

Prototype Wrapup #20

Last week I made 1 prototype, but I suggested that things might pick up this week. As it happens, they did; this week was 5 prototypes, and what's more they were were all pretty small and prototypey. I feel fairly vindicated in my plan to avoid using prototypes for non-prototype work, but I'll wait a couple of weeks before getting too exuberant.

Monday

I wanted to try making something like the next action idea I described a while ago. Something where I can put in a simple list of tasks and just see the next one. I used Elm again, which mostly worked okay, but I didn't quite get to the point of figuring out how to persist anything. It's hard to tell how much is me not being familiar with the platform and how much is the platform being underdeveloped.

Tuesday

I thought it'd be kind of fun to make a hosts file mangler to disable Reddit. I was thinking about making a big physical "distractions/no distractions" switch, which I might still do, but I figured I'd start small with just a command-line tool. It worked great, and actually I got to use someone else's atomic file writing implementation, so hopefully it won't even corrupt the hosts file.

Wednesday

Following on from the distraction switch, I started on a web-based version. I ended up down a giant rabbit hole with routing and mounting and whatnot in Rust's Iron framework, and it took a surprising amount of effort to get that all to work the way I expected. I definitely get the feeling that, at least at the moment, the web isn't Rust's strong suit.

Friday

This was the start of an attempt to work around a limitation of CouchDB that makes it difficult to post things in advance. CouchDB is deterministic, which means it doesn't know how to update views based on the current date. So I wanted to make something that could run in a cron job and update future posts for me. I thought I'd do this one in Rust too, just to get an idea of whether it's good for the kind of standard web api plumbing code I've done a lot of in Coffeescript and Javascript. The answer is no, or at least not yet. Dealing with JSON is really awkward, though I hear there's a new JSON library coming that might be easier.

Saturday

Part 2 of the CouchDB updating was to make it put an update to bump the scheduled document. This turned out to be tricky too (see above about Rust not really being great at web stuff), especially because I ended up fighting with the percent_encoding function for ages before giving up. Still, I got it to the point where it can query and update, though it doesn't yet have the right logic to figure out whether it should update.