In Afghanistan, watching someone learn to code for the first time

I recently had the opportunity to observe how a total newcomer to coding / computer programming starts their learning journey. She was embarking on this journey in Kabul, which brings some special considerations that I thought I’d outline here.

First off, coding in Afghanistan? I actually happen to think it’s not the worst thing someone could do with their time. Power outages and internet costs, while better than they were in previous years, still make the prospect of spending hundreds of hours learning skills a precarious proposition. (Outside Kabul, just to give some context, even big cities like Kandahar are starved of electricity. When I was down there recently we had a couple of hours per day at best.) But all that to one side, there’s a need for skilled professionals in the computer science / engineering world and there seems to be a decent amount of room for remote work. (Note that there’s some debate about the extent of the shortage but I don’t think it’s getting ahead of myself to suggest that proficiency in computer science will be a valuable skill in the years to come.)

If you are going through university in Afghanistan you’re teaching yourself anyway, to some extent. The quality of the tuition — even in some of the private universities — isn’t great and the job market doesn’t reward degree certificates; it rewards competency, something which you aren’t necessarily going to get from a degree / certificate. To use some of the free resources available online would seem like a smart option. Worst case, you make yourself a little more employable because you took some initiative and have skills to contribute to a new employer. Another scenario is that you have some skills you can market around the world, all from the convenience of your home.

All of this requires English. It’s an unfortunate fact that there really is no way round learning English if you want to teach yourself computer science skills. There are ways round it, especially at the beginning, but if you’re going to progress you’ll really have to have a decent level of competency at understanding technical documents / tutorials and the like. I wish it was otherwise, but that isn’t changing any time soon.

I wanted to recommend my friend learn via some resource that was at least partly in her native language, Dari / Persian. After all the fanfare of recent years about how young people are learning how to code around the world in a variety of languages, I was a bit disappointed at the options available. For Dari, there were basically no options. One site, code.org, had some initial courses that had been translated into Persian, but it was the Iranian version / dialect and this caused some problems in comprehending the instructions later on. Worse still, the courses were only half-translated. Videos in between units were in fast idiomatic English, with scenes and situations that seemed representative of some alien universe. Yes, it’s nice that Bill Gates was there to explain how variables work, but first let’s slow down and explain what a variable is, preferably without flashy graphics that don’t actually explain the topic.

There were various assumptions made about students’ experience and skills in mathematics. Early units in the code.org syllabus expect you to understand basic geometry and some algebra. I guess you have to start somewhere, but I wish there were ways of teaching this starting from an earlier baseline of understanding.

I had initially thought that a good way to get my friend thinking about some of the bigger issues would be to have her use a Raspberry Pi and handle some basic Linux administration but that ended up being too many new things at one time. Some of the much-vaunted tropes about learning to code really don’t make much sense.

One other practical problem turned out to be that code.org is a data-hog. My friend ended up having to spend a non-trivial amount of money recharging her 4G phone (from which to tether her laptop) because code.org used fancy javascript animations and who knows what else. I feel like this is a serious symptom of a blind spot in Silicon Valley’s efforts at some kind of cultural imperialism. They can’t imagine themselves into the shoes of a nascent coder, so they produce these bloated websites that mean that while someone can maybe code for an hour or two using fancy javascript animations, maybe they won’t be able to afford the hundreds of hours that it takes to become competent.

I feel like better skills to teach at this level are more fundamental: how do you figure out the answer to a problem? how do you solve problems in general? This is actually the real skill that you want someone to master, not whether they can get a turtle to draw a circle in 4 colours, and this skill is not taught at all in these courses. (A corollary: how do you find good / worthwhile problems? or how do you figure out what things are worth working on? None of this is taught at all).

The kind of education provided by sites like code.org gets it backward. In this way of seeing the world, you can get busy ‘learning to code’ but it won’t necessarily fit into your life and you can’t really make real progress without other external guidance and support. The dream of universal education for a young person in somewhere like Afghanistan, delivered by these kinds of programmes, seems more smoke and mirrors than a real ladder towards education and real skills.

Diagnosing Diabetes with Weka & Machine Learning

[I mentioned two weeks ago that I was working to dive into the practical uses of machine learning algorithms. This is the first of a series of posts where I show what I’ve been working on.]

The Pima Indians dataset is well-known among beginners to machine learning because it is a binary classification problem and has nice, clean data. The simplicity made it an attractive option. In what follows I’ll be mostly following a process outlined by Jason Brownlee on his blog.

The Pima Indian population are based near Phoenix, Arizona (USA). They have been heavily studied since 1965 on account of high rates of diabetes. This dataset contains measurements for 768 female subjects, all aged 21 years and above. The attributes are as follows, and I list them here since they weren’t explicitly stated in the version of the data that came with Weka and I only found them after a bit of digging online:

• preg - the number of times the subject had been pregnant
• plan - the concentration of blood plasma glucose (two hours after drinking a glucose solution)
• pres - diastolic blood pressure in mmHg
• skin - triceps skin fold thickness in mm
• insu - serum insulin (two hours after drinking glucose solution)
• mass - body mass index ((weight/height)**2)
• pedi - ‘diabetes pedigree function’ (a measurement I didn’t quite understand but it relates to the extent to which an individual has some kind of hereditary or genetic risk of diabetes higher than the norm)
• age - in years

This video gives a bit of helpful context to the data and the test subjects:

https://www.youtube.com/watch?v=pN4HqWRybwk

I also came across a book by David H. DeJong called “Stealing the Gila: The Pima Agricultural Economy and Water Deprivation, 1848-1921” which describes how the diverting of water and other policies “reduced [the Pima] to cycles of poverty, their lives destroyed by greed and disrespect for the law, as well as legal decisions made for personal gain.” It looks like a really interesting read.

The Problem

The idea with this data set is to take the attributes listed above, combine them with the labelling (i.e. we know who has been diagnosed with diabetes and who hasn’t) and figure out the pattern as much as we can. Can we figure out if someone is likely to have diabetes just by taking a few of these measurements?

The promise of machine learning and other related statistical tools is that we can learn from the data that we have to make testing more useful. Perhaps we only need your height, genetic risk factor and skin thickness to make such a prediction? (Unlikely, but still, perhaps…). If we emerge from our study with a statistical model, how well does it perform? How much can we generalise from the data? What would be an acceptable error rate in the medical context? Is it 80% or is it 99.99%? The former would save millions of dollars in test costs but would throw lots of errors; the latter would be highly accurate but it might be expensive to calculate the model.

The use case for this specific case would maybe be to identify at-risk individuals who are on the way to a diagnosis of diabetes and intervene somehow. Our motivation here is clear: people don’t want to be diabetic, so how early can we catch this transition? It would save governments money, expose fewer people to unnecessary tests and improve their quality of life.

I’m not a doctor, but to solve this problem manually would seem to require monitoring of blood tests (glucose and insulin levels), perhaps looking at exercise and diet, and also weight. At scale across the population of an entire country, for example, this seems like it might get expensive and/or too much for one person to process in their head. The data isn’t too large or complex, but it still seems to be useful you’d want to automate it to some extent.

There are some potential ethical issues around the data. Everything offered as part of the table of data is anonymised, but there are some outliers (see below) that I have to believe wouldn’t be too hard to find. The applicability of whatever model comes from this data will likely only have a limited application — the data is drawn only from women, after all. I also noticed that while the data is no longer available on the UCI Machine Learning Repository website, it still comes packaged with Weka. There was a notice on the UCI site (which I can no longer seem to be able to locate) stating that the permission to host the data had expired. It is unclear to me what’s going on with the permission there.

Data Preparation

Exploring the data using Weka’s explorer tool plus the attribute list above we can see that we have some blood test data, some non-blood body measurements and this genetic marker (presumably achieved through either blood tests or interview questions about family history). As I was working to understand the various attributes, it occurred to me that for this to be really useful, we’d want our model to work on data that wasn’t derived from blood tests; they’re expensive and they’re invasive. I didn’t get round to doing that for this round of exploration but it’d be high up on my wishlist next time I return to this data.

There are only 768 instances, so it’s still quite a small data set, especially in the context of machine learning examples. This is probably explained by the fact that it’s real medical data (so there are consent issues) plus the fact that it is several decades old and the processing power available then didn’t lend itself to processing mega-huge sets.

Thinking about what attributes might be removed to make a simpler model, I first thought that maybe the number-of-pregnancies might be dispensable, but then I thought to the number of hormonal and other changes that happen and I guess actually it is probably quite important.

There were some outliers in the data that I identified as needing further consideration / processing before we get our model trained:

• There were some women who had been pregnant 16 or 17 times. They were on the far edge of the long tail, but I ended up leaving them in for the model rather than deleting them completely.
• There were 5 people who had 0 as their result for ‘plus’, which seems to be an error. I decided to remove these.
• There were 35 people who had 0 as their blood pressure, which seems to be an error.
• There were 227 people with 0mm skin thickness. This is possible, but I think it’s more likely that no measurement was taken, at least for a lot of them.
• There were 11 people who are listed as weighing 0kg. That seems to be an error.

After I’d identified these various outliers I decided to make a series of transformations to the whole set. From this I’d emerge with three broad versions of the data:

1. the baseline dataset, with nothing removed or changed
2. the outliers removed completely and replaced with NaN values
3. the outliers replaced with mean averages for each particular attribute

For each of these broad versions, moreover, I prepared three separate versions:

1. all values normalised (ranges and values for all attributes transformed to being from 0-1 instead of being in their original ranges. i.e. maximum weight as 1 and minimum weight as 0 etc)
2. all values standardised (set the mean for the data as being zero and the standard deviation to 1)
3. all values normalised and standardised (i.e. both transformations applied)

Producing these various versions of the data was something I learned from Brownlee’s book, “Machine Learning Mastery With Weka”. It turned out to be somewhat fiddly to do in Weka. In particular, every time you want to open up a file to apply transformations the default folder it remembers is often several folders down in the folder hierarchy. By the ninth transformation (there were nine sets in total, by the end of this process) I was ready for a more functional / automated approach to these data conversions!

Weka does offer some nice tools for the initial exploration of the data. Here you can see two charts that are generated in the ‘explorer’ application. First we have a series of simple bar charts visualising all the individual attributes. Then we have a plot matrix showing how all the various attributes correlate to each other (or not, as was mostly the case for this data set).

View fullsize

Visualisation of Pima Indians dataset attributes (auto-generated in Weka)

View fullsize

Plot matrix showing visualisation of correlations between all attributes (auto-generated using Weka)

Choosing Algorithms and Training the Model

Given that I’m very much at the beginning of my machine learning journey, I don’t have any strong sense of which algorithms might be more appropriate or not for this particular data set. I knew that this is a classification problem and not regression (i.e. we’re trying to decide whether people have diabetes or not — two categories — instead of predicting where people fall on a scale / spectrum) so that ruled out a few options, but really the field was wide open.

Jason Brownlee advises taking a sample of around ten different algorithm families to get an initial sense of whether there are any clear outliers (either over-performers or under-performers). Once I have a better sense of the overall space, I can then tweak things, or double down on a particular algorithm family to select a more limited feature set perhaps.

For this algorithmic spot-check I chose 12 algorithms, sampling from all the main families as I currently understand them. Running this set, I immediately came across an error message: Weka was telling me that the function.LinearRegression algorithm doesn’t function for classification algorithms. I removed that and reran the tests.

When doing this kind of test, it helps to have a baseline accuracy figure against which you can compare how much these fancy algorithms are improving predictions. In Weka, this is called a ZeroR algorithm and I think it basically says that everyone has no diabetes. For this dataset, it got 65.11% accuracy, which isn’t bad all things considered!

(Note that everything here is being run through k-fold cross-validation where training and test data are kept separately, and then this is repeated ten times. The final results are averaged out between them. Weka does this all with great ease, making it pleasant to conform to best practices when it comes to data science).

View fullsize

This figure shows how logistic regression was the best performing algorithm out of the box at 77.47% accuracy. I read somewhere that it often performs well on binary classification problems, so this didn’t surprise me. Support Vector Machines (listed as SMO in the Weka GUI) are also supposedly quite good for binary problems and it was only two-thirds of a percent behind logistic regression. Using Weka’s tools for statistical analysis of the result, I came to the conclusion that LMT, logistic regression, SGD and SMO were all worth further exploration and tinkering.

For example, I tried the following with the Support Vector Machines algorithm:

• tweaking the value of c (complexity) to see if 0.25 performed better than 0.75, for example. It turned out that 0.5 was the sweet spot for the c value.
• trying different kernels - I tried most of the options listed in Weka and they all performed pretty poorly. In particular, RBF (radial basis) was really poor.

None of my tweaks really seemed to improve the accuracy of the model. I imagine that some of the algorithms function better with more data, but I am not in a position to generate more.

The next step was to try some ensemble methods where the predictions made by multiple models are combined. In particular, bagging/bootstrap, boosting and voting were all recommended as options to try out.

View fullsize

You can see here that ultimately none of those outperformed logistic regression, which was surprising to me. I’m not at a place where my statistical understanding can explain why that’s the case — ensemble methods seemed to offer the best of many worlds — but I can’t really argue against the results.

Finally, I tried the MultiLayerPerceptron to throw one possible implementation of Deep Learning at the problem. This performed pretty poorly as per default configuration.

Findings / Conclusions

The best accuracy I was able to achieve on this data set was using a logistic regression model. This performed with 77.47% accuracy (standard deviation of 4.39%). We can restate this as an accuracy of between 68.96% and 86.25% accuracy on unseen data. This is slightly disappointing since it isn’t that much better than the ZeroR algorithm.

Towards the latter stages of my work on this problem, I came across a blog post by someone who used a neural network to reach results of 95% accuracy on this same data set, showing that there are models that bring dramatically improved performance. I don’t understand neural networks enough to be able to evaluate what he did (i.e. to know whether this is simply overfitting or actually a performant / real improvement on my results). Nevertheless, it seems like a significant improvement.

As my first big push to work on a real data set using machine learning tools, this process was instructional in the following ways:

• Weka is easy to use and it makes some of the best practices in data science no-brainers to implement
• Constructing the various data sets, implementing the experiments to compare the algorithms and so on was made slightly tedious by the GUI interface. If I wanted to run through many more variations it would have been prohibitively tiresome to have to manually click through all the options.
• Weka is slow (or maybe my linux laptop is slow). Some of the algorithm sets I tried (Support Vector Machines, for example, or ensemble methods using SVMs) took 20+ minutes to run. The data set wasn’t huge at all, so I have to imagine that a real ‘big data’ set would make this kind of quick incremental exploration and iteration difficult to practice. Weka is, of course, a Java app and I’m running that on my Mac. I suspect that if I were to run similar algorithms through Python (or even better, C) on my Mac I’d get significant performance improvements.
• I have very little sense of the variation between the various algorithms, what each one does and where the strengths and weaknesses lie. I want to tackle this from two directions: improving my baseline understanding of statistics and also just getting more experience implementing them for practical problems such as this one.

The next problem I want to tackle is that of the UCI soybean dataset). Each instance describes properties of a crop of soybeans and the task is to predict which of the 19 diseases the crop suffers. Again, the dataset isn’t huge but it is a multivariate classification problem so there are new challenges to be tackled there.

From AI to to Brahms

The past few days have been consumed by a flurry of meetups, conferences and other engagements.

I attended the inaugural PyLondinium conference at the Bloomberg office in London. Lots of interesting speakers. Some went over my head but overall I was pleased at how much was accessible. It has been a while since I’ve done much Python coding, but apparently it hasn’t been completely forgotten. The day before the conference started I did a day-long hackathon with Trans*Code, an organisation that sets up these kinds of events in support of trans community issues in the UK.

Much of Tuesday was spent at CogX, a conference devoted to all things AI. There was a strong emphasis on the commercial application, and it had a distinctly different feeling to PyLondinium a couple of days earlier. Everywhere you looked at CogX, people were pitching, networking or sitting on the sidelines making pitch slide decks on their laptops, it seemed. I went to some great talks:

• Haiyan Zhang talking about innovation at Microsoft’s Research lab in Cambridge
• Sarah Gold talking about safety and accountability in ‘learned systems’
• Zavain Dar got philosophical and talked about how AI and machine learning breakthroughs were reconfiguring how we talk and think about empiricism
• a really great panel on the intersection between AI and education, with a memorable set of contributions from Priya Lakhani.

There were many others. Then I wandered over to the ExCel exhibition centre for TechXLR8 which was quite disappointing.

(This week is Tech Week in London, if you haven’t noticed, so there are many more events and meetups than normal, it seems).

In the evening I went to see the Belcea Quartet perform Mozart’s String Quintet (K515), Shostakovich’s Eighth Quartet and Brahms’ Second String Quintet. I haven’t heard the Belcea Quartet before, but they’re really something. The lead violinist and cellist have a great rapport, and judging from the sound seem to be playing really wonderful sonorous instruments. It was a warmingly affirmative reminder — after all the talk of machines and technology — that humans still have something to contribute.

Tweeting to the Void

I've previously written about how I turned off Facebook's news feed. I keep an account with Facebook because people occasionally contact me there. It is also an unfortunate truth that many companies in Jordan (where I live) or in the wider Middle East only have representation on Facebook instead of their own website. (Why they insist on doing this baffles me and is perhaps a topic for a future post).

I have long preferred Twitter as a medium for filtering through or touching -- however obliquely -- things going on at any particular moment. I have no pretensions to actively follow every single tweet to pass through my feed. Rather, it's something I dip into every now and then.

Increasingly in recent months, I found myself growing dissatisfied with the pull it often has on me. It has become something of a truism to state that 'twitter isn't what it once was', but there's less and less long-term benefit in following discussions as and when they happen.

RescueTime tells me that I spent 86 hours and 16 minutes on Twitter in 2017 -- just under quarter of an hour each day. That feels like a lot to me.

Enter 'Tweet to the Void'. This is a Chrome extension. (For Firefox and other browsers, I have to imagine things like this exist.) When I visit twitter.com, the feed is not visible. All I see is somewhere to post a tweet if that's what I want to do. (There is still some value in posting blogposts and articles there, since I know some people don't use RSS). Of course, I can always turn off the extension with ease, but adding this extra step has effectively neutralised Twitter for me. 

Try it; see how you feel about having something standing in the way of your social media fix. Let me know how you get on.

Tabula for extracting table data from PDFs

Have you ever come across a PDF filled with useful data, but wanted to play around with that data yourself? In the past if I had that problem, I'd type the table out manually. This has some disadvantages:

• it is extremely boring
• it's likely that mistakes will get made, especially if the table is long and extends over several pages
• it takes a long time

I recently discovered a tool that solves this problem: Tabula. It works on Windows and Mac and is very easy and intuitive to use. Simply take your page of data:

View fullsize

A page listing Kandahar's provincial council election polling stations from a few years back. Note the use of English and Dari scripts. Tabula handles all this without problems.

Then import the file into Tabula's web interface. It's surprisingly good at autodetecting where tables and table borders are, but you can do it manually if need be:

Then check that the data has been correctly scraped, select formats for export (from CSV to JSON etc):

And there you have it, all your data in a CSV file ready for use in R or Python or just a simple Excel spreadsheet:

Note that even though the interface runs through a browser, none of your data touches external servers. All the processing and stripping of data from PDFs is done on your computer, and isn't sent for processing to cloud servers. This is a really nice feature and I'm glad they wrote the software this way.

I haven't had any problems using Tabula so far. It's a great time saver. Highly recommended.

ClozeMaster: learn words in context

Clozemaster is a service I discovered recently while researching the resource guide for my new book, Master Arabic. Clozemaster is intended to be a next step for students who have mastered some of the basics of their language of study. It teaches the language through a game where you identify the missing word. Here's what one of their tests look like, for example:

You can see that it has a space where a word is blocked out. Four options are available below, from which you have to choose. Of course, number two is the correct option and by clicking it you move onto the next question.

The sentences are taken from a wonderful open-source project called Tatoeba. Tatoeba is a collection of sentences and translations which are curated and gathered through crowd-sourcing.

Clozemaster hooks into the corpus of sentences available in Tatoeba and serves up tests ordered by how commonly the word is found (i.e. its frequency ranking).

If you're wondering why this is useful, it helps to know a little about how we best learn vocabulary. The gold standard of vocabulary acquisition happens through context. You learn the word when it is used in a real sentence or scenario, and where you derive all sorts of clues from the words around it. This is how we learn words when we're growing up, and it's how we continue to learn words and concepts in adulthood.

Clozemaster, therefore, is a great tool for someone who has mastered the basics of grammar in their language of study but who wants to grow their vocabulary further. At that stage, it doesn't help to learn words in isolation like services like Memrise have you doing. Instead, learn through reading, or learn through services like Clozemaster. The service is free and would be ideal for someone who's just finished Duolingo in a particular language (for example).

Developing for Android with Udacity

I was lucky to be awarded a scholarship to learn to code apps for Android phones. 10,000 such scholarships were awarded (from 70,000 applicants) to citizens of the European Union through a Google-Udacity partnership.

If you'd have asked me a few months ago whether I had an interest in learning to code apps for the Android platform (and learning the requisite Java code) I probably would have said it was low on my priority list. This scholarship allows me to learn the basics over the coming three months (with a possible extension of another three months as part of the full Udacity fasttrack nano degree, however, so I will certainly take the time and make use of the opportunity.

It isn't clear to me how far the course will take me, but I'm already thinking that this might be a really great opportunity to develop a version of my CoachBot language-learning tool for Android, hopefully one that doesn't require users to be online to use it.

The course so far has been fun. As usual, Udacity's platform and teaching style is highly interactive, iterates over problems and gets you solving practice questions from the start. As a language, Java is different from the Python and Javascript that I've encountered thus far, though I'm not deep enough into the weeds to have a strong appreciation of exactly how.

In any case, watch this space. If you're an Android user and would be interested in an app version of the CoachBot tool, drop me an email to let me know.

Introducing CoachBot: Your Personal Language Taskmaster

For languages that aren’t new, I often feel like I’m stagnating and get bored when I reach the intermediate levels. This can reflect a lack of materials from which to study (as was the case with Pashto when I first started studying it) or — more commonly — a surfeit of materials. This creates a kind of choice paralysis where the number of options means I’m far less likely to sit down and pick one of them. (In a similar way, I'll sometimes choose not to watch any of the in-flight entertainment because there are too many choices to pick from.)

Studying a brand new language is (almost) always fun: you’re making quick progress, everything is new so you still have that nice-and-shiny feeling, and you feel like you’re really on your way to success. Continuing that study after two to four years of effort is a little harder. Like with any longer-term project, you start having to find ways to remind yourself of why you’re even working on it in the first place. It can often feel like you’ve lost that original magic somehow, even to the extent that you question whether you actually want to learn the language.

It is useful to address some of these issues ahead of time. That way, when you hit a period of less energy or motivation, you have a pre-formulated plan of action (made by you when you weren’t consumed by whatever mood is dominant). For me, this takes the form of making lists of suggestions to my future-self. I have pre-made task lists for:

• When I’m travelling
• When I’m feeling sick
• When I have no time to study
• When I have oodles of time to study
• When I have lots of energy and enthusiasm for learning
• When I have no enthusiasm for learning

Try to have at least 10 or 15 tasks in whatever lists you do end up creating. Maybe save a few pages at the back of your language notebook to list these tasks. This way, you always have them handy. It helps to have a good amount of variety in the tasks you pre-assign to yourself.

I keep lists as described above, but they weren’t as effective as I'd hoped. I’d glance at the tasks, feel only a limited enthusiasm for the options available and then put the list to one side. I needed a different solution.

I happened to be teaching myself to program/code at around the same time, so I thought this might make an interesting practice problem to try to solve. (I was studying Python and so I found a way to make a web app that uses that to connect to Flask.)

CoachBot is the free tool I designed to solve the problem of study choice paralysis for language-learners. It’s still only a prototype, but I'm soft-launching it here now since I imagine it might help those reading who are in similar situations.

CoachBot gives you a task that you can complete within a specific time-frame. If you have only 5 minutes, it'll pick a random task from the database that I curated and wrote myself. Have an hour? It'll suggest a different kind of task. If you don't want to do a particular task that it suggests, just click a button to get a new one.

These are the kinds of tasks I suggest when working with students one-on-one. They’re also the kinds of tasks I had written down in my lists. As of writing, there are 386 unique tasks in the database, which means that the suggestions are far more varied and creative than anything I was previously using.

I’d suggest you use it as follows: if you ever feel like you don’t know what to do to keep going with your language studies, open up CoachBot, pick a time corresponding to your needs and do whatever it tells you to do. When you’re done, make a note of what you did and how long it took in your learning log. Consider doing another session.

I’ve been using this for a few weeks already and can attest to its value. One of the key benefits I’ve found is just in getting started. Sometimes I’ll only need to do a five-minute task before I realise that there was something else that I wanted to read or study and then I’ll get busy working on that.

There are lots of features that I hope to build in for future versions. I want to include user accounts and tracking of how much time you spend on the different tasks. I want to sub-divide by language skill (i.e. which skill is being trained) and eventually to build in some kind of guidance and interactivity to how the tool functions. But for now, use it as it is: get some studying done by outsourcing the choice of what you’ll be studying.

There are more details on the website itself. You can click through to the project’s roadmap where you can see an updated version of features coming soon. You can also make suggestions for tasks that you’d like included in the Bot and/or specific features you’d like me to build as part of the project.

[Special thanks to Alex, Ian, Kevin and Peter for patiently answering my questions while I was building this initial prototype].

Taskpaper --> Omnifocus

This is a neat trick for anyone who happens to use both Taskpaper and Omnifocus apps. I think this has probably been there for a long time, but I heard Gabe Weatherhead talking about it on the latest Mac Power Users podcast. (Read more of Gabe's writings on Taskpaper if you're interested in a deep dive).

I'll assume you know something about the Taskpaper syntax. This video explains more in case you're lost already. The Taskpaper syntax and .tp extension is useful because I can write lists of projects and things to do in Drafts on iOS without having to mess around with apps and ticking boxes and so on.

The trick which Gabe explained relates to getting hierarchically sorted task lists from Taskpaper into Omnifocus. It's as simple as copy-->paste. So I can go from something complicated in Taskpaper like this:

to a set of tasks like this in Omnifocus:

It's a great tip, and a great time-saver.

Pet Peeve: Tech Switching

I read a decent amount of tech media/press. Barely a day goes by when there isn't someone in my RSS feed explaining how they dropped application X for application Y. This seems to happen most often for frequently-used applications or workflows like scheduling/calendars or email.

I won't call out the specific blog post that set me writing this post, but suffice it to say that I wish there was a clause (in the contract of life) forcing tech writers or bloggers to state why the application they're singing the praises of is better than the one they were using up to now. Specifically, are there any new features, or does it just look shinier? Also, have you been using it for longer than a day or two?

I'm pretty solid and stable in the applications I use. It'll take something pretty seismic to rid me of DevonThink or Tinderbox or Mailmate. But if you catch me flip-flopping in my tech-related writing, please call me out on it.

Talking DevonThink with Gabe Weatherhead

I’ve been on a bit of a DevonThink kick these past weeks, and the catalyst for all of this was a conversation I had with Gabe Weatherhead (@macdrifter over on Twitter, though that account is no longer active).

You can listen to the full episode on your podcast player of choice or over on the Sources and Methods site. Towards the end of the episode we get into the weeds on how he uses DevonThink Pro Office and several other pieces of software. I’m looking forward to hearing Gabe’s much-anticipated appearance on MacPowerUsers in January, since I imagine he’ll go into even more detail there.

We also discussed social media and some of the ways he found himself drifting away from commonly-used sites like Facebook and Twitter. For me, this was the most interesting part of the podcast.

DevonThink Resurgent

There has never been a better time to get into DevonThink and Tinderbox. Winterfest 2016 is on, and you can get 25% reductions on both those apps, as well as a number of other really useful pieces of software like Scrivener, TaskPaper, Bookends, Scapple and PDFPen.

If you’re unsure if DevonThink is something you’d be interested in, they have a 150-hours-of-use free trial for all their different apps. MacPowerUsers podcast just released a useful overview of the current state of the app — an interview with Stuart Ingram. ScreenCastsOnline also published the first part of a trilogy of video learning materials on DevonThink.

If you’re a Mac user who is perhaps uncomfortable with Evernote’s privacy policies or just seeking to get more out of the data you’ve stored on your hard drive, give DevonThink a try.

Knot 3: Encryption, Race and Tunnels

More focus this week on finishing a big project, so less time for books or outside reading, but nevertheless some articles to recommend.

Articles

• Ursula Lindsey - “Funds Cut for Prominent Arabic Language Program” (Al-Fanar Media)

An extremely short-sighted decision. The CASA programme has been a really important source of support over its forty-nine years. This is a decision that affects American students of Arabic, but we shouldn’t forget that it’s symptomatic of how language studies are treated elsewhere.

• Rosalind Adams - “Intake: Locked On The Psych Ward” (Buzzfeed)

Anyone who still says Buzzfeed doesn’t do important reporting just isn’t reading. This piece looks at the way a for-profit system works to the disadvantage of patients’ welfare and the healthcare system in general. A must-read for an American audience, and for Europeans / Brits as a portent of what probably is headed our way.

• Bron Gondwana - “Why we don't offer PGP” (Fastmail Blog)

PGP has been much discussed (more than usual, at any rate) this past week. This post offers a useful overview of some of the things that PGP can help protect against, versus some of the places where trust in what it can do are misplaced. Pair this article with Bjarni Rúnar’s “Too Cool for PGP” to cover your bases.

• Annalee Newitz - “Evernote apologizes for its new privacy policy” (Ars Technica)

A useful corrective from Ars Technica (one of the best places to go for informed reporting on science and technology) about Evernote’s recent policy changes relating to privacy. The important point to take away from all this is that Evernote probably isn’t where you want to be storing your notes if you care about your privacy. Try DevonThink instead for a far saner approach to encryption and storage.

• Amal el-Mohtar - “'Iraq + 100' Is Painful, But Don't Look Away” (NPR)

This review of a new collection of stories seems (from the review, at any rate) like it is worth reading. Imaginative and creative responses to events on this scale are, I think, the kinds of things that will ultimately survive far longer than the political analysis and commentary that so far has dominated the discussion of what happened in Iraq. I look forward to reading this book.

• Ryan Holliday - “Want to Really Make America Great Again? Stop Reading the News.” (Observer)

You can ignore the focus on US politics and still take away the broader argument, that following the day-to-day news cycle is (for the broad majority of us) unnecessary and probably actually detrimental to our health and ability to do work that really matters (to us or others). A useful reminder.

• Laura Reiley - “Farm to Table: At Tampa Bay farm-to-table restaurants, you’re being fed fiction” (Tampa Bay Times)

A great piece of investigative work, Reiley shows how a lot of the promises of the farm-to-table movement in California are false claims. This piece is the first in a series that is worth reading in its entirety.

• Roger Cohen - “Broken Men in Paradise” (New York Times)

This is an important reminder of the realities of life for those refugees refused entry to Australia following harrowing journeys across the seas. Authorities in the places they are sent to operate with seemingly little restraint or care for their wellbeing.

• Ta-Nehisi Coates — “My President Was Black: A history of the first African American White House—and of what came next” (The Atlantic)

This piece seems to capture a moment. Based on several interviews with Obama as well as his usual command of the context that explains this moment, this is a (long) must-read.

• Gideon Lewis-Kraus - “The Great A.I. Awakening” (New York Times)

Important from perhaps a different angle, this is a long overview of the ways so-called artificial intelligence is poised to change our world. It views the transformation through the lens of developments in Google’s online translation service. It offers context and explanation to two different approaches to machine intelligence and the different algorithmic decisions being made. If you’re interested in technology and the ways that computers will continue to be embedded in our daily lives, you won’t want to miss this one.

Books

This week I read Lynne Kelly’s “The Memory Code: The Secrets of Stonehenge, Easter Island and Other Ancient Monuments” in preparation for an upcoming podcast episode interview. Dr Kelly makes a convincing case for how old archeological sites (like Stonehenge) were probably used as forms of memory aids to recall information important for local communities. The book is best accompanied by a glance at a truly fascinating series of memory experiments that she’s been working on. I also read Rebecca Solnit’s “Men Explain Things to Me”, a very useful and sobering reminder of the ways that society works against women, whether directly through violence or other perhaps less visible means.

Films / TV

I wrote yesterday about “Into Eternity” (trailer here), one of my favourite documentaries of the past year. It’s filmed with an idiosyncratic approach to detail and atmosphere. I am ignorant about where Finnish director Michael Madsen gets his influences from, but I certainly look forward to seeing more. Just today I saw another documentary, “13th”, which focuses on the ways that racial inequality has manifested itself in the United States, specifically in the prison system. This is a powerful statement, particularly for American viewers, I imagine, and it made me think about the ways that similar trends are to be found back in Europe.