Language Learning Crash Course: from slightly more than zero to slightly less than advanced

A colleague at work asked me for some tips on learning German and her situation and goals were fairly common so I thought I'd write up some notes here.

For the specific scenario under consideration, and to ground what follows: my colleague's mother tongue is not English, but she wants to learn German since she now lives there. She's interested in all the skills and wants to be able to read complex literature and philosophy, for example, but in the short-term building up spoken abilities is the main focus and intention. She knows a few words / phrases here and there but not too much more than that.

🪨 Context / Foundations

I'll start with some general context and foundations that I think should form the basis for what follows:

• If you're about to invest a bunch of time / effort (and probably money, either in the form of your time or for books/teachers) into learning a language you should make the investment in a spaced repetition system like Anki. It'll repay the effort you put in and is anyway a more efficient way to learn.
• Use focused feedback of various kinds to improve your ability to do things with your target language. This can take many forms, but it's a core principle that you should keep returning to. For example, this might mean feedback on pronunciation, or sentences you're writing, or even words that you're studying (in the form of active recall with Anki).
• Pick good quality materials to support your study. For some languages (popular / commonly-spoken languages) these will probably be cheap. For lower-resourced languages these are often eye-waveringly expensive. Bit the bullet. Good materials make a big difference, though it's worth taking advice from someone who's studied the same language as to whether a particular book is good or not.
• Stay laser focused in what you do / apply the 80:20 principle. With language learning, especially at the early stages, there's a fair amount of boilerplate work that just needs to be done (see most of the things in the first 'True Basics' section below) and there are strong reasons why you should do those things etc. Basically: stick with these things and don't deviate too much. (This only applies if you're trying to be as efficient as possible in your language learning. It's entirely within reason that you'd not need to be efficient, in which case many options open up!)
• Related: Be clear about what it means to level up. There are a million and one things that you could do at any single point during your language learning and it helps to be clear about what you're doing and why, then work on it, complete it and then reflect and move onto the next thing.
• Ingest comprehensible input where possible. This is a deep topic, but the basic idea here is that you should try as much as possible to read authentic materials that are almost completely within your capacity/level to understand. Ideally they should be interesting materials and you should be motivated to understand their contents. (Note that spoken conversations can be comprehensible input as well!)

These are the core foundations that I (with a half-dozen languages under my belt and experience learning bits of quite a few more, plus lots of time spent reading about and trying out language learning methods) believe are at the core of any efforts you should take. If any of the above are unclear, it can be useful to clarify (with me, or with some other source) but don't get too lost down the (deep) rabbit-hole of methodology as it will detract from your original goal.

All of what follows assumes that you're serious about your language studies and are willing to invest some time into it. To my mind, this means something like 30-60 minutes every day minimum, six days per week. You'll see the most progress early on and it can actually help to invest a little more time early on since seeing your fast progress will increase your motivation in turn, but only you know your circumstances and what works.

🍼 True Basics

This is the phase where you start from nothing and no knowledge to where you have learnt some of the most commonly used words and phrases as well as have a good understanding of the pronunciation and basic grammar. Depending on how much time you put in, this phase should take somewhere on the order of 2-5 weeks.

The things you should work your way to accomplish here include:

• learning some basic spoken phrases that you'll use frequently (think 'thank you', 'good bye', 'how much is …', counting from 0-100, 'what does … mean' and so on). You can get these (and the pronunciation for them) from many places on line or in a basic phrasebook.
• learn roughly 600 of the most frequently-used words for your target language. Many languages have something called a 'frequency dictionary' (or an equivalent list published online) which orders the words of that language in order of how common those words are. There are different approaches to exactly which list to pick (go with a Gabe Wyner 625 list if it exists (here's an extract/preview for German), otherwise go with the Routledge frequency lists, otherwise try to find something else!), but basically you are aiming to bootstrap your language abilities. The 'why' behind this is that first 500-100 words make up a really high % of the words that you'll see in any text (80+% depending on the difficulty of the language) so if you invest in those common words earlier you'll be in a good place to progress forward.

You'll want these materials to end up into Anki so you can continue to be tested on them so you don't forget them. Here, for example is a card I created for a word that means 'ripped / torn' in Arabic.

I get shown the picture and then need to come up with the word in Arabic for that concept. Note that I chose a quirky image on purpose.

• Learn the alphabet: (I'm thinking of languages like Russian / Arabic etc here, and not something like Japanese / Chinese). You're in this for the long-haul, and most comprehensible input that is efficient for you to ingest is in the form of written texts so you should just bite the bullet and learn the alphabet and how to pronounce it. It's not that hard! Even an alphabet with a bad reputation like Arabic can be learned in a weekend.
• Get a basic understanding of some core grammar. Good phrasebooks will usually have a section where they spend a few pages outlining how the language works with some basic grammar. (i.e. things like 'verbs always go at the end of sentences' or 'nouns take a bunch of different endings depending on the role a word has in the sentence', and so on.) You're not trying to master any of this, but you are trying to bootstrap your awareness of some of the core structures of how a language works.
• Try to listen to the language being used / spoken. If possible, try to make whatever you're listening to also something that you can mostly understand. (For some people this might be family or friends conversations, for others this might be other audio materials at a beginner level.) The goal here is to start to get the sound and feel for the rhythms of the language to bury its way into your ear.

How / What?: Basics

There are many (many) options at the total beginner level. My go-to sources / methods are some combination or selection of the following. It's really hard to give generic advice here so YMMV, but all these are suitable for this level:

• Michel Thomas course (if it exists for your language, do all courses / extensions that exist (usually 2 levels plus a vocab extension pack)). These are audio courses that will give you a solid basic foundation. As an alternative or sometimes additional option, you might want to try Pimsleur courses (again, if they exist). These are not cheap, but they're a good way to get going.
• Frequency dictionary and/or lists: Gabe Wyner's Fluent Forever app is based around the concept of frequency dictionary lists so if it supports your language then you can either use that app or just get the raw word list and create your own flashcards + study those basic words.
• Something like a Duolingo / Drops / Language Transfer course if it exists for your language. These are a double-edge sword since Duolingo by now has quite extensive courses available for popular languages. I wouldn't recommend using it as your core source of studies because the likelihood is that you'll drop away from your studies before completing it. That said, they do have mostly solid materials and they have spaced repetition (of some kind) built in so if you're someone who lives to complete things and can invest a bit of time into moving through the materials quickly, then these kinds of courses aren't the worst idea. (But definitely see them as a means to the wider end and not the core of what you do.) If you're someone who where possible prefers to study / learn away from technology / apps, try the excellent 'Assimil' books / course as the equivalent for this. (Note that this section potentially takes you very far off from the 2-5 weeks estimate I mentioned above. Only you know the extent to which you can afford to really invest in foundations.)
• Very easy readers. Again, if your language is popular then you might have access to these kinds of readers. Sometimes they're developed for children, but other times they're also available for adult students too. You're looking for something (it'll often be advertised as such) with a small / limited vocabulary size of words used. This is on the border with the next phase of study, so this can and will also be part of what you do next.

📚 Invest in sentences and your comprehensible input

At this point, somewhere in the middle of the basic phase and on your way to the next phase, you'll want to start investing into generating your own sentence-level comprehensible input. One way is to take the 600+ word frequency list and use those words as the basis of making basic sentences. Note that these can and will be really basic. You will likely want to do this in collaboration either with a teacher, or a friend or someone who can correct the things you're writing / producing. All these sentences and materials should then be fed into your spaced repetition system / Anki. For example, here's the back of a card where I'm being tested to produce a particular phrase that was missing (where the ... is in the sentence).

There's a picture above it to jog my memory if the sentence doesn't. And you can't hear it here, but there's also audio for the missing word / phrase. You'll want to be generating a lot of these kind of cards, not only because vocabulary learning in the context of actual sentences / real language is extremely powerful but also because it solidifies the root grammar structures, syntax and common ways of building language in your mind.

🚘 Middle Earth: Getting Independent

Some people actually stop here. If you're just going on holiday and you want to be able to be active using the language, understand some basic things and have some simple conversations, you actually don't need too much more. Everything you do from here on is because you are interested in levelling up and (eventually) reaching greater comprehension of a wider variety and level of materials.

This middle stage is where a lot of people get stuck. It's important to know that the intermediate plateau is a real thing and you will need strategies to not end up here forever (or quit, which will also eventually happen if you stay here for too long). (I wrote a whole book about strategies to get out of the intermediate plateau for Arabic, but the first section of the book applies to any other language as well. The book is available for free along with a bunch of audio interviews with language learning experts.)

The specifics of what you do in this middle stage is also heavily dependent on what your interests / motivations are for studying this language. In other words, are you mostly interested in getting really good at spoken conversations + listening, or do you actually not care about spoken language at all and mainly want to be able to read authentic materials without ever needing to to speak to anyone?

Core parts of this stage:

• continuing down the road of the frequency dictionary. Probably at the end of this stage you want to have mastered at least the 2000 most frequently used words, but ideally you'd get through most of the top 5000 most frequently used words. There are lots of ways that you can accomplish this, and it shouldn't just be a case of you rote memorising the words. Everything you do in this phase should be intertwined with comprehensible input of some kind, and vocabulary should always be learned within the context of authentic language but there are lots of ways of doing that so you'll have to figure out what applies best for your situation and needs.
• collocations / context: at this stage you want to be maximising your input to the core structures and ways your target language is used. Most words don't exist in a vacuum, and the further you get down the frequency list (or the more advanced your language source materials get) the more small bits of context and shifts in how individual words are used will make more difference. All of this is just me repeating that you should be reading and exposing yourself to lots of pieces of language.
• skill-dependent exercises and approaches: this is unfortunately vague, but depending on your interests and motivation you'll want to do one thing or another. For speaking, to give some examples, usually what I recommend is that you tackle a series of 'islands' that are defined by your interests. This means starting with a situation or a kind of conversation that you know you want to be really comfortable having. Perhaps you want to be able to talk about your daily routine, or maybe you want to be able to talk about food and the things you eat or cook. Then there are a series of things you'd do to prepare for mastering this 'island': gathering relevant vocabulary, thinking through the kinds of questions and/or answers that would come up during the conversation, generating some possible options or prepared answers or questions, practicing those with yourself and then later others, and so on. But really there are a million different ways to slice this depending on your interests and specific circumstances.

How / What?

What does this mean in specific? Some variation or combination of the following would be common:

• reading a lot in graded readers (hopefully these exist for your target language)
• graded listening: again, these hopefully exist for your language
• italki lessons or targeted spoken practice with a teacher (depending on what specific approaches you take)
• Clozemaster or Glossika: these are two tools that allow you to work your way through the most frequently used words using spaced repetition and context-rich language selections. Both also offer some kind of audio component. You might not need this if you have a solid method / rhythm already going with your use of Anki, but it's worth trying out at least to see if it clicks.
• writing sentences / paragraphs, then having them corrected, and then using the corrected versions of the language in Anki along with cloze deletion (a variation of what I showed above with the sentence and then a word or phrase missing that you have to guess) to practice the language used in a particular sentence.

😎 Onwards: Independent Study Land

At this point in your journey, you're at the B2 level and maybe soon to be C1. Your goal and process at this point is independent study: you're choosing materials and areas motivated by your interests. If you're still going + studying and you reached this level, the process never ends from here on out. It's just an ongoing set of little deep-dives into lots of different areas depending on your needs and interests.

Everything at this level is dependent on the skills you're working on, and your needs in terms of vocabulary / topic areas. Your vocabulary goals will be to move from the 5000 most frequent words on out into the next 5000-10,000 words after that, but usually it doesn't make sense to look at a literal frequency list for the words beyond 5000. For starters, they often don't exist, and your 'next words' will differ from person to person. You'll likely have a bunch of sub-areas that you get good at beyond the core 5000. For example, for me, I got really good at medical vocabulary (cardiology + neurology) in Dutch but I'm much weaker in other areas. The specific areas you spend time on will be driven by your needs and interests.

How / What?

Almost everything here is driven by specific needs, but it will involve and include:

• lots of learner-driven comprehensible input
• needs-driven activities (whether you're still focused on spoken prowess vs reading vs listening vs writing)
• open-ended practice like the kinds of tasks recommended by my (free) CoachBot self-study tool.

Going deeper

Most of what I wrote above was some concrete thoughts on paper that would serve as the basis for a conversation and I'm fully aware that lots of details were buried within or barely mentioned. Usually what I'd do after sketching out this overview is drill down into your individual situation and start to sketch out a more specific / precise plan, but this is a blog and I don't do coaching professionally any more!

Nevertheless, it's useful to be aware of the bigger picture and I hope that this blog served as a highly general overview that can drive further exploration and experiments in your own language-learning journey!

All the things I wish I knew about studying at school

My niece reached out to me a few days back asking about tips for studying at school. She was specifically interested in any ideas I had about how to excel in her maths studies. I wrote up my thoughts for her and it occurred to me yesterday that there might be some benefit from putting these notes online as well.

Without further ado, therefore…

Study Skills Advice

As I understand it, your question takes two parts:

1. how do I learn / study better
2. how do I specifically learn / study for my upcoming exams (in particular for mathematics)

I've split my response up into two parts since they're somehow related but also different. Part one ('how do I learn new things') is sort of required for part two ('what do I have to do to get good grades on my exams'), since it's pretty hard to try to cram ideas and concepts into your head if you don't first understand them.

1. Learning to Understand

Something you just have to 'learn'

First off, you'll want to learn the small tiny 'fact-like' things first. Some of this you just have to put in some work and learn them. It'll help you a LOT when it comes to getting a grasp on the bigger-picture concepts. To give an example, if there are certain equations or definitions of concepts, it can help to take a bit of time and just learn those by heart (in whatever way usually works for you for that). Note that this is limited to just small fact-like things. For sciences, maybe there are some facts or properties of something derived from the periodic table, etc, that are important. In the specific case of quadratics that you asked about, it might be simple things like:

• the usual form of a quadratic equation
• the quadratic formula
• 'what is a discriminant'
• what does the completed square form look like, and how do you solve it?
• what does this or that quadratic look like when you graph it out?

The key thing is to isolated and master those tiny morsels of information / facts early on, since a) it needs to be done anyway and b) it'll help you with the big-picture stuff. Sometimes textbooks help out by highlighting or bolding those key concepts. Just make sure you're not just trying to learn EVERYTHING by rote, since that advice is as good as no advice in my opinion!

Learning for understanding: foundations

Something else I'll say early on is that there are some core foundations that will serve you well:

• start with a / the problem — it will help to try to wrestle with the kinds of things you're expected to do with this particular topic you're studying. It can actually help you to do this at the very beginning of starting a new topic: before you even study things, take a look at the kinds of questions that you'll be tested on and expected to answer. Maybe even try to solve them. You'll learn a lot about the subject area just by rolling around in the mud and struggling a bit.
• curiosity — this almost goes without saying, but you should try to find a way to be curious about whatever you're meant to be studying. It'll help a LOT with your motivation and interest, which will in turn help you keep moving forward and give you energy at moments when things get hard. Sometimes the trick to this is just trying to ask questions about whatever it is you're studying: why is this important? how does it connect to topic X or Y we already studied? what is most surprising about this topic? Sometimes it can also help to know a tiny bit about the history of a topic. Calculus is a bit down the road in terms of your studies, but back in the day (late 19th century, early 20th century) there were really epic debates between the various people who were developing this new area of maths. Lots of drama and falling out between people. So sometimes just knowing a bit about the personalities behind things can help.
• growth mindset — I wrote a blog on this a while back but the core thing is just to believe that you have it in you to master this thing. Once you have that, and if you start from that place, then you'll have much of what you need to keep moving forward. And as one teacher once told me, as long as you never give up then you'll eventually master it. Sometimes you just have to be a bit patient. Most things come if you give them enough time :)

Retrieval Practice

This is sort of the core of most learning, in my opinion, and I have a lot of thoughts and practice around this. (Have read a LOT about what research recommends in this area, but I figure you're interested mostly in whatever is practical so I'll keep it mostly at that level.)

The core: if you want to learn something / master it, you have to retrieve it from your memory somehow. (Retrieve can mean lost of things: for German vocabulary it might mean knowing the gender or translation of a particular word. For Maths it might mean a particular formula, or even a high level understanding of how one concept relates to another.)

The important thing about this retrieval is that it will be and should be hard to do. (This is one reason why people don't necessarily enjoy doing this, and even fewer actually do it.) It's hard to struggle to put together a coherent explanation of topic x or y, but that struggle is what helps create neural pathways that cement the understanding going forward.

One possible way you could do this is to write short summaries of what you understood of a particular topic, then check your notes to see if you were correct. IMPORTANT: the key thing is to do this from memory / without notes. Otherwise you're not actually reinforcing materials in your head.

Another way of testing things (e.g. in your specific case of quadratics) is to do practice examples. Your school or textbook probably gives you some practice examples, but you shouldn't confine yourself just to doing those. Make up your own examples, or go online and find more examples (use Khan Academy or whatever).

It can also help to make distilled summary sheets at some point during your studies which gather together your understanding on a single piece of paper for the entire topic. Here actually you can see the one I made digitally for my own study of quadratics a few months ago:

The key thing with retrieval practice is to get a lot of it, and to try to make it the 'hard' / 'difficult' kind of retrieval. Mostly this starts with a blank sheet of paper and then you try to write down what you know about a topic, or a concept, or whatever specific thing you're trying to understand. Writing things down will help you realise (quickly! painfully!) which parts you don't actually understand. So it's as much to reveal to you which parts you need to work on as it is for anything else.

(There's a much-praised technique named after a well-known American scientist, Richard Feynman, and you can do something like this, too:

1. Write the title of a topic that you want to study / test yourself on
2. Write or map out an explanation of that subject intelligible / appropriate to a non-specialist. Do this from memory.
3. Identify any gaps in your explanation / understanding.
4. Relearn / restudy / interrogate to fill in the gaps.

You can use narrative / diagrams to condense and clarify your explanation. It's basically the same idea. And yes, bullet points or spider diagrams are all possible ways of doing this.)

Developing mental models

There's this idea that the whole thing you're doing when you learn something is developing 'mental models', which I personally find a bit hard to wrap my head around, but it is a thing… It's maybe the next layer up in what's happening when you try to learn something.

Mental models are, for me, about making a topic your own somehow. They're also about making the concepts of that topic manipulable somehow.

The 'making it your own' part has a lot to do with confidence, somehow, but it's also just feeling familiar and effective with the concepts that you feel comfortable solving problems in that area. If you see a problem, e.g., you know which techniques (or which subset of techniques) are needed to solve it. If there are multiple possible ways to solve something, you'll have a good feel for the tradeoffs: i.e. why this way is better than that way etc. In the case of quadratics, for example, we know that there is this amazing thing which is the quadratic formula, but you probably don't want to use that formula the whole time because it's easy to make a mistake with it and it's a bit cumbersome. Instead, we often use other simpler techniques that work for many (if not totally 100%) of the problems that you'll be exposed to.

One way to help develop mental models is to try to explain the topic to someone else. You already did a bit of this in the retrieval practice above: trying to explain it on paper is already some of this. But trying to explain a topic to people at different levels of understanding can be really clarifying. I.e. if you had to explain quadratics to a 5-year old it's probably different to how you'd explain it to a 40-year old. (Along with this, you can test out this approach by chatting with a chatbot about the topic. I'm sure you've heard of ChatGPT, but Claude is also another good option, esp for things like maths. The key thing is to start the conversation by saying something like "I would like to have a conversation about quadratics. I've been studying it and I'd like to test out my explanations of some core concepts with you. I would like you to tell me if things feel unclear about what I'm saying, or if you notice that there are some areas where I could improve my understanding.")

(While we're here, using things like ChatGPT to develop mental models can be useful. I will often have conversations that begin with something like "What is a good way to think about the discriminant in relation to quadratic equations? Please make your explanation simple to follow and use some concrete items in your reply, like only items that you'd find in a kitchen.")

Making mental models is hard! But the work you do to solidify things and make them your own is really worth it!

Anyway, the big point here is to reflect on what you're studying. Make sure to also give some time to connecting it to other things either in your maths studies or outside, or even life in general. It's not the best to just view everything completely isolated and disconnected from the other topics, so try to take a step back from time to time! (Unfortunately, most schools aren't built to encourage that process much, but it's important!)

Specific contexts

There are some other specific contexts that require different / more targeted advice, but you didn't mention them so I'll ignore them a bit. But language learning is one of them, and learning some kind of 'motor skill' is another (i.e. that requires coordination or physical movement like playing golf or the yoyo or whatever).

In Practice: Understanding Quadratics

To summarise the practical points listed above:

• learn the small fact-sized pieces early on
• get lots of retrieval practice (a mixture of examples of doing whatever the skill requires of you, and/or writing or explaining the topic at various levels)
• develop mental models where you can.

2. Studying for Exams

I'll take it for granted that you agree that you can't study for something without properly understanding it, so somehow the things in part 1 are sort of a prerequisite for this section; you can't get ready for an exam if you don't understand what's going on.

That said, there are some tactical things you can do to help your chances of success once you do have an understanding of a particular topic. Note that for all of this, it's a bit of a question of picking which parts seem doable / manageable. It's probably unwise / counterproductive to necessarily try to do EVERYTHING :)

Ground rules

You should understand the requirements of the exam. Take a bit of time to read through some previous exam papers. I'm sure your teachers have also given you clear guidelines on what kinds of things to expect. That will give you a map for how to prepare, so be sure to do this.

Foundations: Exam Study

There are some basic foundations here which for various reasons get forgotten when you're under exam pressure, but it's good to remind yourself of these, since if you neglect these it'll negatively affect your ability to study etc.

• sleep
• eating things that nourish your body instead of just feeding cravings
• taking breaks (every hour, ideally, get up and walk around for a minute or two)
• minimise distractions (put your phone in airplane mode or in a lock box while studying)
• movement in general / going out of the house for walks a few times a day is a good minimum.
• 'managing your energy' — this one's a bit hard to quantify / explain, but I'd say it's worth trying to embody the principle that you should only study as much today as allows you to keep studying tomorrow. I.e. if you overdo it and you study a lot today, but it's a bit too much and tomorrow then you can't do any study etc, then that was counterproductive. (Hope that was clear!)

Mnemonic / memory tricks

There are a TON of memory tricks out in the world. All of them are useful, but not all of them are equally useful for every situation :)

Things like the major system, the link system and the peg system are all useful, but they require a bit of time and probably also someone who knows how they work to explain them to you.

If you already have a bit of experience with these things, then I'd encourage you to use them in your studies, but if you don't have much experience then I'd say probably that it's not going to be the difference between an A and a B grade so probably it's a waste of your time to try to get into that in the run-up to exams.

That said, it would TOTALLY be a really useful investment to learn a few of these during summer holidays in a non-stress / fun way. You can play around with learning the order of decks of cards etc — I can explain all this if you're interested — and then you'll have that skill available to you if you need it next year or throughout your life.

There are some general memory principles that you can rely on in general terms:

• when trying to remember something, make it memorable in your mind! so maybe try to imagine the concepts as characters in some kind of image in your mind, and use all your senses and bring in some shock or drama etc etc. (LMK if you want more of these kinds of advice. I have a lot, but not sure how useful it is for you right now.)

Spaced Repetition

This is a really useful tool, but it requires a bit of upfront (time) investment and unless you're feeling super comfortable / not stressed at all, I might suggest to add it to the list of 'things to learn about over summer / winter (?) holidays'.

Basically this means testing yourself with (digital) flashcards, but the twist is that you only get shown the flashcard at exactly the optimum time / day when you need to be tested on it. (There's a whole science to this which I won't go into, but there's a TON of backing to the fact that this is the way to make things get into your memory.)

The best option for this is a piece of software called Anki. It runs on your laptop and phone etc, but it has a bit of a steep learning curve mainly because the defaults it comes with aren't great. So if you were interested I could help you set that up, but the key thing to know is that using this requires a bit of extra work.

The main idea is that you create (digital) flashcards for all the things you need to know, and then every day you check in with Anki to review whatever cards it says you need to know. There's an algorithm that calculates which cards you should review. It should mesh well with your intuitive sense of how memory works: i.e. over time you slowly forget things, so Anki will prompt you to recall a particular concept just at the point before you forget it, since that exact moment is the best time to review it. When you review it at that moment and you get it right, it'll really strengthen your memory for that thing. If you don't remember it, then it'll reset the status of that card and it'll know to show it more often for a few days etc.

There are more manual ways to get the same effect, but (for a lot of reasons) they're not as effective since humans don't work / behave like computers so really using a digital tool is the only way to go.

For quadratics, to use my experience, I have a bunch of cards relating to that that I get tested on. Writing a 'good flashcard' is a bit of an art, and we can get into that if you're interested, but I'll just lay it here as an option for now.

Interleaving

This is a fancy word for saying: 'don't study just one topic on its own'. When you're testing yourself on things that you'll need to know for exams, make sure to switch things up a lot. This means doing one problem from quadratics, then another relating to trigonometry, and another relating to topic z etc etc.

There is again quite a bit of evidence that this makes you much stronger in your understanding / learning, even though (or maybe because!) it's a bit harder to do.

If at least part of your review of topics / facts are handled by Anki it'll take care of giving you random flashcards anyway, so this more relates to things like solving maths problems by hand.

So don't just do 50 iterations of the same maths problem, in other words. Make sure you're switching topics etc.

3. Next Steps

• Check the practical suggestions above
• let me know if anything's unclear / or you want to know more about how to do thing x or y
• gather some problems to solve so you can make sure you're practicing the things you need to study
• get into some good habits around retrieval practice (i.e. writing things down to test whether you know them or not)

Automating social media posting for my new blogposts

I love blogging and I've benefitted a lot from what it's done for me ever since I started my first Geocities page in the mid 1990s. I maintain a technical blog at mlops.systems and a somewhat less technical blog at alexstrick.com/blog, though hope at some point to merge these together.

In the past I would have been content with ensuring that my blog published an RSS feed and known that anyone wanting to follow what I was writing could do so simply by connecting their feed reader and subscribing. I've become more conscious in recent years of a healthy brew of ambivalence, ignorance or even outright hostility to even the idea of RSS feeds and readers. It seems many people don't have RSS as an essential part of their informational hygiene any more. (I'll put my sadness / confusion about this to one side for now.)

And if I love blogging, I really dislike having to post my new blog posts to social media one by one, coming up with some catchy yet not overtly breathless summary of what I wrote, since this is apparently what many people use instead of RSS.

I've been grumbling under my breath about this situation for this for a few years now, but when ChatGPT came out it seemed like an obvious use: summarise my blogpost and repost to all my social media accounts taking into account their particular needs. (Mastodon uses hashtags more than the others, whereas LinkedIn posts can be a bit longer, vs Twitter which needs to be a bit shorter and so on.)

I held off, thinking I'd want to set up some system fully under my control involving serverless function calls and so on, but then I was reminded that I already use Zapier for some other administrative tasks. So this afternoon I set up and turned on some automation for social media posting to my Mastodon, Twitter and LinkedIn accounts. Posting happens at one step removed since I queue my posts in Buffer so that they go out at a time when people are more likely to see them. I apologise / don't apologise for this. My blog writings remain wholly un-automated; it would completely remove the point of 'learning through writing' if I were to automate the things that I blog about. My social media postings (just one post per blogpost so as not to spam you all) are from now on automated. As an additional courtesy / discourtesy, I've tweaked the prompt such that the social media posts should always read just slightly 'off' and will be labelled with an #automated hashtag.

Vermeer at the Rijksmuseum

I visited the bumper Vermeer exhibition today in Amsterdam. I had previously seen many of the works separately in different galleries in the USA and here in the Netherlands, but there's certainly a raw power to bringing them all together in the same space.

Above all, it was the still city landscapes from Delft that left the biggest impression. These small scenes captured so much and seemed to pull you in as if weighted down from within.

I'm glad I managed to get tickets to the exhibition, but will enjoy revisiting the two Delft paintings when they return to the Hague later in the year.

2022 Readings

I read 75 books this year (over 22,000 pages), and a few days still remain. Looking back over the full list, I'm both surprised at how many were only of middling reward. I think the key is to take one's sweet time on the true gems and speed through the should-have-been-a-blog-post dross. Out of the gems, the following stand out:

• Sabine Hossenfelder's Lost in Math: How Beauty Leads Physics Astray. We're told so often about how mathematics and physics is beautiful, how it makes sense and that finding those kinds of beauty-centric explanations is what we should be aiming for, so I was surprised and invigorated by a book about how the opposite might well be the case. It's a few years old and I'm sure it's part of a wider argument and debate in the field, but without knowing much more it certainly opened my mind.
• Emmanuel Ameisen's Building Machine Learning Powered Applications: Going from Idea to Product was one of the first things I read as I transitioned into a new field and I often return to it. Lots of hard-won wisdom about the 'whole game' of machine learning in the real world, and a story economically-told.
• I didn't read as much science fiction as I'd planned this year, but of the ones that I did, Samuel R. Delany's Babel-17 blew my mind. It's a classic for a reason, but still amazing to think that it was first published in 1966!
• For strong emotions and characters that keep you rooting for them, look no further than Shobha Rao's Girls Burn Brighter. Not a light read by any means, but a strong showing from start to finish.
• I devoured the eight currently-available parts of Martha Wells' Murderbot Diaries over the course of a few weeks. They're all fast-paced and once you've read the first the general conceit of the series wears off, but the character is so incredibly enjoyable. Come for the smart science fiction, stay for the strong character development. Definitely my favourite find of the year.
• Kate Crawford's Atlas of AI: Power, Politics and the Planetary Costs of Artificial Intelligence was a recommendation from a friend and it was a sharply sobering read in a discipline where ML boosterism is all too common. Nothing in there was particularly news, per se, but the picture as a whole raised important questions. Moreover, the balance between fine-grained detail and the big-picture worked well in this book.
• I reread Chanel Miller's Know My Name this year. Still as devastating and powerful as when I first read it, and a searing indictment of the criminal process for cases of sexual assault in the United Status. Required reading.
• Dostoevsky's Crime and Punishment, a late entry this year, was my first Dostoevsky and I'm still thinking about quite a few of the scenes, weeks after putting the book down. In particular, the last time Raskolnikov meets with his mother.
• Some books are unexpected gifts, like Sarah Polley's collection of stories-almost-not-told entitled Run Towards the Danger: Confrontations with a Body of Memory. Each essay is unexpected, brave and deftly constructed. I've watched most of what she's been involved in since The Sweet Hereafter and this book certainly didn't disappoint.
• Finally, I read through the two books written by David Goggins, Can't Hurt Me and Never Finished and something about his take on suffering and the lessons it has to teach resonates a lot with me. I'll be returning to these to sort through his experience at a later date.

On the interpretability of models

A common criticism of deep learning models is that they are 'black boxes'. You put data in one end as your inputs, the argument goes, and you get some predictions or results out the other end, but you have no idea why the model gave your those predictions.

This has something to do with how neural networks work: you often have many layers that are busy with the 'learning', and each successive layer may be able to interpret or recognise more features or greater levels of abstraction. In the above image, you can get a sense of how the earlier layers (on the left) are learning basic contour features and then these get abstracted together in more general face features and so on.

Some of this also has to do with the fact that when you train your model, you do so assuming that the model will be used on data that the model hasn't seen. In this (common) use case, it becomes a bit harder to say exactly why a certain prediction was made, though there are a lot of ways we can start to open up the black box.

FastAI Lesson Zero: video notes

[These are mainly notes for myself, based off Jeremy Howard’s ‘Lesson 0’ video that was recently posted. It doesn’t capture the entirety of what was said during the course, but it includes pieces that I felt are relevant to me now and that might be relevant to me in the future.]

• decide when you’re studying
  • be precise about how much time you’re going to spend
  • think about how it’s going to fit into your day or your life
  • give yourself deadlines and goals, perhaps, but also don’t worry if disruptions happen.
  • Mainly make sure that if something does come up, make sure you get back on the horse and keep going. (Tenacity counts for a lot)
• Finish. The. Course.
  • make a commitment to complete the course, and make sure you actually do that.
  • If you’re attending the course and working through it, you should follow through on your original commitment and actually work through the course.
• Finish a Project
  • build a project and make it really great.
  • You’ll probably have several projects here and there that you work on during the course of the fastai course, but at a minimum make sure you pick one of those and make it really great.
  • (It doesn’t have to be unique or world-changing. Even replicating something that’s already in existence can still be worth it).
• Find good role models
  • Jeremy raises up the example of Radek Osmulski (who recently published an ebook called Meta Learning).
  • (Jeremy himself is a good role model too).
• Learn by doing. Use what you learn and figure out the rest as you go. (Don’t get paralyzed by trying to learn ‘pre-requisites’ like complex mathematics topics, esp since most of them aren’t actually needed to become a practitioner).
• Share and communicate your work
  • (Jeremy doesn’t mention the book, but I’ll insert here that the book “Show Your Work” by Austin Kleon is a great starter on this point).
  • If you consistently blog during your studies, at the end of it you’ll likely have a huge collection of artefacts of that study, showing what you’ve learned and accomplished.
  • Alongside that, being a good citizen and contributing in the forums etc is also a really solid way to extend whatever knowledge you have to others, and quite possibly cement things in your own mind as you reply.
• How to do a lesson
  • watch the video / read the chapter
  • Run the notebook & experiment — play around with things + make sure you actually understand what’s happening
  • Reproduce the notebook from scratch — (and really start with nothing here, and try to reimplement whatever was done during the lesson. From previous experience, this work will be hard, but it’s super worth it. Recall learning is the best kind of learning)
  • Repeat with a different dataset — use the techniques you learned in the course on a dataset of your own / or solve some related problem using these techniques
• Using a notebook server vs a full linux server
  • the notebook server allows you to get going much faster
  • A full linux server is more ‘your own’ and you get to also practice a bunch of other not-specifically-deep-learning skills along the way
  • With the fastsetup library, Jeremy has made getting going with an EC2 instance pretty easy.
  • the video spends a fair amount of time showing how to do this with Colab Noteboks and a AWS EC2 instance. Refer to the FastAI website and the full video for more details.
• Get better as a developer
  • just doing the course, you’ll also work on your development skills along the way
  • Two important things to do to help with this:
    • Read a lot of code
    • Write a lot of code
• Start with a simple baseline & get a basic end-to-end solution up and running
  • When you’re working on a project, it’s a really good idea to start with a naive / super-basic baseline so that you know whether you’re making progress or whether you’re achieving anything with the work you’re doing.
  • Successful ML projects that Jeremy has seen start with the simplest possible end-to-end solution and then incrementally grow from there.
  • The work of getting your pipeline working / your data imported etc will take a bit of time, and if you get that all sorted upfront it’ll help you focus on the actual work you want to be focused on.
• (At some point during the course) join a Kaggle competition and make a serious attempt to do your best
  • just getting a model on the leaderboard tests your knowledge and your skills
  • just work regularly on things, show up every day, try to make your model a little better each day
  • Do these things:

• For getting a job in the space
  • having a public-facing portfolio of writings and projects will take you a really long way
  • Some companies are more interested in people having the right credentials etc and will never choose you.
  • Startups are a great place where this matters less.
• Try to take the second course
  • The first course gets you going as a practitioner of deep learning, but the second course allows you to implement algorithms and models from scratch and digs far more into the depths of the subject.
  • Jeremy wishes more people would take part two + encourages them to do so.
• The fastsetup library is great for installing everything on a Ubuntu machine (like an AWS EC2 instance)
• Experiment tracking software
  • The two big players are TensorBoard and Weights & Biases.
  • Jeremy doesn’t use these. Finds it too tempting to spend your time watching your models train instead of doing something else that is probably more valuable.
  • There are some cases where it might help to use this software.
  • Weights & Biases seems like a good company to work for & they’ve hired FastAI grads in the past.

Arthur Samuel and the 'Frontier of Automation'

The use of neural networks / architectures is a powerful pattern, but it's worth remembering that this pattern is part of the broader category of machine learning. (You can think of 'deep learning' as a rebranding of neural networks or what was once more commonly referred to as connectionism).

In a classic essay published in 1962, an IBM researcher called Arthur Samuel proposed a way to have computers 'learn', a different process from how we normally code things up imperatively (see my previous post for more on this):

"Suppose we arrange for some automatic means of testing the effectiveness of any current weight assignment in terms of actual performance and provide a mechanism for altering the weight assignment so as to maximise the performance. We need not go into the details of such a procedure to see that it could be made entirely automatic and to see that a machine so programmed would "learn" from its experience"

Within this essay and this quote specifically, we can find some of the key building blocks of machine learning:

We have our inputs (our data) and our weights. Our weights (or the weight assignments) are variables that allow for different configurations and behaviours of our model. Our results are what the computer has assumed based on the weights and the model, and we have some kind of a metric (our performance) to judge whether this model was accurate or not. The computer then updates the weights based on that performance, tweaking it such that it tries to get better performance.

This is a slightly amended version which language or jargon that are more commonly found today. As you might expect would happen, the language used in the 1960s is in many cases different from what gets used today:

The main difference here is that we have some labels which are used to know whether the predictions are correct or not. The loss is a way of measuring the performance of our model that is suited for updating our parameters (that used to be referred to as weights).

Telling Cats from Dogs

One of the main ways that using neural networks to train models is different from traditional (imperative) programming can be illustrated with a specific task: let's say you want to use a computer to tell you whether any particular photo you give it is a cat or a dog.

An imperative approach might be to make a mega list of certain kinds of features that cats and dogs have, and try to encode the differences into some kind of list of logical features. But even knowing how to tell the computer how it should recognise those features is a potentially massive project. How would you even go about that?

Instead, with neural network layers, we turn that pattern on its side. Instead of this:

We have something closer to this:

So the neural networks are learning on the basis of data provided to it — you give it a bunch of images which you've pre-labelled to say 'this one is a cat and that one is a dog' and so on.

If you use transfer learning, too, you even can use a pretrained model (which is already pretty good at recognising features from images). You can then fine-tune that model to get really good at the specific task you need it to do. (Note, that's exactly what you do in the first chapter of Howard/Gugger's Deep Learning for Coders).

Getting Out of the Intermediate Language Plateau: Arabic Edition / Principles

[This is part of a series on getting out of a language-learning plateau at the intermediate-advanced level. Check out the other parts here.]

Seasoned language learners are familiar with the concept of the 'language plateau'. If you're learning a second language for the first time, you will inevitably reach a point in your studies where your progress seems to flatten. You will find this place and period extremely frustrating.

When you are in your plateau, it's hard to improve because you're already at a point of (some kind of) self-sufficiency. You can express yourself. You understand most of what is going on in a conversation or TV series you watch. You can write things and people will understand what you're saying. You could (and many do) stop your studies at this point and still be 'functional' in the language.

Getting out of this flat, dead zone is what I want to talk about today. It's hard, but it's by no means impossible, and making this kind of progress is possibly the most valuable work you'll do in your language studies, because all of it will be specifically tailored to your needs.

The starting point, though, is to identify your current status. What can you do? You don't (necessarily) need to take a formal language certification test to get a grade, though that can sometimes be useful. The kind of measurements you want to take are more subjective. You want to take stock of your capacity in certain situations, what level you are able to achieve in different contexts (your skills in reading will be different from writing vs listening or speaking, for example) and you want also to assess your experience on the cultural level as well -- i.e. how much experience do you have navigating all the unspoken parts of culture, whether that is body language, or behaviours and so on.

Principles of Skill Acquisition

Now a slight detour into some more general principles of skill acquisition. Some of this is derived from my own personal experience, other parts from interviews with experts in this field (such as my conversation with K. Anders Ericsson, who more or less invented the field of expert performance studies), and other parts still from reading a bunch of books on the subject.

Three things are relevant here:

1) Stretch

When you're learning a new skill, you want to step outside your comfort zone. This is usually difficult work, and work that is mentally (and possibly emotionally) taxing. Thus, if you want to get better at speaking in Arabic, you'll need to speak more, but at the beginning this practice (i.e. talking with other people) will feel pretty horrible, simply because you're not used to doing it. It's a paradox that you need to do the thing to get better at doing the thing. It is this difficulty, pushing yourself a little past what you're capable of doing, that allows for personal growth. (I wrote about this in an entirely different context a few weeks ago with respect to my attempts to get better at climbing.)

2) Lots of practice coupled with speedy feedback

These two parts (practice and feedback) go together. It isn't practice alone that will allow you to improve, but rather the combination of making efforts to use new skills alongside getting some kind of feedback that tells you when you're getting it wrong vs when you're not. An implication of this, too, is the reality that this kind of practice is going to involve you making lots of mistakes. This can feel crappy, especially when you're getting immediate feedback on exactly when this is happening. You need to adopt a flexible mindset, if possible, in which you see the mistakes as indicators of growth rather than as any kind of personal or intellectual failures on your part.

3) Know what you're practicing and focus on that

This is basically Ericsson's principle of "deliberate practice":

"Rather than chilling out in the comfort of skills you've already acquired, as an expert-to-be, you're relentless about heading to the frontier of your abilities. The practice shouldn't be so difficult that it overwhelms you—that would be depressingly demotivating, but not so easy that you're unconsciously languishing. In other words, you're arranging for flow, that space where you're right at the boundary of your abilities."

See also this summary of the routines that 'experts' tend to have around deliberate practice:

They can only engage in practice without rest for around an hour.
They practice in the morning with a fresh mind.
They practice the same amount every day, including on weekends.
They only have four to five hours of deliberate practice a day.
If they don't get enough rest, they can get overtraining injuries or burnout.

If you're hoping that 'using the language' in a general and non-specific way will get you out of your plateau, you'll be disappointed. It's perfectly possible to exist in the plateau zone without improvement ad infinitum. If you want to improve at a certain skill, you'll need to isolate that element and focus on it in a targeted way. This can be vocabulary, or speaking about a certain topic, or even something as small as 'using conditional sentences'. Whatever it is, you'll only get better if you concentrate your efforts.

Customisation & Your Individual Needs

Learning languages at the post-intermediate level will be a different experience from what you are used to in the early stages. Early on, you're doing a great deal of necessary-but-boring work to learn basic patterns, vocabulary and grammar.

Once you have mastered that, and you can explain yourself in most basic contexts, you reach the point where you have to customise. There's a great deal of science and research behind this claim. Check out this talk, by the always stimulating Alexander Arguelles, for an overview of some of that research.

You'll need to pick which areas you're most interested in. This is the hard work of advanced language studies -- you pick one area or context, conquer it, and then pick another area and repeat. This fulfils the princicle of focus that I mentioned above.

To give an example from my own studies. My current big push for Arabic is to be able to read serious fiction (i.e. short stories and novels written for native speakers). I've written previously that this was a personal goal, but various realities of how modern literature is written really make it hard to take the leap into complex native-reader-level fiction (especially novels). Arab writers like to use many synonyms (for poetic effect, or perhaps as an attempt at pretension?) for words, so when reading I often find myself stuck referring to dictionaries the whole time. Fortunately, a new textbook offering graded literature at just that 'stretch' level was released recently, which is allowing me an entry point into that world. None of the texts are simplified, and the language is hard and the number of unknown words is pretty large, but it's not too far down the scale of difficulty.

On Making a Self-Study Plan

My next post will cover and offer a host of suggestions for resources you can use to get out of this plateau / dead zone. Before you start reading through and diving into things that seem interesting, I'd strongly advise you take the time to figure out your specific goals. "Improve my Arabic" is not a useful goal. It's too unspecific. Even "improve my spoken Arabic" may not be particularly useful at the intermediate-advanced level. Once you figure out your goal, write it down somewhere. Maybe stick it to your wall or on the inside of your notebook. It's good to be reminded why we're doing the work.

Once you have your goal, then you want to set yourself small targeted bursts or challenges to push out into your stretch zone. You don't want these challenges to feel like you're straining against the limits of what you are capable. You want it to be just challenging enough that you feel uncomfortable, but not so much that you are constantly questioning yourself and your abilities in any kind of fundamental sense.

The scale of these challenges will be pretty variable, so examples will span a range of tasks from taking a week to learn and read deeply in a niche topic, to something more longer-term (over six months, perhaps) like my modern literature challenge. The characteristic that you need to look for, however, is that you'll be able to tell when you're finished with the challenge. Part of defining the goal is finding a specific (and somewhat measurable) definition of what it means to have achieved what you want.

Then the rest of the trick is basically keeping moving, tracking your progress and achievements along the way. There are various ways of doing this, some of which will depend on what else you have done in this regard. You can add in things like Beeminder to encourage compliance and regularity, or you can do that in other ways.

When I work with people 1-on-1 to learn a language, a lot of what we do is figuring out this kind of ongoing goal setting and progress assessment. (If you want to learn more about this, click here and read through what I offer).

The next posts will offer a roadmap to the different resources available to the intermediate student of Arabic and some of the ways you can utilise these resources. It won't be exhaustive, but I'm pretty sure that most will find something of use in them. Feel free to get in touch if you have specific things you want me to tackle in terms of skill development in Arabic.

Small, unexpectedly powerful boxes

Graphics Processing Units or GPUs are what your computer uses to quickly display your screen. Most computers (desktop or laptop) have one of these, and they are used to good effect to keep the screen refreshed and display everything in effectively realtime speed. The world of gaming is also, perhaps unsurprisingly, quite dependent on fast GPU performance, with Nvidia as the lead provider of these hardware units.

It was discovered a while back that GPUs are also pretty great at performing certain kinds of computation at incredible speed. Certain calculations which, if you would do them on a standard CPU, would take ages to complete are much faster when run on a GPU. For this reason, they're the hardware of choice for training deep learning models.

GPUs also happen to be heavily used (for similar reasons) for cryptocurrency mining and accordingly there has been a worldwide shortage for some time. Between the crypto bros and the deep learning practitioners, the price got inflated for a while. Nvidia has made some attempts to limit crypto miners from using their hardware, but to inconclusive effect.

Held back by misunderstanding

The field of deep learning seems to have had a rough journey into public consciousness and adoption. In particular, two theoretical misunderstandings lead to funding being pulled and energy and attention moving away from the field:

1. Minsky/Papert's book Perceptrons showed how a neural network using only one layer was unable to learn some critical functions like XOR. Later in the same book, they show how using more layers addresses this problem completely, but for some reason the 'fix' to the problem was ignored and people fixated on the problem with using a single layer and its drawbacks.
2. By the 1980s, many people were using two layers in their neural networks, and while this did solve the problems identified in 'Perceptrons' and people were using neural networks to solve real problems, it was unwieldy in that form. Yes, you could theoretically approximate any mathematical function with two layers, but it was impractical and slow to do so. People thought that this meant that the principle was broken, whereas really the misunderstanding was that two layers were just not enough and that the number of layers could continue to increase.

These are two key misunderstandings identified by the Howard/Gugger short introduction and I'm sure I'll read more of these in Genius Makers. It's amazing, but not entirely surprising, that a non-generous and unimaginative misreading of the literature could be responsible for such an effective trashing of a research path.

PDP: a precursor to modern neural networks?

Parallel Distributed Processing: Explorations in the Microstructure of Cognition, a multi-volume publication by David Rumelhart, James McClelland and the PDP Research Group, was released in 1968 and is recognised as one of the most important works relating to neural networks.

They lay out eight features necessary to perform what they called 'parallel distributed processing' (which I suppose you can think of as a sort of precursor to modern-day deep learning):

• processing units
• a state of activation
• an output function for each processing unit
• a pattern of connectivity among units
• a propagation rule (for propagating what is learned through the network)
• an activation rule
• a learning rule (where 'patterns of connectivity are modified by experience')
• an environment in which the system operates

I haven't read the book, and I don't fully understand all these different pieces, but it isn't particularly hard to see the pattern of what would later come to be handled by modern-day neural networks in these features. The vocabulary used to describe it is slightly different, but you have the connectivity between neurons, and you have a process through which you update the layers…

This feels like a book that would reward returning to for a proper in-depth read later on in my studies.