Rosenblatt's Mark I Perceptron

in , , ,

I've now read a little about Rosenblatt's Perceptron in two different places: in the Howard/Gugger Deep Learning book, and also in Cade Metz' Genius Makers.

The Mark I Perceptron

Built in 1958, it is usually described as the first machine which was based on the principle of the artificial neutron. It used a single layer in this initial configuration, and even in that simple way you could already see glimpses of where it might go.

Unfortunately, Marvin Minsky and Seymour Papert's apparently perceptive but also damning assessment of the perceptron as a technology without a future ushered in the first of the so-called 'AI winters', and the idea of using neural networks was buried for several years.

Thankfully, some ignored the herd and stuck with it.

Deep Learning: Best in Show?

in , , ,

Deep Learning is an incredibly powerful technology and there are a number of (focused / specific) areas where it already surpasses human-level abilities. Here are some examples:

  1. Translation: If you haven't been watching closely, the quality of Google Translate translations has really been improved in recent years. This 2016 story is a little dated, but it explains how they made a big push a few years back and it continues to improve as the technology improves.
  2. X-ray interpretation: In a matter of a few years, the performance of Deep Learning in reading and making diagnoses from x-rays has surpassed top radiology practitioners. See how DeepMind raised the bar on identifying breast cancer.
  3. Playing Go: Watch the AlphaGo documentary if you haven't already.
  4. Protein Folding: Check out AlphaFold from last November, where DeepMind blasted through a notoriously complicated problem in biology.
  5. Colourising images: A former fast.ai student, Jason Antic, made great progress with his work on DeOldify.

The really great thing about the fastai course is how it successfully has managed to democratise Deep Learning as a technology. I always enjoy reading about niche areas where specific burning problems were solved because someone took the opportunity to educate themselves.

Removing Barriers: Deep Learning Edition

in , ,

I've been re-reading Jeremy Howard & Sylvain Gugger's Deep Learning for Coders with Fastai and PyTorch and I really appreciate the reminder that a lot of barriers to entry into the Deep Learning space can be productively put to one side.

Gatekeepers make four big claims:

  1. You need lots of maths to use Deep Learning to solve problems
  2. You need lots of data (think prodigious, Google-sized quantities) to use Deep Learning
  3. You need lots of expensive computers and custom hardware to use Deep Learning
  4. You need a PhD, preferably in Maths or Physics or some computation-heavy science

Needless to say, it's not that maths or more data or better hardware isn't maybe going to help or improve your experience. But to say that if you don't have those things then you shouldn't start is also (seemingly) inaccurate or not helpful.

If you are a domain expert in something that has nothing to do with Deep Learning or data science, you probably have a lot of problems that are like low-hanging fruit in terms of your ability to use powerful techniques like Deep Learning to solve them.

How to use jQuery and Handlebars in your website

in ,

jQuery and Handlebars are both external to the core functionality of JavaScript. Both are libraries that we can use and include when making websites. Doing so is very simple. We include <script>s in the head of our HTML file, as in the following example:

<html>
  <head>
    <script src="https://cdn.jsdelivr.net/npm/handlebars@latest/dist/handlebars.js"></script>
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
  </head>
  <body>
    <h1>Hello, World</h1>
  </body>
</html>

Note that jQuery is now modular, so you may want to consider whether you want to include the entire library. Above I chose to download it from Google's CDN, but there are other options listed here.

Using APIs to make things happen on the web

in

Towards the end of the Launch School core syllabus, we start to work with API calls. These allow us to pass information between servers. It turns out, this is really useful to be able to do.

Many if not most of the things you do online involve API calls. That little widget on the side of the web page that shows today's weather: an API call. Even things like Siri which aren't exactly web pages: using API calls.

Being able to make those calls and to interact with the services available through the internet gives users all sorts of power. The creativity comes, then, in how these things are all combined together.

That said, the dreams of the connected web have been somewhat oversold in the past. What are we on now, web 4.0?.

From a technical perspective, in this part of the course I enjoyed seeing how simple text in the form of JSON objects was behind so much of our communication and interactivity online these days. (All the more reason to have more secure ways of exchanging those plain text data).

I was also conscious of how much creativity and widened thinking has gone into expanding the possibilities of what HTML, CSS and a bit of JavaScript can do over the internet. Some of these capabilities mean that we're straining the possibilities in this area or that, but above all I take away some inspiration in how people made do with what they had instead of feeling like they needed to reinvent the wheel.

How events drive programming for the web

in

As part of my studies for Launch School, last month I was introduced to the idea of 'events' and how we can use the DOM and JavaScript to cause things to happen without having to force a full reload of the page. This is sometimes known as AJAX requests, which stands for Asynchronous JavaScript And XML.

In this mental model, we can think of the HTML and CSS code loaded by the browser (when you visit a website, for example) as the part which we can interact with. Absent any JavaScript code we include, this is somewhat static.

If we add in some JavaScript, we can first make sure that the DOM is fully loaded. For this we add an event listener for the DOMContentLoaded event.

Then after that, the sky is the limit. We can add event listeners to basically anything in our document (or just the whole document object itself). This is the list of events that we can watch for.

From reading outside the course, I know that companies' monitoring of user behaviour via JavaScript events triggering is a really big source of privacy violations or issues. I am not sure how the current model of how the web works serves to limit these violations. For instance, do people know that their mouse movements and clicks and patterns of mouse behaviour can all be monitored while they're on a site. So even if you don't click on an object, the website can still track that you're hovering over a particular element. That, to my mind, is a step too far, but it's the world we live in. You can disable JavaScript completely, but then you break a majority of sites that you're visiting.

Of course, if you're browsing in something like emacs' eww, or in brow.sh, then you bypass this problem somewhat.

Back to the JavaScript, though, the most interesting thing I found was the sheer amount of events that the browser was built to encounter. I'd love to see a chart showing how and when all these events were added to the browser's capabilities.

The Four Positions for Inserting Elements into the DOM

in

As part of my Launch School studies, I'm revising the ways we can use JavaScript to insert nodes into the DOM. These can be simple text nodes, or they could be elements, but the difficult part I've found is recalling the arguments for position. This is an argument you add into your method call which states where the node, for example, should be inserted. It looks something like this in action:

let newNode = document.createElement('p');
document.body.insertAdjacentElement('beforebegin', newNode);

Where you see beforebegin is where you can include one of four different text strings:

  • beforebegin — before the calling element
  • afterbegin — immediately inside the element, before its first child
  • beforeend — inside the element, just after its last child
  • afterend — after the element itself

The words chosen for these position arguments never felt fully clear to me, so writing it out has been useful to clarify their meaning. This is also a useful code excerpt, from the MDN docs:

<!-- beforebegin -->
<p>
  <!-- afterbegin -->
  foo
  <!-- beforeend -->
</p>
<!-- afterend -->

Different ways of accessing the text contents of DOM nodes in JavaScript

in

For a while now while studying the DOM and JavaScript's interaction with the web browser, I've been wondering about various properties available on nodes that relate to their contents or text values.

I spent a bit of time the other day unpacking what each of those do. This is a bit of code you can play around with to evaluate how they all work:

<!doctype html>
<html lang="en-US">
  <head>
    <title>title</title>
    <meta charset="UTF-8">
  </head>

  <body>
    <div>
      <h1>First bit of text.</h1>
      <p>Some <span>old</span> text</p>
      <textArea>pre-written</textArea>
    </div>
  <script>
    console.log(document.querySelector('div').innerHTML);
  </script>
  </body>
</html>
  • .textContent

This property concatenates the text of the element plus all child elements including any whitespace (from the HTML markup itself).

  • .data

This is the text content of a text node.

  • .nodeValue

For text nodes, this is the text content of that text node (i.e the same as .data). For comments, it is the comment content. For element nodes (and most other types), it is null.

  • .value

This is either an attribute property on an element (in which case the value is whatever was assigned to that property), or it is the contents of a textArea element.

  • .innerText

This is the rendered text (i.e. as displayed on the browser page) of the node and child nodes. Note, if it is not being rendered, then this will be identical to the .textContent property.

  • .innerHTML

This is all the HTML markup contained within the element (including nested elements and text (and whitespace). Note that sometimes this will just be plain text. Note too, that this can be used to create HTML inside an element as well.

Using CSS selectors with JavaScript DOM methods

in

I've been using JavaScript methods that interact with the DOM all week as part of my Launch School course. Among them, document.querySelector() and document.querySelectorAll() seem really useful. I realised I didn't fully understand the selectors that you were supposed to pass in as an argument, so I'm writing up some of what I discovered about them here. (See here for the documentation).

The simple part is when you have a single selector. Three important selectors are:

  • id selectors (#) — so if the html contains an id attribute of 'some-id', then you could write #some-id.
  • class selectors (.) — so if the class was 'special-style', then you can write .special-style.
  • tag selectors — for these, you just write the page itself

When combining tags, there is a complex set of options depending on whether things are siblings or descendants or children. For the most part that is TMI. The important ones to remember are:

  • descendant selector combinations — place a space between elements — so if you want to select all <div> tags that descend (however far down in the tree) from a <p> tag, then you can write p div.
  • child selector — place a > between the elements — this allows you to find elements that are direct children (i.e. no intermediary levels) of other elements. p > div will find all div elements that are the direct children of paragraph elements.

(For a more complete exploration of this topic, specifically the combination of selectors, read this blogpost.)

Turning array-like objects into arrays with JavaScript

in

I've long been wondering how the following piece of code works:

// assuming a website with a bunch of elements with 'h1' tags
let liveCollection = document.querySelectorAll('h1');
let arrayVersion = Array.prototype.slice.call(liveCollection); // returns that live collection of elements in the form of an array

So I thought I'd write up a little on my understanding of how it works.

Array.prototype.slice is simple. The slice() function is stored inside the prototype object property on the Array constructor. That's all as you might expect given how JavaScript handles things with the prototypal chain.

But why are we using call here, and how does that work when you pass in liveCollection as the object within which you want to invoke slice()?

Normally when we have an array that we want to call slice on, we have to do that using slice as a method. Arrays have the slice method available to them through the prototypal chain (also in the Array.prototype object). But our array-like object (i.e. the live collection in the example above) don't have those methods available to them.

Under the hood, when the slice method is invoked, what it does is iterates over the array as part of its more special functionality (which I'll ignore for now). If we have an array-like object with a length and with elements that you can sequentially iterate over, then we can run something like slice on that array-like object.

So how do we bring these two pieces together? We use call. call is a way of using a function inside a different execution context. In our case, we want to use the execution context (i.e. what this is set to) of the live collection, but still have it use the functionality defined in slice. call and apply are both ways we can do this.

(For more on this, there are some very useful explanations in this stackoverflow post.)

Understanding the use cases for Closures in JavaScript

in

I spent today revising some of my Launch School JS225 topics. Among those was the idea of closures, or in other words how functions can capture and package away any variables or 'state' in scope at the point of function definition.

I hadn't fully realised just how useful that can be. Other languages have other ways of realising the same functionality. In JavaScript, it turns out this is a really useful feature because it effectively allows you to keep some variables, functions and whatever else available to our objects or functions, but we can control access to them. They are accessible via whatever interfaces we provide, but to an outsider trying to access those elements via our API or function calls, they are effectively private.

There are a bunch of ways that this is useful in things that I don't yet understand and haven't studied yet (async or promises seem high up that list) but closures play a role in those as well.

I write all of this mainly because it's useful from time to time to step out of the weeds and remind oneself why any or all of this is actually useful.

What is Lexical Scope?

in

Scope defines what functions / variables and values are available at any point in a programme. Lexical scope is when you can derive the scope from looking at the code, or by looking at its position within the code. Lexical scope is also known as static scope.

The source code of our JavaScript code — in my example — defines the scope. It particularly relates to functions and nested functions. In those cases, we can create a hierarchy of scopes.

The inner functions have access to everything outside them, but the outer functions don't have access to everything from the inner functions.

Every function creates a new local variable scope. Every block also creates a new local variable scope. The code doesn't have to be executed for these scoping rules to be true, for these inner and outer scopes to exist.

This example below will log 'hello from inner scope' but will then give a ReferenceError since the variable secret is only available / accessible inside the inner scope function:

function outerScope() {
  function innerScope() {
    let secret = 'this is in inner scope';
    console.log('hello from inner scope'); 
  }

  innerScope();
  console.log(secret);
}

outerScope();