JavaScript in the Browser¶

Unlike a standalone program, JavaScript in the browser is being run in tandem with the browser's own code.

When viewing a website, the browser is running code that is live-updating the user interface defined by the HTML and CSS. When things happen: a window is resized, a link is clicked, or any number of "user generated events" occur, the browser is primarily responsible for doing the right thing. The JavaScript that you write exists within this context. It primarily will be called by the browser in response to events: a button is clicked, a slider value is changed, an external API has returned results. It is also possible to ask JavaScript to call your code after a timer, a common technique used in animation to update the page on regular intervals.

Most of the challenge for developers who have written in languages like Python but are new to JavaScript is in understanding how programming within the browser differs from CLI/TUI/GUI programs they're more accustomed to. This article will discuss some of the peculiarities of working in this environment.

JavaScript in Context¶

JavaScript is a programming language introduced in 1995 as a feature of the dominant web browser of the era, Netscape.

At the time, a webpage was typically a straightforward document or form. Web pages were written entirely in static HTML, meaning that whatever loaded when you visited a URL was what you got— the arrangement of words & images on the page didn't change after the page was loaded from the server. Allowing programmers to write code that ran in the browser would mean that pages could have interactive elements; someone could dynamically alter the contents of a page, creating more interactive experiences like games and spreadsheets.

The original version of the language was written by one person, and it took 10 days. That is both an impressive feat and also an indication of the level of planning that went into that first version. In 1995 nobody foresaw what the web would become and the role that JavaScript would play. That is important because it results in a lot of the shortcomings we're left with 30 years later.

!!! tip JavaScript vs. Java

The name JavaScript was essentially a marketing term.
The language has no formal relationship to Java, which at the time was emerging as a popular and important language.
Java is closer to C++ or C# in semantics and purpose, a compiled language used for building standalone applications.
Java is a by-the-book object oriented language, and JavaScript is, as we will see, definitely not.

JavaScript borrowed heavily from more obscure origins, Self and Scheme.
It is from these that it gets two of its most unique characteristics: prototypal inheritance, and a wealth of [functional programming](https://en.wikipedia.org/wiki/Functional_programming) features.

Soon after Netscape introduced JavaScript, other browsers like Internet Explorer had their own versions (JScript). Without cooperation each web browser developed a unique implementation of the language. These implementations were more-or-less compatible, but each had its own quirks. While there was some core agreement, each browser would add custom methods and attributes to their DOM implementations and browser APIs.

Developing for the web requires making a choice:

stick to the standardized subset of the language that mostly works across all browsers
use features specific to a single browser, limiting your audience as a result
use libraries (sometimes called polyfills) that add unsupported features to browsers that lack them

This was particularly challenging in the early "Web 2.0" era when Chrome, Firefox, Safari, and Internet Explorer all had significant market share and rapidly evolving implementations which often diverged.

The need for this persists today, though it was reined in significantly when JavaScript became a standard, ECMA-262.

ECMA is a standards organization, the European Computer Manufacturers Association.

The name of this standardized version of the language is ECMAScript, which is now often abbreviated to ES. A large portion of the JavaScript community prefers ES as a name, and quite a few tools use it as a prefix (e.g. eslint, esbuild).

`<script>` tags¶

<script> tags typically live in the <head>, or occasionally at the very end of the <body> (helping ensure the entire page has loaded prior to their execution).

A <script> tag can either contain inline JavaScript:

<script>
function f() {
  console.log("Hello World");
}

f();
</script>

Or, can reference a remote JS file:

<script src="external.js"></script>

</script>

Script is not a void or self-closing element! In both forms, </script> must be present!

Regardless of whether the code is inline or external, there are two ways the browser can be told to interpret each separate <script>:

In the examples above, the JavaScript on a page is added to a global context. Similarly to how CSS works, the various JS files are appended together in the order they appear. This means two blocks on the page share variables:

<head>
  <script src="d3.min.js"></script>
  <script>
    const TOKEN = "abc123";

    function exampleFunc(s) {
       alert("Hello " + s + "!");
    }
  </script>
</head>
<body>
  ...
  <script>
    // a single shared context, even if this were in separate .js file
    exampleFunc(TOKEN);
    // d3 is available on the page, no import needed
    d3.select("body").style("background", "tomato");
  </script>
</body>

It is also possible to add type="module", which changes the semantics of the <script> tag. Each of these will have its own scope, similar to different .py files. It also enables the import/export statements and some other newer ES features:

<script type="module">
import * as d3 from "https://cdn.jsdelivr.net/npm/d3@7/+esm";
</script>
<script type="module" src="main.js"></script>

This is ECMAScript Module mode, or ESM.

While there are valid reasons to prefer either approach, it is recommended you don't mix the two styles on the same page as interoperability is complex and error prone.

Sandboxing¶

Of course, HTML is typically downloaded via HTTP. That means a web server sends your browser a chunk of HTML and your browser turns that into a webpage you can read & interact with. HTML alone poses a limited threat, it can only present content, but now executable code is being sent over the network and your computer is expected to execute it without even showing it to you first. What if someone sends the equivalent of os.system("rm -rf /")?

This should set off alarm bells!

To cut down on the risks here, JavaScript in the browser runs "sandboxed", meaning it can't access key parts of your operating system: your local disk is off limits. The browser can store things, and JavaScript may request to read or write to these, but in a much more limited way than being able to open any file on your drive the way the default Python runtime can.

DOM Manipulation¶

Instead of printing to a terminal, JavaScript programs manipulate the structure and content of the HTML on the page. This simple idea opens up a ton of possibilities, and the key innovation that drives it is referred to as the document object model or the DOM.

In the context of the browser, JavaScript code can always rely upon the existence of a global variable named document that represents the root of the tree. It exists as a global variable each JavaScript program can access and manipulate.

mindmap
  root((&lt;html&gt;))
    &lt;body&gt;
      &lt;div class=sidebar&gt;
        &lt;ul&gt;
            &lt;li&gt;
            &lt;li&gt;
      &lt;div class=content&gt;
        &lt;p&gt;
        &lt;img&gt;
        &lt;p&gt;
    &lt;head&gt;
        &lt;title&gt;
        &lt;meta&gt;
        &lt;script&gt;
        &lt;link type=text/css&gt;

HTML parses into a tree. (If you've done web scraping, you've navigated this tree via XPath or CSS selectors.)

The original DOM implementation did not support querying by CSS selectors. Instead you would use:

// iterate through the divs one at a time
document.getElementsByTagName("div");
 // navigate by position in tree
document.documentElement.firstChild.nextSibling.nextSibling;

The purpose of JavaScript in the browser is to modify what is on the page.

This means your code won't use print statements. Instead, you will typically be manipulating the HTML via the document object model, DOM.

document.getElementById - Get a single element, since IDs are supposed to be unique.
document.getElementsByClassName - Get list of elements with a given class in their classList.
querySelector/querySelectorAll - A powerful API for selecting elements based on CSS selectors.

Element¶

All of the above methods return collections of or instances of the Element type, which has the commonly used attributes:

elem.classList - List of classes on this element. (Already split, so class="one two" becomes classList=["one", "two"]).
elem.innerHTML - Directly access/modify HTML between start & end tag for this element.
elem.attributes - A dictionary-like object mapping attributes to their values.

See MDN: Element for the full list.

// grab elements by class name and modify their style
for (let e of document.getElementsByClassName("odd")) {
  // can set entire style
  e.style = "font-weight: bold;";
  // or per-property in a dictionary-like manner on .style
  e.style["color"] = "green";
}

Event Handlers¶

Any JavaScript that is outside a function in a bare <script> tag or .js will be executed. This code is typically limited to defining functions and setting up callbacks.

Callbacks are functions that we ask the browser to call when certain things happen. These functions receive an event object. The type and properties of this object vary depending on the kind of event:

For example:

function eventDebug(e) {
  // you can use a handler like this to help check when
  // event fires and what data is received by callback
  console.log("event fired =>", e.type, e.target, e);
}
document.querySelector("#the-button").addEventListener("click", eventDebug);
document.querySelector("#the-select").addEventListener("change", eventDebug);

In D3, this can be written as:

// one key difference, when using `on`, each element is limited to one handler per event type
d3.select("#the-button").on("click", eventDebug);
d3.select("#the-select").on("change", eventDebug)

Loading Data¶

In the early web, the only way to change the data on a page was to follow a link, reloading the entire page.

In 1999, Microsoft introduced an unofficial API known as XMLHttpRequest which allowed web pages to make additional HTTP requests from within a page without reloading. This additional data could then be used to update a small region of the page, instead of a full page reload. This was a relatively simple idea that revolutionized the web and laid the technical foundation for what we consider "Web 2.0". You may come across the term AJAX. This is a (now somewhat dated) term for asynchronous JavaScript and XML that was coined in the mid 2000s to describe this approach.

Modern web applications like GMail or Slack rely heavily upon this idea, to present something that feels more like an application than a web page.

A traditional HTTP request:

sequenceDiagram
    autonumber
    browser->>web server: GET /index.html HTTP/1.1
    web server->>browser: ... 200 OK ...
    browser->browser: render
    browser->>web server: POST /login HTTP/1.1
    web server->>browser: ... 200 OK ...
    browser->browser: re-render

Each request generates a new response, and the browser reloads the page each time.

Instead, with AJAX:

sequenceDiagram
    autonumber
    browser->>web server: GET /index.html HTTP/1.1
    web server->>browser: ...200 OK ...
    browser->browser: render
    browser->>web server: fetch(POST, /login, payload)
    web server->>browser: json | data={"success": ...}
    browser->browser: handleResponse(data)

While they achieve similar results, at step 4 things diverge. The request is sent via JS, in the background, instead of the browser URL bar, avoiding a page reload.

At step 5, JSON is returned, containing just the information that the site needs to update.

At step 6, the page is updated via a JS function handleResponse, instead of reloading the entire page.

`fetch`¶

Instead of waiting to execute like a call to requests.get in Python, the call returns immediately, returning something called a promise. This means if you want to use the data you need to ask JavaScript to call a callback function once the data is loaded:

fetch("example.csv")
  .then((response) => response.text())
  .then((text) => {
    // this lambda function runs once the content has been loaded
    // your code goes here
    const data = d3.csvParse(text);
    doSomethingWith(data);
  });

If you are using type=module, you can also use the await keyword at the top level (not merely inside functions) to have your code wait until a result is available before executing the next line:

const response = await fetch("example.csv");
const text = await response.text();
// now we can use `text` as we did above
const data = d3.csvParse(text);
doSomethingWith(data);

For more, see MDN: Using Fetch.

D3 Fetch¶

D3 also has an async form of its data loading methods: d3-fetch.

// await needs to be in a script type=module
const planetData = await d3.csv("example.csv");
d3.select("#planets")
  .selectAll("li.planet")
  .data(planetData)
  .join("li")
  .text((d) => `${d.planet} has ${d.moons} moons`);

If you aren't using modules, there is a callback-based version instead:

d3.csv("example.csv").then(
 (data) => {
    console.log("LOADED", data);
    // callback function executes once data is loaded
    d3.select("#planets2").selectAll("li.planet").data(data).join("li")
      .text((d) => `${d.planet} weighs as much as ${d.mass_earths} earths`);
 }
);

Web Components¶

Despite many other uses today, JavaScript development is still driven by the browser. The browser runtime has grown a lot of very useful features, notably a much improved replacement for the aging XMLHttpRequest in fetch, support for asynchronous functions, and lots of powerful new APIs to interact with 3D graphics, audio, location services, bluetooth peripherals, and much more.

Also, the browser dev tools, which you likely have used in web scraping, have integrated debuggers and other excellent tools right into the browser. This more than anything else has made writing frontend code less painful. If you'll allow a brief editorial: It is a pretty neat thing that every browser, not just some special "developer edition" comes with a full suite of developer tools. This is fundamental to the spirit of the web as a participatory medium, not different tools for different people.

Browsers are much more compatible than they were, with far more cooperation between the browser vendors. Of course, occasionally one of the big players like Google will attempt to force through unpopular changes that are advantageous to them.

Another recent development that's changing how JavaScript is written is Web Components. Web Components allow developers to author custom HTML elements with cross-browser behavior. That way instead of being constrained to the built-in form tags like <input>, <select>, <textarea> a developer can create their own interactive component and share it with others. Maybe a <usable-date-picker> instead of one that forces you to scroll from 1900 to your birth year for example.

See MDN: Web Components for more details.