May 19, 2023

Convert your Typescript codebase to No Unchecked Indexed Access

Introduction

Flux.ai was started around 3 years ago in TypeScript with the default compiler settings. If we could go back in time, there is one setting we would surely change: noUncheckedIndexedAccess. By default, this setting is false. Many people believe it should be true.

The flag

What does noUncheckedIndexedAccess do? By default, TypeScript assumes any array element or object property you access dynamically actually exists:

function uppercaseFirst(str: string) {
  return str[0].toUpperCase() + str.slice(1);
}

uppercaseFirst('') // runtime error BAD!!!

In the example above, the function will throw an error if the string is empty, because str[0] returns undefined and doesn't have a toUpperCase function. TypeScript doesn't warn you about that, regardless of whether strict mode is enabled or not. This is a huge hole in type safety.

The flag noUncheckedIndexedAccess will plug that hole and force you to deal with the possible undefined:

function uppercaseFirst(str: string) {
  return str[0]?.toUpperCase() + str.slice(1); // note: ? nullish operator
}

uppercaseFirst('') // returns ''

So, why can't we just turn on noUncheckedIndexedAccess? You can, but in a large codebase like that of Flux.ai, you are likely to get thousands of type errors. We had 2761 errors across 373 files! For one speedy engineer converting one file every minute, it would have taken 6+ hours of mind-numbing work to convert all 373 files.

The solution we describe here is how to smoothly convert your codebase with some simple heuristics and automation.

Heuristics

According to Wikipedia, a heuristic technique

is any approach to problem solving or self-discovery that employs a practical method that is not guaranteed to be optimal, perfect, or rational, but is nevertheless sufficient for reaching an immediate, short-term goal or approximation.

That is definitely true here.

The goal was to get the codebase compiling with the new flag, not to fix any bugs. The fixing can come later.

To that end, we intentionally added type assertions ! to suppress all new type errors from undefined types without changing the runtime behavior of the code.

const firstLetter = str[0] // needs too be str[0]!
const rest = str.slice(1)
const upperFirst = firstLetter.toUpperCase()

Expanding the scope of replacements to preceding lines allowed us then to automate more fixes with few false positives.

Automation

The full script we ran on our codebase is below. Note: it did not fix all the errors. It fixed around 2400 out of 2761 errors, leaving around 100 files for us to fix by hand.

Pro-tip: when experimenting with the replacers and precede, you can simply reset your changes with git reset --hard HEAD (assuming you are working in a git repo).

#!/usr/bin/env ts-node

// To generate noUncheckedIndexedAccess.txt, run
// $ npx tsc | grep 'error T' > noUncheckedIndexedAccess.txt

import {readFileSync, writeFileSync} from "fs";

type ErrorLines = {path: string; lineNum: number; message: string}[];

// NOTE: these should be idempotent for safety!
const replacers: [RegExp, string][] = [
    [/(\w+\.\w+\.\w+)\.(\w+)/g, "$1!.$2"], // a.b.c.d to a.b.c!.d
    [/(\w+\[(\w|\.)+\])!*/g, "$1!"], // add ! after []
    [/(\w+\])(\[\w+\])/g, "$1!$2"], // add ! between [][]
    [/(\[\w+\])(\.\w+)/g, "$1!$2"], // add ! between [] and .
    [/(\[\d\]?)!*/g, "$1!"], // add ! after [0]
    // START CORRECTIONS
    [/\]!\) =>/g, "]) =>"], // correcting add ! above
    [/\]! =/g, "] ="], // correcting add ! above
];

const precede = 2;

function main() {
    const txt = readFileSync("./noUncheckedIndexedAccess.txt", "utf-8");
    const errorLines = parseErrorLines(txt);
    errorLines.forEach((errorLine) => {
        let lineText = readLine("../" + errorLine.path, errorLine.lineNum, precede) as string;
        replacers.forEach(([match, replacement]) => {
            const newLineText = getNewLineText(lineText, match, replacement);
            if (newLineText) lineText = newLineText;
        });
        console.log("\n---");
        console.log(errorLine.path, errorLine.lineNum, "\n", lineText);
        console.log("---\n");
        writeLine("../" + errorLine.path, errorLine.lineNum, lineText, precede);
    });
}

function getNewLineText(lineText: string, match: RegExp, replacement: string) {
    return (
        lineText
            .split("\n")
            // @ts-ignore: ignore missing string method
            .map((line) => line.replaceAll(match, replacement))
            .join("\n")
    );
}

function parseErrorLines(txt: string): ErrorLines {
    return txt
        .split("\n")
        .filter(Boolean)
        .map((line) => {
            const [pathPlus, message] = line.split(": error ");
            const pieces = pathPlus?.split("(");
            if (!pieces || !pieces[0] || !pieces[1] || !message) {
                throw new Error(`Missing bits in line: ${line}`);
            }
            const numberPieces = pieces[1].split(",", 1);
            if (!numberPieces || !numberPieces[0]) {
                throw new Error(`Missing numbers in pieces: ${pieces}`);
            }
            const lineNum = parseInt(numberPieces[0], 10);
            if (!(lineNum > 0 && lineNum < 1000000)) {
                throw new Error(`Bad line number: ${lineNum}`);
            }
            return {
                path: pieces[0],
                lineNum,
                message,
            };
        });
}

function readLine(filename: string, lineNum: number, precede: number) {
    const lines = readFileSync(filename, "utf8").split("\n");
    return lines.slice(lineNum - 1 - precede, lineNum).join("\n");
}

function writeLine(filename: string, lineNum: number, lineText: string, precede: number) {
    const lines = readFileSync(filename, "utf8").split("\n");
    lines.splice(lineNum - 1 - precede, precede + 1, ...lineText.split("\n"));
    writeFileSync(filename, lines.join("\n"));
}

main();

Greg Dingle

Building the future with friends

Go 10x faster from idea to PCB

Work with Flux like an engineering intern—automating the grunt work, learning your standards, explaining its decisions, and checking in for feedback at key moments.

Try Flux for Free