Extensible programming

In computer science, extensible programming is a special way of writing programs. It helps people add new features to the programming language, the tools that turn code into working programs, and the system that runs the programs. This idea was very popular in the 1960s, but it didn't get much attention for a while. Now, in the 21st century, people are becoming interested in it again.

Historical movement

The idea of making programming languages easier to change and grow started in the 1960s. A paper by Douglas McIlroy in 1960 talked about using small pieces called macros to help with writing code. Another paper by Brooker and Morris the same year also described how to make compilers more flexible.

The most active time for this idea was in 1969 and 1971, with special meetings for researchers. But by 1975, it seemed like this way of thinking was fading away. One language, Forth, tried this idea but didn’t get much attention.

Character of the historical movement

These special programming languages usually had a basic set of tools and a way to change or add to those tools. Programs were made by changing the basic tools and then writing code with the new, changed tools.

The main way people changed these languages was by using macros. They also changed the rules of how words and symbols worked in the language, which led to something called adaptive grammar.

The Lisp community did not join this movement, even though Lisp could also be changed in similar ways.

At a meeting in 1969, Simula was shown as a language that could be changed and grown.

Standish, who wrote about this movement, described three ways to add new features to a language.

Paraphrase shows how to replace old parts with new ones, like using macros or defining new actions.
Orthophrase adds new features that couldn’t be made with the old language, like adding ways to read and write information. This is similar to adding extra tools that work with the basic language.
Metaphrase changes how the computer understands the old code, which is like a modern idea called reflective programming.

Death of the historical movement

Standish said the big problem with these changeable languages was that it became very hard to keep track of all the changes. Each new layer of changes needed to know about all the layers below it.

Because this got too complicated, a new idea called abstraction took over. This new idea focused on hiding the smaller details so programmers wouldn’t get lost in layers of changes.

Even though Simula was once seen as a changeable language, by 1975 it wasn’t really part of this movement anymore. By the late 1980s, macros were slowly accepted into the new abstraction idea, but under a different name: syntactic abstractions.

Modern movement

In modern extensible programming, systems include all the important features needed to let users add to and change the programming language.

Extensible syntax

This means the programming language can grow. You can add new words, ideas, and structures to the language. Some languages that let you add your own shapes and rules include Rocq, Racket, Camlp4, OpenC++, Seed7, Red, Rebol, and Felix.

Extensible compiler

An extensible compiler is not just one fixed program. It is made of pieces that help change source code into different kinds of outputs. Users can add their own steps to the compiler to change how the code works.

Extensible runtime

At the time a program runs, the system should let the language add new actions. For example, if the system uses a special set of instructions called byte-code, it should let users create new instructions.

Content separated from form

Programs should be seen as data to be worked on. How they look on a screen should be separate, so programs can be used in many ways.

Source language debugging support

Debugging tools should work with the original language's ideas, even after changes. They should show data in ways that match the language, like charts or diagrams.

Examples

Some programming languages that let you add new features or change how they work include: