One of the essential programs for advanced physics calculations is on the verge of being abandoned. As the sole long-term maintainer reaches the age of 73, the fate of the calculation system FORM becomes uncertain. For over 30 years, this program has been regarded as one of the fundamental tools for particle physics research, capable of calculating gamma matrices, parallel computations, and pattern matching.
The software package FormCalc, used for calculating Feynman diagrams, is also built on this program.
It’s worth mentioning that Feynman diagrams describe the possible outcomes of particle collisions in large particle colliders through graphical representations, claiming to “help change the way physicists perceive the world.”
In addition to that, FORM is also used in the study of advanced topics such as QCD (Quantum Chromodynamics) beta functions and multiple zeta values (MZV) mathematical structures.
Even after several decades since its inception, numerous cutting-edge research still relies on it.
Read More
Since the year 2000, on average, there has been a publication of a particle physics paper based on FORM every few days.
However, such an important calculation program now solely depends on a retired 73-year-old individual, Jos Vermaseren, a Dutch particle physicist who is the program’s developer. Currently, as the elderly gentleman advances in age, no successor has been found.
And there doesn’t seem to be a program that can fully replace it yet, although Mathematica can also perform calculations, it simply cannot match its speed.
Solving Lengthy Equations through Disk Space
Simply put, FORM is a program that can perform symbolic computations.
Compared to the more general-purpose Mathematica, it focuses more on handling large-scale symbolic expressions.
But fundamentally, FORM is a mathematical algebraic system, and the specific operation process is roughly as follows:
In the function f, if any parameters appear before x, they will be swapped.
It has two main features:
First, it is fast in computation.
FORM has established several specialized algorithms, such as rapidly multiplying certain parts of Feynman diagrams; reducing multiplication and addition steps by rearranging equations.
Second, it can handle super large equations.
As long as the disk space is large enough, it can handle equations of any length.
This is precisely what makes FORM unique.
Computer storage can be divided into two types. One is primary storage, commonly referred to as RAM (random access memory). It is the most important storage in a computer system, capable of exchanging data directly with the CPU, allowing for fast read and write operations, but the storage space is extremely limited. The other type is secondary storage, such as hard disks, solid-state drives, and magnetic disks. Their advantage lies in their large capacity, but the computational speed is slow.
For example, a laptop may have only 16GB of memory, but the storage space can reach 2TB.
If one wants to solve super complex physical equations, they must rely on primary storage.
However, the problem is that with such a small capacity, it is impossible to handle lengthy equations. Moreover, FORM was born in the 1980s, when the primary storage capacity was even smaller.
FORM has chosen a clever approach—using the disk as primary storage.
By “paging” primary storage and disk space, the equations are placed on different “pages,” and each term is assigned a fixed storage location. During program execution, it can quickly locate the positions of various terms and bring them back to the actual primary storage without accessing other terms.
The benefit of doing so is that while expanding primary storage, it bypasses inefficient memory swapping operations, enabling fast computation of complex and massive equations.
Thanks to this characteristic, FORM has become one of the key tools in particle physics since its birth.
Even now, FORM remains crucial—after all, the development speed of computational storage can never catch up with the speed at which physics equations lengthen…
On the GitHub homepage of FORM, it is also stated:
FORM is an essential tool for many state-of-the-art calculations in high-energy physics.
on Form’s Github
It is worth mentioning that FORM cannot be regarded as an enhanced version of a computer algebra system (CAS); their program logics are different.
Although getting started with FORM may require some initial effort, once you cross that threshold, a whole new world will open up. By combining FORM and CAS, numerous complex mathematical and physical problems that are difficult to calculate can be solved.
Thomas Gehrmann, a professor at the University of Zurich, mentioned that most of the high-precision results his research group has achieved over the past 20 years heavily rely on FORM.
Matt von Hippel, an assistant professor of physics at the Niels Bohr Institute, also mentioned in an article on Quantamagazine that a colleague of his recently used FORM to push the precision of a calculation to new heights.
The Lack of Recognition Despite its Significance 🏗️
But contrary to expectations, this tool that has brought significant advancements to the field of high-energy physics has been quietly maintained throughout its development, to the point where the entire software is now at risk.
The beginning of FORM dates back to 1984 when the role of computers was rapidly changing, and PC machines were becoming popular during this period.
Its predecessor was a program called Schoonschip, created by Dutch physicist Martinus Veltman.
Unlike many commonly used computer programs today, the programs back then were mostly embedded in external ROM chips that had to be inserted into the computer to run (think of a CD) – Schoonschip was no exception.
Jos, however, wanted to create a program that was more accessible and could be downloaded by universities around the world.
At the inception of FORM, Jos used the FORTRAN language (which is also an important source for the name FORM), a language that is adept at “doing math.”
FORTRAN was developed by IBM for scientific and engineering applications and is considered the first-generation high-level programming language.
Since the 1950s, FORTRAN had become the preferred language for scientific and engineering calculations. From the late 1960s to the early 1970s, most high-performance computers supported FORTRAN, and many specialized compilers and tools were available for writing algorithms.
However, as computer technology advanced and other programming languages emerged one after another, FORTRAN gradually got replaced by languages like C, C++, Python, and Matlab because it lacked support for object-oriented programming and had cumbersome syntax.
In 1989, before the official release of FORM 1.0, Jos rewrote FORM in the C language.
But in fact, FORM had been gradually promoted and tested since its inception. From 1984 to 1986, the earliest support for FORM was on Apollo workstations, which were the first batch of graphic processing workstations in the 1980s.
By the early 1990s, over 200 institutions worldwide had downloaded FORM, and this number kept rising.
At the same time, FORM also developed three different versions:
- FORM: Sequential version designed to run on a single processor.
- ParFORM: Multiprocessor version with processors having their own memory, enabling the use of clusters and systems, supporting two or more processors simultaneously.
- TFORM: Multithreaded version with processors sharing a memory system, primarily used for systems with limited processor count.
FORM and FormCalc complement each other, with FORM being a general-purpose symbolic calculation and formula management software, while FormCalc is a tool specifically designed for high-energy physics research.
Looking at it this way, FORM seems to have been growing and thriving. But in reality, the total number of people maintaining FORM since its development has been only a dozen or so.
Now, there is only Jos, a 73-year-old, who is struggling alone to keep it going.
Why did it come to this?
One significant reason is that the efforts in developing programs within the field of physics are often undervalued.
Jos, with a hint of helplessness, expressed:
Over the years, I have seen people in the field of physics dedicating a significant amount of time to developing computational tools without securing a lifelong position. And compared to others, Jos and FORM can be considered fortunate because he himself has a lifelong position – as a long-term theoretical researcher at the Netherlands National Institute for Subatomic Physics (Nikhef).
Jos Vermaseren
If Form Truly Fails…
Moreover, FORM has also gained attention from the European Research Council (ERC).
In 2012 alone, ERC provided €1.7 million in funding for relevant projects of FORM. At that time, Jos proposed using the Monte Carlo method from the field of gaming to solve high-energy physics equations (which often require high precision and extensive computation).
And this is the third significant allocation of funds that ERC has granted to Jos and other Nikhef researchers.
However, in the field of particle physics, Italian physicist Stefano Laporta didn’t have the same luck. He also developed a useful simplified algorithm but didn’t receive much funding throughout his career…
Nowadays, even for FORM, which used to develop quite well, it has become challenging to find subsequent maintainers.
Because this not only consumes energy but often fails to yield high returns, it also requires developers to have excellent interdisciplinary skills. (One of the disciplines being particle physics…)
Some netizens pointed out:
The most challenging part is not writing the code, but ensuring that it processes the data correctly.
For example, it needs to effectively handle the reference book “PDG Review of Particle Physics” compiled by the International Committee on Particle Physics.
If subsequent maintenance is not carried out, FORM will quickly fall behind the pace of computer updates and become increasingly unusable.
Scholar Ben Ruijl has recently been trying to develop a new version of FORM called DreFORM using Rust to reduce bugs in FORM.
However, the new version has not been completed so far because Ben Ruijl had to put it aside due to his primary research tasks.
In response to this, Matt von Hippel, an assistant professor of particle physics at the University of Copenhagen (Niels Bohr’s alma mater), expressed concern:
(If FORM truly fails,) physicists may have to rely solely on Mathematica, which is several orders of magnitude slower than FORM.
Matt von Hippel
Particle physics may stagnate, and only a few individuals will be able to handle the most challenging computational tasks.
Currently, old man Jos is actively contemplating solutions. It is said that in April of this year, he will hold a FORM users summit to gather everyone for discussions on subsequent maintenance issues.
On GitHub, many users are continuously searching for bugs and providing feedback for this program.