In the realm of mathematical sciences, where complexity and challenges often seem insurmountable, the advent of Large Language Models (LLMs) has brought about a transformative tool—FunSearch. This groundbreaking method, introduced in a Nature paper, seeks not only to solve complex problems in mathematics and computer science but to discover entirely new knowledge. Today, we delve into the mechanics of FunSearch, exploring how it utilizes the creative capabilities of LLMs to drive scientific discovery.

The Challenge of Creative Problem-Solving with LLMs:

While LLMs excel at combining concepts and providing solutions, they face a significant challenge when it comes to making verifiably correct discoveries. The tendency to "hallucinate" factually incorrect information poses a hurdle. FunSearch aims to overcome this challenge by identifying and building upon the very best ideas generated by LLMs.

FunSearch Mechanics:

FunSearch operates by pairing a pre-trained LLM with an automated evaluator, forming a symbiotic relationship. The LLM's role is to creatively propose solutions in the form of computer code, while the evaluator acts as a gatekeeper, preventing the propagation of hallucinations and incorrect ideas. Through an iterative back-and-forth process between these components, initial solutions evolve into new knowledge. The system specifically searches for "functions" written in computer code, leading to the aptly named FunSearch.

Key Achievements:

This pioneering work marks the first instance of a new discovery for challenging open problems in science or mathematics using LLMs. FunSearch has successfully tackled the cap set problem, a longstanding mathematical challenge that has eluded solutions for decades. Moreover, it has demonstrated its practical utility by discovering more effective algorithms for the widely applicable "bin-packing" problem, with implications for enhancing data center efficiency.

Driving Discovery through Evolution with Language Models:

FunSearch employs an evolutionary method driven by LLMs, promoting and developing the highest-scoring ideas expressed as computer programs. This approach allows for the automatic evaluation of proposed solutions, contributing to a self-improving loop. The system uses Google's PaLM 2, although it remains compatible with other LLMs trained on code. This innovative method represents a significant step toward harnessing the creative potential of LLMs for solving complex problems.

Breaking New Ground in Mathematics:

Addressing the cap set problem, renowned mathematician Terence Tao's favorite open question, FunSearch collaborated with Jordan Ellenberg, a professor of mathematics at the University of Wisconsin–Madison. The cap set problem involves finding the largest set of points in a high-dimensional grid, where no three points lie on a line. Traditional computing approaches fall short due to the problem's immense complexity. FunSearch generated solutions in the form of programs that discovered the largest cap sets ever found, marking a substantial advancement in the past two decades.

Favoring Concise and Human-Interpretable Programs:

While FunSearch's ability to discover new mathematical knowledge is impressive, its unique strength lies in the interpretability of its outputs. Unlike traditional black-box solutions, FunSearch generates programs that describe how solutions were constructed. This transparent approach aligns with the scientific method, where discoveries are not only presented but also explained through the process used to derive them. FunSearch favors concise and human-interpretable programs, enhancing its scalability to tackle large, complex problems.

Collaborative Procedure between Humans and FunSearch:

The interpretability of FunSearch's programs provides actionable insights to researchers, fostering a collaborative procedure between humans and this innovative tool. By studying the generated programs, researchers gain conceptual richness beyond a mere list of numbers. The collaboration between humans and FunSearch, as demonstrated in the cap set problem, opens avenues for refining problems and achieving even better solutions.

LLM-Driven Discovery for Science and Beyond:

FunSearch stands as a testament to the potential of LLMs when safeguarded against hallucinations. It not only produces new mathematical discoveries but also reveals potentially impactful solutions to real-world problems. The envisioning of effective and tailored algorithms using LLM-driven approaches becoming common practice in science and industry is a promising outlook. FunSearch is just the beginning, with continuous improvement expected as LLMs evolve, broadening its capabilities to address society's pressing scientific and engineering challenges.

 

In the ever-expanding field of mathematical sciences, FunSearch emerges as a powerful ally, pushing the boundaries of what is achievable with LLMs. Its innovative approach to problem-solving, coupled with the collaborative synergy between humans and machines, sets the stage for a new era of discovery. As FunSearch evolves alongside the progress of LLMs, the potential applications in science and industry are boundless, promising groundbreaking solutions to some of the most challenging problems we face.