Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models

Many approaches beyond Chain-of-Thought (CoT) rely on external control of the generation process, incurring large query counts and compute. Algorithm of Thoughts (AoT) instead guides LLMs along algorithmic reasoning paths using fully in-context algorithmic exemplars, enabling broader idea exploration with one or few queries. AoT outperforms prior single-query methods and competitive multi-query tree-search strategies with fewer tokens, suggesting that instructing an LLM with an algorithm can even surpass the algorithm itself. Code and materials: https://algorithm-of-thoughts.github.io.