To achieve this, two components need to be defined: a world model and a reward function. Different from Chain-of-thoughts reasoning which autoregressively samples the next action, our goal is to efficiently search in the reasoning space for the optimal reasoning chain. LLM Reasoners formulate reasoning as planning ( RAP). ![]() For Blocksworld dataset where the problem above comes from, even the strongest GPT-4 model can only reach the success rate of ~30%. Unfortunately, it doesn't always work for complex reasoning problems. This simple method is often referred to as Chain-of-thoughts reasoning. At each time step, the next action is sampled from the LLM conditioned on the previous actions. Regarding each reasoning step as an action, we have $a_1=$" pick up the orange block", $a_2=$" stack the orange block on top of the blue block", and so on. Pick up the orange block stack the orange block on top of the blue block unstack the red block from on top of the yellow block put the red block on the table pick up the yellow block stack the yellow block on top of the orange block My goal is to have that the orange block is on top of the blue block and the yellow block on top of the orange block. For the problem above, the prompt inputted to the LLM and the expected output (in bold) is shown below: I am playing with a set of blocks where I need to arrange the blocks into stacks.Īs initial conditions I have that, the red block is clear, the blue block is clear, the orange block is clear, the hand is empty, the red block is on the yellow block, the yellow block is on the table, the blue block is on the table and the orange block is on the table. Let's start with a naive method for LLM reasoning: Prompted with a few examples of problem-solving step by step, an LLM can generate a chain of thoughts (or a sequence of actions) to solve a new problem. Some results are on the subsets of the first 100 examples (*). We list the results reported in their paper / reproduced from their official repositories for reference (†). Our library has been tested against official repos of Tree-of-Thoughts and Guided Decoding. We tested different reasoning algorithms on the following benchmarks (to be updated). Specifically, we integrated LLaMA with the option of using fairscale backend for improved multi-GPU performance or LLaMA.cpp backend with lower hardware requirements. Huggingface transformers, OpenAI API, etc. Even for the most complex reasoning algorithms like Monte-Carlo Tree Search, users can easily diagnose and understand what occurred with one line of python code.Ĭompatibility with any LLM libraries: Our framework is compatible with any LLM frameworks, e.g. Intuitive Visualization and Interpretation: Our library provides visualization tools to aid users in comprehending the reasoning process. These advanced algorithms enable tree-structure reasoning and outperform traditional chain-of-thoughts approaches. ![]() 10, 2023: Llama-2 is supported! You can run examples with Llama-2 now.Ĭutting-Edge Reasoning Algorithms: We offer the most up-to-date search algorithms for reasoning with LLMs, such as RAP-MCTS, Tree-of-Thoughts, Guided Decoding, and more. ![]() Now you can try llama-2-70B with 2 x 24G GPUs.Īug. 21, 2023: A batch of quantized Llama-2 models has arrived! BitsandBytes with huggingface API, GPT-Q with exllama are available. 23, 2023: Reasoning-via-Planning is accepted to EMNLP 2023! Check our paper with updated results and discussion!Īug. 25, 2023: A video tutorial on the visualizer of LLM Reasoners are available. ![]() Given any reasoning problem, simply define the reward function and an optional world model (explained below), and let LLM reasoners take care of the rest, including Reasoning Algorithms, Visualization, LLM calling, and more! News It approaches multi-step reasoning as planning and searches for the optimal reasoning chain, which achieves the best balance of exploration vs exploitation with the idea of "World Model" and "Reward". LLM Reasoners is a library to enable LLMs to conduct complex reasoning, with advanced reasoning algorithms.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |