The difference between inference big model and ordinary big model What is the inference model? On September 12, 2024, OpenAI officially announced the OpenAI o1 reasoning model. OpenAI claims that the reasoning ability of the OpenAI o1 model has been greatly improved compared to the current large language model (GPT-4o). Because the training of the OpenAI o1 model uses a new AI training method, emphasizing the importance of the "thinking chain" process and reinforcement learning, it has ultimately led to a significant improvement in mathematical logic and reasoning. People began to call itThe big model with thinking process is the reasoning big model. On OpenAI's official website, OpenAI defines reasoning models as thinking before answering, andGenerate a long chain of thought processes internally before responding to the user.Thought chaining is a way to prompt a large language model to reason step by step. It allows the model to explicitly write down the intermediate steps of reasoning before reaching the final answer. This is just like when humans solve complex problems, they write down their thinking process first. In other words, if the model has a long chain of thinking process before replying to you (this process must be able to display output), and gives an answer after exploring many different paths, then a large model with this ability is a reasoning large model. Although there is no formal definition,At present, there is a consensus in the AI industry and academia that the core of the reasoning model is to deal with those that require multi-step logical deduction.Complex problems that can be solved.Reasoning LLM How is it different from the Standard Large Model (Standard LLM)?Let's take a simple example:Non-inferential questions: Give the answer directly without any reasoning (Where is the capital of South Korea?)Reasoning problems: You need to first understand what the problem is and what the relationship is between them, and then solve the corresponding problem step by step (a train of trucks traveled at a speed of 60 miles per hour for 4 hours. How far did it travel in total?). Then you need to first understand the "distance = speed * time" problem in mathematics.An ordinary large language model (LLM) may directly output a short answer (such as "240 kilometers"), while the characteristic of the reasoning model is to show the intermediate derivation process.Step 1: Identify the type of problem (speed, time and distance)Step 2: Apply the formula distance = speed x timeStep 3: Plug in the numbers and calculate 60 mph × 4 hours = 240 miles How to choose the appropriate large model for different scenes?Reasoning models are good at solving complex tasks such as puzzles, math problems, and challenging coding tasks, but they are not always necessary or efficient for simpler tasks such as summarization, translation, or knowledge-based question answering. Using reasoning models for every task may be inefficient and error-prone. In other words, not all scenario tasks are suitable for reasoning large models, so it is particularly important to choose the right large model for different scenarios. The following are some views and summaries for reference.FeaturesStandard Large Model (Standard LLM)Reasoning about large models(Reasoning LLM)Applicable scenariosText generation, translation, summarization, basic knowledge question answeringComplex reasoning, puzzle solving, math, coding challengesComplex problem solving skillsGenerally, it is difficult to handle complex problems with multiple stepsExcellent, able to think deeply and reason logicallySpecialized areas of workWrite press releases, translate articles, generate product descriptions, answer general knowledge questionsSolve complex logic puzzles, write complex algorithms, mathematical proofsOperational efficiencyHigher, faster response, and relatively less resource consumptionLow, long inference time, high resource consumptionGeneralization/universalityRelatively weak, more dependent on training dataStronger and better able to adapt to new problems and unknown scenariosRisk of hallucinations/data distortionLower, more dependent on known knowledge and patternsHigher, may give wrong answers due to "overthinking"Cost ControlUsually lowerUsually higher Inference large models usually need to be trained and strengthened.There are currently four main methods for training large inference models: inference-time expansion, pure reinforcement learning, supervised fine-tuning combined with reinforcement learning, and pure supervised fine-tuning and distillation.. This part of the content is more professional and in-depth. Interested friends can read more official texts or papers. More details will not be repeated for the time being.
