AI Problem Formulation vs Prompt Engineering
Prompt Engineering is the art of formulating questions and providing examples to an AI model, to get usable output. The demand for dedicated prompt engineers, as a job title, may be short lived however, as AI models get better at creating their own prompts, and companies get more experienced in training their own Large Language Models.
So what is the key skill for consultants training a Large Language Model to make it effective for customer experience? The skill involved really comes down to Problem Formulation. You need to be able to train your model to correctly analyze the problem or issue, ideally from a typical customer's point of view.
Is Problem Formulation, an Alternate Discipline, or a Core Step in Prompt Engineering?
In an article for Harvard Business Review, titled Prompt Engineering isn't the Future, Oguz A. Acar, a chair in marketing at King's College London, argues in favor of Problem Formulation, as an alternative discipline to Prompt Engineering. In a deeper dive into the argument, however, it becomes questionable as to whether Problem Formulation is really an alternative discipline to Prompt Engineering, a more holistic approach, or simply an essential step in good prompt engineering.
While stating a problem may seem intuitive, it is often an underdeveloped skill, because of our focus on problem solving. Our work culture celebrates finding solutions over defining problems, and often most of our language about problems is phrased in a way that showcases our optimism as seeing each problem as an opportunity. For example, "What's the best way to charge an EV battery to prolong its life?" may be how the customer would phrase the question, but it is important to not let that language influence how we present a problem to the AI model.
Effective steps to Problem Formulation, according to Acar, can be broken down into: problem diagnosis, decomposition, reframing, and constraint design
Problem Diagnosis
Identifying the issue we want AI to solve. Often at this point we may be confused between whether we are addressing a symptom, or identifying a root cause of the issue. This can often be clarified using the “5 why's technique”: repeatedly asking “why” after seemingly stating a problem, to eventually get to the root cause.
Problem Decomposition
Breaking down the problem into manageable sub-problems is particularly useful when dealing with complex issues, with multiple concerns, or points of view.
Problem Reframing
Opening the problem up for alternative interpretations.
For example, if tasking AI with the problem of a lack of available parking for an office building, the question could be framed initially to generate solutions for optimizing layout and finding additional spaces, or could be framed to look at alternative solutions, like mass transit, or remote work. Proactively questioning problem objectives is key, here.
Problem Constraint Design
Defining the input, process, and output restrictions. For example, for product managers to generate useful, promotional, social media content from AI, there may be restictions on tone, length, and target audience.
Acar's conclusion highlights the argument of Problem Formulation over Prompt Engineering:
"Overemphasizing the crafting of the perfect combination of words can even be counterproductive, as it may detract from the exploration of the problem itself and diminish one’s sense of control over the creative process. Instead, mastering problem formulation could be the key to navigating the uncertain future alongside sophisticated AI systems."
Depending on the source, Prompt Engineering can either be broken down into two principles:
- Write clear and specific instructions
- Give the model time to "think"
Or as 5 steps:
- Define the desired output
Format, text or image, or the answer to a specific question.
- Understand model capabilities
Using an LLM designed for the type of task it is being used for.
- Consider context
The context in which the prompt will be used. Is it a question for a chatbot, or the design of a system?
- Use explicit instructions
Guide the model's behaviour, using cues such as “describe”, “compare”, etc.
- Experiment and Iterate
Experiment with different prompts, and iterate based on responses.
Conclusion
So is Problem Formulation actually a separate discipline from Prompt Engineering, Or is it a different or better way of defining the steps for prompt engineering?
Most of these steps seem to overlap, quite a bit.
Is the problem diagnosis, decomposition, and reframing of Problem Formulation, the same thing as the “Use explicit instructions” step of prompt engineering?
Is the constraint design part of Problem Formulation the same thing as the “Define the desired output” step of prompt engineering?
Ultimately the goal of either is to create prompts, and viewing it from the perspective of problem formulation may well be an approach that both creates more effecitve prompts, and also may instruct human stakeholders in better understanding the domain of the issues they are trying to solve. Breaking things down effectively for a Large Language Model could be viewed as similar to describing personal problems to a therapist. The act of describing the issue to a third party, human or AI, creates additional and more comprehensive understanding of the problems and solutions of the business domain.
