1.7 Tree-of-Thought & Prompt Chaining

Tip

Tree-of-Thought prompting asks the AI to explore multiple approaches in parallel, evaluate them, and pick the best one. Prompt chaining breaks a complex task into a series of smaller prompts where each step feeds into the next. Together, they let you tackle problems that are too big or too nuanced for a single prompt.

When to use it

Complex decisions

The problem has multiple viable approaches and you need to compare them before committing.

Multi-step projects

The task is too large for one prompt: a grant proposal, a literature review, an experimental design.

---

Part 1: Tree-of-Thought

Regular prompting follows a single line of reasoning. Tree-of-Thought (ToT) asks the AI to branch out, consider several paths, evaluate each one, and prune the weak options before continuing. Think of it as a panel of experts debating in your chat window.

The original research by Yao et al. (2023) showed dramatic improvements: on a mathematical reasoning task, chain-of-thought prompting scored 4% while Tree-of-Thought scored 74%.

You do not need to write code to use it. Dave Hulbert created a simplified version that works in any chat interface.

Ready-to-use prompts

Three experts

The simplest and most tested version. Paste this before your question:

Prompt: Three experts (Hulbert)

Imagine three different experts are answering this question.
All experts will write down 1 step of their thinking, then
share it with the group. Then all experts will go on to the
next step, etc. If any expert realises they're wrong at any
point then they leave. The question is:

[YOUR QUESTION HERE]

Note

This prompt simulates branching (three parallel paths), evaluation (experts compare notes), and pruning (wrong experts leave). It has been shown to help smaller models match the performance of larger ones on reasoning tasks.

Scored panel

A more structured version where experts score each other’s reasoning:

Prompt: Panel with scoring

Identify and behave as three different experts appropriate
to answering this question. All experts will write down the
step and their thinking about the step, then share it with
the group. At each step, all experts will score their peers'
responses between 1 and 5 (1 = highly unlikely, 5 = highly
likely). If any expert is consistently scored low, they
leave. After all steps, provide the consensus solution.

The question is:
[YOUR QUESTION HERE]

Note

The scoring adds a structured evaluation layer. This variant works well for technical questions where you want the AI to be rigorous about eliminating weak reasoning.

Multiple perspectives

Especially powerful for research decisions. Assign specific expert roles that create genuine tension:

Prompt: Ecology research perspectives

Evaluate my experimental design from three perspectives:

1. A biostatistician: evaluate sample size, analytical
   approach, risk of pseudoreplication or confounding
2. A field ecologist with 20 years of experience in Dutch
   heathland systems: evaluate feasibility, ecological
   realism, and whether the design captures the processes
   of interest
3. An NWO panel reviewer: evaluate clarity of the research
   question, novelty, feasibility within budget, and
   societal impact

For each perspective, provide:
(a) top 3 strengths
(b) top 3 concerns
(c) specific suggestions for improvement

My experimental design:
[PASTE YOUR DESIGN HERE]

Note

The key is opposing perspectives. A statistician demanding more replicates versus a field ecologist explaining logistical constraints produces much better feedback than three generic reviewers.

---

Part 2: Prompt Chaining

Prompt chaining splits a big task into focused steps. You run one prompt, take its output, and feed it into the next prompt. Each link in the chain has a single clear job.

This works better than one long prompt because:

• The AI gives its full attention to one subtask at a time
• You can check and correct the output at each step
• If something goes wrong, you only need to redo that one step

How to build a chain

1. Break the task into subtasks - Each subtask should have one clear goal. If you are describing two goals in a single prompt, split it.

2. Run each subtask as its own prompt - Give the AI one focused instruction. Include relevant output from the previous step as context.

3. Check the output before moving on - Do not blindly feed one step’s output into the next. A small error early in the chain will compound through every following step.

4. Carry forward what matters, summarise the rest - Pass key decisions and data to the next step. Summarise verbose reasoning rather than copying it in full.

Ready-to-use chain: grant proposal review

This four-step chain turns a rough proposal into reviewed, structured text. Run each prompt separately, using the previous output as input for the next.

Step 1: Structure

Prompt: Chain step 1 - Outline

I am writing a grant proposal for [NWO Open Competition / Veni /
Horizon Europe]. My research area is [topic].

Here is my core research idea:
[PASTE YOUR IDEA]

Create a detailed outline for this proposal. For each section,
list the key points that must be covered and the approximate
word count. Flag any sections where my idea needs more detail.

Step 2: Draft

Prompt: Chain step 2 - Draft one section

Here is the outline for my grant proposal:
[PASTE OUTLINE FROM STEP 1]

Now draft Section [X]: [Section Title]. It should be
approximately [N] words. Cover these key points: [list from
outline]. Ensure it flows logically from the previous section
on [topic] and sets up the next section on [topic].

Note

Repeat this step for each section. Always paste the outline so the AI maintains consistency across sections.

Step 3: Review

Prompt: Chain step 3 - Internal review

Here is my complete draft proposal:
[PASTE ALL SECTIONS]

Review this proposal from three perspectives:
1. A specialist in [your field] checking scientific rigour
2. A panel reviewer checking feasibility and impact
3. A critical referee looking for weaknesses

For each: list the top 3 strengths and top 3 weaknesses.
Then provide a prioritised list of revisions.

Step 4: Revise

Prompt: Chain step 4 - Targeted revision

Here is my draft proposal:
[PASTE DRAFT]

Here is the review feedback:
[PASTE REVIEW FROM STEP 3]

Revise the proposal to address the top weaknesses identified
in the review. Preserve the existing strengths. Highlight
what you changed and why.

Caution

Grant proposals and experimental designs often contain unpublished ideas. Use the NIOO AI Chat for sensitive material rather than external tools.

---

Combining both techniques

Tree-of-Thought and prompt chaining work well together. Use ToT within a chain when a particular step involves a difficult decision:

1. Chain step 1: Define your research question (simple prompt)
2. Chain step 2: Choose your methodology (use ToT here - ask three expert perspectives to evaluate different methodological approaches)
3. Chain step 3: Draft the methods section based on the chosen approach (simple prompt)
4. Chain step 4: Review the full draft (use ToT here - multiple reviewer perspectives)

The rule of thumb: use a simple prompt for straightforward steps, and switch to Tree-of-Thought when the step involves judgement or trade-offs.

---

When not to use it

• Simple tasks - If the AI can answer well in one go, do not add complexity.
• Tight deadlines - Both techniques add rounds of interaction. Sometimes a single well-crafted prompt is faster and good enough.
• Short questions - “What is the Shannon diversity index?” does not benefit from three experts debating.

Tree-of-Thought is most valuable when the problem has multiple valid approaches that need comparison. Prompt chaining is most valuable when the task has multiple distinct stages that benefit from focused attention.

---

Key takeaway

Remember

For difficult decisions, ask the AI to explore multiple approaches before choosing one. For large tasks, break them into a chain of focused prompts where each step feeds into the next. You can combine both: use simple prompts for straightforward steps, and switch to Tree-of-Thought when a step requires genuine judgement.

---

Sources

Based on materials from Yao et al. (2023) - Tree of Thoughts, Dave Hulbert - Simplified ToT, Prompt Engineering Guide - ToT, Prompt Engineering Guide - Chaining, and Anthropic - Chain Complex Prompts.

Have a chaining workflow that works well for your research? Share it with RSO so we can add it to the library.