Following error will pop up sooner or later when working with OpenAI large language models such as GPT-4.

An assistant message with ‘tool_calls’ must be followed by tool messages responding to each ‘tool_call_id’. The following tool_call_ids did not have response messages

The API expects that the initial message containing the tool_calls array is followed by the actual tool call result message. Each tool call (= decision to call a function) must have a corresponding tool call result (= function result). This is how the LLM maps a function result to the decision to call a function.

I figured out following two points by trial and error, which are not mentioned in the official OpenAI documentation:

  1. The tool call result message must come right after the tool calls message. It’s not allowed to have any other messages in-between.
  2. Every id in the tool_calls array of the tool calls message must have a corresponding tool call result in the array of the tool call result message.

Following is not allowed, for example:

  1. tool call messages with ids 1, 2, 3
  2. tool call result message with ids 1, 2
  3. tool call result message with id 3

If you ran into the issue above, you might have multiple processes writing to the same conversation. At least, that was the issue I had.