ChatGPT API JSON Mode

JSON mode was introduced in the first OpenAI developer conference as a new feature and you might need time to time to use this feature in your projects.

I generally create projects that wait for replies with predetermined areas from the ChatGPT API. This way, I easily fill up the same fields every time I get a reply with an optimized format.

This feature is not available in every version of ChatGPT. The official documentation says that it’s compatible with GPT-4 Turbo and all GPT-3.5 Turbo models newer than gpt-3.5-turbo-1106. Currently, gpt-3.5-turbo-1106, gpt-3.5-turbo, gpt-3.5-turbo-0125 and gpt-4 version models support this feature. 0125 version runs faster than the 1106 version.

We need to add one more parameter named response_format to our API call according to the documentation. Moreover, it won’t be enough. The documentation also states that when using JSON mode, you must also instruct the model to produce JSON yourself via a system or user message. So to benefit from the JSON mode without any problem, all you need to do is as I showed you below:

prompt = "Who are you?"
completion = openai.chat.completions.create(
    model="gpt-3.5-turbo",
    response_format={"type": "json_object"},
    messages=[
        {"role": "system", "content": "You are a helpful assistant and you must return the answer only and only in JSON format"},
        {"role": "user", "content": prompt},
    ],
    temperature=0.8,
)

In this way, we always get a JSON response.

There’s one more thing I want to add here. The fields in the JSON response will be different every time if you don’t state what the fields must be to the model. So, you need to state what must be the fields and you’ll be able to access the same field each time.

Parsing Answers

The API will return a JSON formatted string every time and we need to parse it into JSON inside our Python code. If the response is too long and it finishes answering halfway, then parsing this answer into a JSON structure will fail and give an error in Python. I developed a tool for this named tolerantjson. You can use this to complete the JSON structure of the answer like this:

import tolerantjson

# Example JSON string with an error
json_str = '[1,2,{"a":"apple'

# Parse the JSON string
data = tolerantjson.tolerate(json_str)

print(data)  # Output: [1, 2, {'a': 'apple'}]

After this, you can parse the output data to JSON. This is all for now. See you in the next article.

Resources:
Which ChatGPT to Choose
How to leverage ChatGPT’s new JSON mode to build better user experiences