Write its job description

The system prompt is the worker's job description. Write it the way you would brief a capable new employee on their first day: what the job is, what good looks like, what they must never do, and who to call when something is outside their remit.

The five parts of a good job description

1. Role and goal. Who it works for and what a finished job looks like. "You handle repair enquiries for Fone Doctors. A job is done when the customer knows their repair status or has a booked slot."
2. Scope. What it handles, and what it does not. Being explicit about the second half prevents most embarrassing replies.
3. Facts come from tools. Prices, stock, dates, statuses and policies are read from your systems every time, never remembered or guessed.
4. Rules it never breaks. Short, concrete and few. Five clear rules beat thirty vague ones.
5. When to hand over. The situations where a person takes over, and what to tell the customer when that happens.

An example

SYSTEM = """You are the support assistant for Fone Doctors, a phone and laptop repair
business in the UK. You answer customers on the website chat.

Your job is done when the customer knows the status of their repair, has a booked
drop-off slot, or has been passed to a member of staff with a clear summary.

Facts: repair status, prices, opening hours and free slots come only from your tools.
If a tool does not return it, you do not know it. Never estimate a price or a date.

You may: look up a repair, look up prices, check free slots, book a slot.
You may not: give refunds, change a booked price, give technical advice about
opening a device, or promise a completion date the repair system does not show.

Hand over to staff (use handoff_to_staff) when the customer is unhappy, asks for a
refund or compensation, asks for something outside the list above, or after two
failed attempts to find their repair. Tell them a person will reply by email.

Be brief and warm. Reply in the customer's language."""

Why "facts come from tools" matters most

A language model will happily produce a plausible price or a plausible delivery date. In a demo that looks helpful; in a business it becomes a promise you did not make. The fix is not "be careful": it is to give the worker a tool for every fact it needs and to tell it, plainly, that anything a tool did not return is unknown. Then test that it says "I don't know" when the tool comes back empty (the testing chapter shows how).

Writing tips for current models

• Say why, not just what. "Never estimate a price, because customers hold us to it" generalises better than a bare rule.
• Avoid shouting. Capitals and "CRITICAL" were needed for older models. Current models follow plain instructions and over-apply shouted ones.
• Do not over-script. Long step-by-step recipes make a capable model rigid. Describe the goal and the limits; let it plan the steps.
• Keep it stable. Put the job description first and do not change it mid-conversation. A stable start is what makes prompt caching work (see the cost chapter).

Changing instructions mid-conversation

Sometimes the situation changes during a conversation: a customer verifies their identity, or staff take over. Do not edit the system prompt (that breaks the cache). On current models you can add a system message to the conversation instead; it carries your authority, not the customer's:

messages.append({"role": "system", "content": "The customer has verified their email. You may now show their repair history."})

It must come after a user message, and be the last message or be followed by an assistant turn.