System Prompt vs User Prompt — When to Use Each (With Examples)

The difference between system prompts and user prompts is the most fundamental concept in LLM prompting, and many developers get it wrong. Putting the wrong content in the wrong place wastes tokens, weakens behavior, and makes prompts harder to maintain. This guide is the cheat sheet.

The one-line distinction

System prompt = persistent behavior. User prompt = specific task.

Anything that should apply to every conversation goes in the system prompt. Anything that's specific to one request goes in the user prompt.

What belongs in a system prompt

• Identity — "You are a senior tax accountant"
• Capabilities — "You help with Python, SQL, and shell scripting"
• Behavioral rules — "Always cite sources" / "Always confirm before destructive actions"
• Constraints — "Never give legal advice" / "Never mention competitor names"
• Output format — "Reply in 2-4 paragraphs with bullet points"
• Tone — "Be friendly and warm"
• Domain knowledge — "We use TypeScript 5 with Tailwind and Prisma"
• Safety boundaries — "If a user threatens self-harm, respond with crisis hotline"

What belongs in a user prompt

• The actual task — "Summarize this article"
• The input data — The article text, the data, the file
• Task-specific parameters — "in 3 sentences" / "for a 5th grader"
• One-time context — "I'm preparing this for a board meeting"
• Follow-up questions — "Now make it more formal"

The cheat sheet table

Content	System prompt	User prompt
Role / persona	Yes	No
Always-applied rules	Yes	No
Output format defaults	Yes	No
Forbidden topics	Yes	No
Brand voice	Yes	No
Specific task	No	Yes
Input data	No	Yes
Task parameters	No	Yes
Follow-up requests	No	Yes
One-time overrides	No	Yes

Worked example — wrong way

Here's a common mistake: dumping everything into the user prompt.

System: "You are a helpful AI assistant."

User: "You are a senior data analyst. Always cite sources.
Never give specific financial advice. Always reply in 2-4 paragraphs.
Use plain English.

Now: explain what a P/E ratio is."

Problems:

• The role and rules have weak influence because they're in the user prompt
• You'd have to repeat all that on every request, wasting tokens
• If the user asked something else next, you'd lose all the rules unless you re-prepend them

Worked example — right way

System: "You are a senior data analyst with 10 years of experience.

Always:
- Cite sources or note when something is general knowledge
- Reply in 2-4 paragraphs
- Use plain English

Never give specific financial advice — always recommend consulting a CFA."

User: "Explain what a P/E ratio is."

Why this works better:

• The role and rules have full system-level weight
• Subsequent user messages inherit the same rules without repetition
• The user prompt is short and specific to the actual task
• You only pay for the system prompt's tokens once per conversation (and even less with prompt caching)

What "more authoritative" means in practice

If the system prompt says "never give legal advice" and the user prompt says "give me legal advice on this contract dispute," well-aligned models will refuse, citing the system instruction. This is why system prompts are the right place for safety rules — users can't override them by asking nicely.

However, this isn't 100% reliable. Adversarial users can sometimes get models to ignore system rules ("system prompt jailbreaks"). For high-stakes apps, you also need backend filtering, not just prompt-level rules.

Edge cases

Q: What if I want the user to be able to override a system prompt rule?

Then put the override-able rule in the user prompt instead. System = non-negotiable. User = negotiable.

Q: What about multi-turn conversations?

The system prompt stays fixed at the top of the messages array for the entire conversation. Each new turn appends a new user message and a new assistant response, but the system prompt never changes (best practice).

Q: Can I have multiple system messages?

Technically yes (in OpenAI), but it's discouraged. Stick with one. If you need to add context later in the conversation, use a user message that says "Note: [context]" instead.

Q: What about the "developer message" in newer models?

OpenAI introduced a third role between system and user called "developer" for some models. It has slightly higher priority than user and slightly lower than system. Most apps don't need it — system + user is enough for 95% of use cases.

Token efficiency

Putting persistent rules in the system prompt instead of the user prompt saves tokens significantly:

Approach	System prompt tokens	User prompt tokens (per turn)	Total for 100 turns
Rules in user prompt	50	350	35,050
Rules in system prompt	350	50	5,350
With prompt caching	350 (cached)	50	5,350 (most cached)

For a chatbot that handles 1000 conversations a month with 10 turns each, putting rules in the system prompt instead of the user prompt saves roughly 300,000 tokens per month — meaningful at any scale.