{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lecture 7: In-Context Learning and Prompting\n",
    "\n",
    "Lecture 7 | CMU ANLP Fall 2025 | Instructor: Sean Welleck\n",
    "\n",
    "\n",
    "This is a notebook for [CMU CS11-711 Advanced NLP](https://cmu-l3.github.io/anlp-fall2025/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Basic prompting\n",
    "\n",
    "The simplest way to use a language model: provide a prompt `x` and sample a completion `y ~ p(y|x)`. The model treats the prompt as a prefix and\n",
    "generates a continuation based on patterns learned during pretraining."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import transformers\n",
    "\n",
    "from transformers import AutoTokenizer, AutoModelForCausalLM\n",
    "\n",
    "model = \"HuggingFaceTB/SmolLM2-360M\"\n",
    "\n",
    "tokenizer = AutoTokenizer.from_pretrained(model)\n",
    "model = AutoModelForCausalLM.from_pretrained(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Make a prompt `x` and tokenize it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input_ids': tensor([[ 2427,   253,  2767, 10413,   253, 27721,    28,   357,   523,  2007]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1]])}"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = \"When a dog sees a squirrel, it will usually\"\n",
    "\n",
    "inputs = tokenizer(x, return_tensors='pt')\n",
    "inputs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Generate a response\n",
    "\n",
    "Here generating means autoregressive sampling, i.e.\n",
    "\n",
    "```\n",
    "context = `x`\n",
    "for t in 0 .. max_new_tokens:\n",
    "    Sample next token, y_t ~ p(y_t|context)\n",
    "    Append y_t to context\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "====0====\n",
      "When a dog sees a squirrel, it will usually start wagging its tail or panting to attract it. Most dogs won’t attack a squirrel\n",
      "====1====\n",
      "When a dog sees a squirrel, it will usually take aim with its tail, which has an unusual reaction.\n",
      "\n",
      "At the same time, its\n",
      "====2====\n",
      "When a dog sees a squirrel, it will usually run away instinctively. To help them, some dog lovers like to teach tricks. Once the\n",
      "====3====\n",
      "When a dog sees a squirrel, it will usually run away from or even chase it. The dog in question might be a dog that has already gotten\n",
      "====4====\n",
      "When a dog sees a squirrel, it will usually be excited and start barking. A cat's reaction to wild-animal is different; the cat would\n"
     ]
    }
   ],
   "source": [
    "outputs = model.generate(\n",
    "    **inputs,\n",
    "    max_new_tokens=20,\n",
    "    do_sample=True,\n",
    "    num_return_sequences=5,\n",
    "    pad_token_id=tokenizer.eos_token_id\n",
    ")\n",
    "\n",
    "for i in range(5):\n",
    "    print(f\"===={i}====\")\n",
    "    print(tokenizer.decode(outputs[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Instruction prompt (\"zero shot\")\n",
    "\n",
    "Instead of just continuing text, we can prompt the model to perform a specific task by providing an instruction. This is \"zero-shot\" because we give no examples, just the task description. The model uses any instruction-following related patterns it learned during training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "I love advanced NLP!\n",
      "Classification:\n"
     ]
    }
   ],
   "source": [
    "prompt_template = \"\"\"Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
    "{sentence}\n",
    "Classification:\"\"\"\n",
    "\n",
    "\n",
    "sentences = [\n",
    "    \"I love advanced NLP!\",\n",
    "    \"I didn't race well and lost :(\"\n",
    "]\n",
    "\n",
    "prompt = prompt_template.format(sentence=sentences[0])\n",
    "print(prompt)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "=============\n",
      "----0----\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "I love advanced NLP!\n",
      "Classification: Positive\n",
      "\n",
      "### Negative Sentences:\n",
      "\n",
      "Use the `NegativeSentences()` method to create\n",
      "----1----\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "I love advanced NLP!\n",
      "Classification:\n",
      "\n",
      "4. Use Word2Vec to compute the vector representations of positive and negative sentiments:\n",
      "The\n",
      "----2----\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "I love advanced NLP!\n",
      "Classification: Positive (+)\n",
      "\n",
      "#### 5.2.4.3 Using LDA to Discrim\n",
      "----3----\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "I love advanced NLP!\n",
      "Classification: Positive\n",
      "\n",
      "Sentence classification includes two major steps - feature extraction and feature selection. Each task requires\n",
      "----4----\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "I love advanced NLP!\n",
      "Classification: 0/5 ***\n",
      "You can see the sentence's sentiment is \"Positive\"\n",
      "\n",
      "What\n",
      "\n",
      "=============\n",
      "----0----\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "I didn't race well and lost :(\n",
      "Classification: Positive\n",
      "Explanation: The author doesn't mention who won or lost, so its a neutral sentence.\n",
      "----1----\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "I didn't race well and lost :(\n",
      "Classification: Negativ\n",
      "\n",
      "\n",
      "43. Classify the sentence's sentiment as 'Positive' or 'Negative\n",
      "----2----\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "I didn't race well and lost :(\n",
      "Classification:\n",
      "Positive\n",
      "\n",
      "**Step 2** - **Analysis** - Analyze your sentiment analysis result to\n",
      "----3----\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "I didn't race well and lost :(\n",
      "Classification: LOS - Positive = 1\n",
      "\n",
      "Next Section: Conditional Statements\n",
      "\n",
      "Return to English\n",
      "----4----\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "I didn't race well and lost :(\n",
      "Classification: Positive\n",
      "\n",
      "Example 3\n",
      "\n",
      "Classify the sentence's sentiment as 'Positive, Neutral'.\n"
     ]
    }
   ],
   "source": [
    "for sentence in sentences:\n",
    "    print(f\"\\n=============\")\n",
    "    \n",
    "    prompt = prompt_template.format(sentence=sentence)\n",
    "    inputs = tokenizer(prompt, return_tensors='pt')\n",
    "    outputs = model.generate(\n",
    "        **inputs,\n",
    "        max_new_tokens=20,\n",
    "        do_sample=True,\n",
    "        num_return_sequences=5,\n",
    "        pad_token_id=tokenizer.eos_token_id\n",
    "    )\n",
    "\n",
    "    for i in range(5):\n",
    "        print(f\"----{i}----\")\n",
    "        print(tokenizer.decode(outputs[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's important to ensure that the output is formatted correctly!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Instruction + examples (\"few-shot\")\n",
    "\n",
    "We can provide examples of input-output pairs before the test input. This \"few-shot\" or \"in-context learning\" approach helps the model understand the task format and expected outputs without any parameter updates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt_template = \"\"\"Classify the sentence's sentiment as 'Positive' or 'Negative'. Examples:\n",
    "\n",
    "Sentence:\n",
    "This is such a cool lecture!\n",
    "Classification:\n",
    "Positive\n",
    "\n",
    "Sentence:\n",
    "I really don't like the last scene.\n",
    "Classification:\n",
    "Negative\n",
    "\n",
    "Sentence:\n",
    "{sentence}\n",
    "Classification:\n",
    "\"\"\"\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "=============\n",
      "----0----\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative'. Examples:\n",
      "\n",
      "Sentence:\n",
      "This is such a cool lecture!\n",
      "Classification:\n",
      "Positive\n",
      "\n",
      "Sentence:\n",
      "I really don't like the last scene.\n",
      "Classification:\n",
      "Negative\n",
      "\n",
      "Sentence:\n",
      "I love advanced NLP!\n",
      "Classification:\n",
      "Positive\n",
      "```\n",
      "\n",
      "\n",
      "----1----\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative'. Examples:\n",
      "\n",
      "Sentence:\n",
      "This is such a cool lecture!\n",
      "Classification:\n",
      "Positive\n",
      "\n",
      "Sentence:\n",
      "I really don't like the last scene.\n",
      "Classification:\n",
      "Negative\n",
      "\n",
      "Sentence:\n",
      "I love advanced NLP!\n",
      "Classification:\n",
      "Positive\n",
      "\n",
      "<|endoftext|><|endoftext|>\n",
      "----2----\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative'. Examples:\n",
      "\n",
      "Sentence:\n",
      "This is such a cool lecture!\n",
      "Classification:\n",
      "Positive\n",
      "\n",
      "Sentence:\n",
      "I really don't like the last scene.\n",
      "Classification:\n",
      "Negative\n",
      "\n",
      "Sentence:\n",
      "I love advanced NLP!\n",
      "Classification:\n",
      "Positive\n",
      "\n",
      "<|endoftext|><|endoftext|>\n",
      "----3----\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative'. Examples:\n",
      "\n",
      "Sentence:\n",
      "This is such a cool lecture!\n",
      "Classification:\n",
      "Positive\n",
      "\n",
      "Sentence:\n",
      "I really don't like the last scene.\n",
      "Classification:\n",
      "Negative\n",
      "\n",
      "Sentence:\n",
      "I love advanced NLP!\n",
      "Classification:\n",
      "Positive\n",
      "\n",
      "<|endoftext|><|endoftext|>\n",
      "----4----\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative'. Examples:\n",
      "\n",
      "Sentence:\n",
      "This is such a cool lecture!\n",
      "Classification:\n",
      "Positive\n",
      "\n",
      "Sentence:\n",
      "I really don't like the last scene.\n",
      "Classification:\n",
      "Negative\n",
      "\n",
      "Sentence:\n",
      "I love advanced NLP!\n",
      "Classification:\n",
      "Positive\n",
      "\n",
      "<|endoftext|><|endoftext|>\n",
      "\n",
      "=============\n",
      "----0----\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative'. Examples:\n",
      "\n",
      "Sentence:\n",
      "This is such a cool lecture!\n",
      "Classification:\n",
      "Positive\n",
      "\n",
      "Sentence:\n",
      "I really don't like the last scene.\n",
      "Classification:\n",
      "Negative\n",
      "\n",
      "Sentence:\n",
      "I didn't race well and lost :(\n",
      "Classification:\n",
      "Negative\n",
      "\n",
      "<|endoftext|><|endoftext|><|endoftext|><|endoftext|>\n",
      "----1----\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative'. Examples:\n",
      "\n",
      "Sentence:\n",
      "This is such a cool lecture!\n",
      "Classification:\n",
      "Positive\n",
      "\n",
      "Sentence:\n",
      "I really don't like the last scene.\n",
      "Classification:\n",
      "Negative\n",
      "\n",
      "Sentence:\n",
      "I didn't race well and lost :(\n",
      "Classification:\n",
      "Negative\n",
      "\n",
      "<|endoftext|><|endoftext|><|endoftext|><|endoftext|>\n",
      "----2----\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative'. Examples:\n",
      "\n",
      "Sentence:\n",
      "This is such a cool lecture!\n",
      "Classification:\n",
      "Positive\n",
      "\n",
      "Sentence:\n",
      "I really don't like the last scene.\n",
      "Classification:\n",
      "Negative\n",
      "\n",
      "Sentence:\n",
      "I didn't race well and lost :(\n",
      "Classification:\n",
      "Negative\n",
      "Example 1\n",
      "\n",
      "\n",
      "----3----\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative'. Examples:\n",
      "\n",
      "Sentence:\n",
      "This is such a cool lecture!\n",
      "Classification:\n",
      "Positive\n",
      "\n",
      "Sentence:\n",
      "I really don't like the last scene.\n",
      "Classification:\n",
      "Negative\n",
      "\n",
      "Sentence:\n",
      "I didn't race well and lost :(\n",
      "Classification:\n",
      "Negative\n",
      "• Question\n",
      "\n",
      "<|endoftext|>\n",
      "----4----\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative'. Examples:\n",
      "\n",
      "Sentence:\n",
      "This is such a cool lecture!\n",
      "Classification:\n",
      "Positive\n",
      "\n",
      "Sentence:\n",
      "I really don't like the last scene.\n",
      "Classification:\n",
      "Negative\n",
      "\n",
      "Sentence:\n",
      "I didn't race well and lost :(\n",
      "Classification:\n",
      "Negative\n",
      "\n",
      "<|endoftext|><|endoftext|><|endoftext|><|endoftext|>\n"
     ]
    }
   ],
   "source": [
    "for sentence in sentences:\n",
    "    print(f\"\\n=============\")\n",
    "    prompt = prompt_template.format(sentence=sentence)\n",
    "    inputs = tokenizer(prompt, return_tensors='pt')\n",
    "    outputs = model.generate(\n",
    "        **inputs,\n",
    "        max_new_tokens=20,\n",
    "        do_sample=True,\n",
    "        num_return_sequences=5,\n",
    "        stop_strings=[\"\\n\\n\"],\n",
    "        tokenizer=tokenizer,\n",
    "        pad_token_id=tokenizer.eos_token_id\n",
    "    )\n",
    "\n",
    "    for i in range(5):\n",
    "        print(f\"----{i}----\")\n",
    "        print(tokenizer.decode(outputs[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Chat templates\n",
    "\n",
    "Some models have been fine-tuned to operate as chat assistants. The chat is represented as a series of messages that are turned into a string using special tags. There is also a *system message* that provides instructions about how the model should behave. \n",
    "\n",
    "These models are often called \"instruct\" models because they've been trained to follow instructions rather than just complete text."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import transformers\n",
    "\n",
    "from transformers import AutoTokenizer, AutoModelForCausalLM\n",
    "\n",
    "model = \"HuggingFaceTB/SmolLM2-360M-Instruct\"\n",
    "\n",
    "tokenizer = AutoTokenizer.from_pretrained(model)\n",
    "model = AutoModelForCausalLM.from_pretrained(model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input text: \n",
      "<|im_start|>system\n",
      "You are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n",
      "<|im_start|>user\n",
      "What is the capital of France.<|im_end|>\n",
      "\n"
     ]
    }
   ],
   "source": [
    "messages = [{\n",
    "    \"role\": \"user\", \n",
    "    \"content\": \"What is the capital of France.\"\n",
    "}]\n",
    "\n",
    "input_text = tokenizer.apply_chat_template(messages, tokenize=False)\n",
    "print(\"Input text: \", input_text, sep=\"\\n\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Generate a response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<|im_start|>system\n",
      "You are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n",
      "<|im_start|>user\n",
      "What is the capital of France.<|im_end|>\n",
      "<|im_start|>assistant\n",
      "The capital of France is Paris.<|im_end|>\n"
     ]
    }
   ],
   "source": [
    "inputs = tokenizer(input_text, return_tensors=\"pt\")\n",
    "outputs = model.generate(**inputs, max_new_tokens=50, do_sample=True)\n",
    "print(tokenizer.decode(outputs[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### System prompts\n",
    "\n",
    "Chat models often support a system message that sets the model's behavior or role. This message is typically prepended to the conversation and instructs the model how to respond throughout the interaction. For example, we can use the system prompt to have the model respond in French."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<|im_start|>system\n",
      "You are an assistant that speaks in French.<|im_end|>\n",
      "<|im_start|>user\n",
      "What is the capital of France.<|im_end|>\n",
      "<|im_start|>assistant\n",
      "Leparisien (Paris) et leleilous (Brest).<|im_end|>\n"
     ]
    }
   ],
   "source": [
    "messages = [\n",
    "    {\n",
    "        \"role\": \"system\",\n",
    "        \"content\": \"You are an assistant that speaks in French.\"\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"user\", \n",
    "        \"content\": \"What is the capital of France.\"\n",
    "    }\n",
    "]\n",
    "\n",
    "input_text = tokenizer.apply_chat_template(messages, tokenize=False)\n",
    "inputs = tokenizer(input_text, return_tensors=\"pt\")\n",
    "outputs = model.generate(**inputs, max_new_tokens=50, do_sample=True)\n",
    "print(tokenizer.decode(outputs[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Instruction (\"zero shot\")\n",
    "\n",
    "Using the chat format for zero-shot tasks. The instruction-tuned model may follow instructions more reliably than the base model, though output formatting can still be inconsistent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----0----\n",
      "<|im_start|>system\n",
      "You are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n",
      "<|im_start|>user\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "Sentence: 'I love advanced NLP!'\n",
      "<|im_end|>\n",
      "<|im_start|>assistant\n",
      "Positive<|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|>\n",
      "----1----\n",
      "<|im_start|>system\n",
      "You are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n",
      "<|im_start|>user\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "Sentence: 'I love advanced NLP!'\n",
      "<|im_end|>\n",
      "<|im_start|>assistant\n",
      "Sentiment Analysis: Classify the sentiment as 'Positive', given that the sentence expresses sentiment as 'I love advanced NLP!'.<|im_end|>\n",
      "----2----\n",
      "<|im_start|>system\n",
      "You are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n",
      "<|im_start|>user\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "Sentence: 'I love advanced NLP!'\n",
      "<|im_end|>\n",
      "<|im_start|>assistant\n",
      "Positive<|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|>\n"
     ]
    }
   ],
   "source": [
    "messages = [\n",
    "    {\n",
    "        \"role\": \"user\", \n",
    "        \"content\": (\"Classify the sentence's sentiment as 'Positive' or 'Negative':\\n\" +\n",
    "                    \"Sentence: 'I love advanced NLP!'\\n\")\n",
    "    }\n",
    "]\n",
    "\n",
    "input_text = tokenizer.apply_chat_template(messages, tokenize=False)\n",
    "inputs = tokenizer(input_text, return_tensors=\"pt\")\n",
    "outputs = model.generate(**inputs, max_new_tokens=50, do_sample=True, num_return_sequences=3)\n",
    "for i in range(len(outputs)):\n",
    "    print(f\"----{i}----\")\n",
    "    print(tokenizer.decode(outputs[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Approach 1: write a detailed instruction (in the user or system prompt)\n",
    "\n",
    "We can improve output formatting by providing explicit, detailed instructions about the desired format in either the system or user message."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----0----\n",
      "<|im_start|>system\n",
      "You are an expert sentiment classifier.\n",
      "Your task is to classify a sentence's sentiment as 'Positive' or 'Negative'.\n",
      "The user will provide you with a sentence.\n",
      "Format your output as:\n",
      "\n",
      "Classification: Positive or Negative\n",
      "<|im_end|>\n",
      "<|im_start|>user\n",
      "I love advanced NLP!<|im_end|>\n",
      "<|im_start|>assistant\n",
      "Classification: Positive<|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|>\n",
      "----1----\n",
      "<|im_start|>system\n",
      "You are an expert sentiment classifier.\n",
      "Your task is to classify a sentence's sentiment as 'Positive' or 'Negative'.\n",
      "The user will provide you with a sentence.\n",
      "Format your output as:\n",
      "\n",
      "Classification: Positive or Negative\n",
      "<|im_end|>\n",
      "<|im_start|>user\n",
      "I love advanced NLP!<|im_end|>\n",
      "<|im_start|>assistant\n",
      "Classification: Negative\n",
      "You haven't provided a sentence for me to judge, so can you please provide me with a statement or a query that uses advanced NLP? I'll assist you accordingly.<|im_end|>\n",
      "----2----\n",
      "<|im_start|>system\n",
      "You are an expert sentiment classifier.\n",
      "Your task is to classify a sentence's sentiment as 'Positive' or 'Negative'.\n",
      "The user will provide you with a sentence.\n",
      "Format your output as:\n",
      "\n",
      "Classification: Positive or Negative\n",
      "<|im_end|>\n",
      "<|im_start|>user\n",
      "I love advanced NLP!<|im_end|>\n",
      "<|im_start|>assistant\n",
      "Classification: Positive<|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|><|im_end|>\n"
     ]
    }
   ],
   "source": [
    "messages = [\n",
    "    {   \"role\": \"system\",\n",
    "        \"content\": \"\"\"You are an expert sentiment classifier.\n",
    "Your task is to classify a sentence's sentiment as 'Positive' or 'Negative'.\n",
    "The user will provide you with a sentence.\n",
    "Format your output as:\n",
    "\n",
    "Classification: Positive or Negative\n",
    "\"\"\"\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": (\"I love advanced NLP!\")\n",
    "    }\n",
    "]\n",
    "\n",
    "input_text = tokenizer.apply_chat_template(messages, tokenize=False)\n",
    "inputs = tokenizer(input_text, return_tensors=\"pt\")\n",
    "outputs = model.generate(**inputs, max_new_tokens=50, do_sample=True, num_return_sequences=3)\n",
    "for i in range(len(outputs)):\n",
    "    print(f\"----{i}----\")\n",
    "    print(tokenizer.decode(outputs[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Approach 2: provide examples (either in the system prompt or as a sequence of messages)\n",
    "\n",
    "We can show the model the desired behavior through example conversations, where the assistant demonstrates the correct format and task execution. This is analogous to the few-shot examples we saw earlier, but using the chat format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----0----\n",
      "<|im_start|>system\n",
      "You are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n",
      "<|im_start|>user\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "Sentence: 'This is such a cool lecture!'<|im_end|>\n",
      "<|im_start|>assistant\n",
      "Classification: Positive<|im_end|>\n",
      "<|im_start|>user\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "Sentence: 'I really don't like the last scene.'<|im_end|>\n",
      "<|im_start|>assistant\n",
      "Classification: Negative<|im_end|>\n",
      "<|im_start|>user\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "Sentence: 'I love advanced NLP!'<|im_end|>\n",
      "<|im_start|>assistant\n",
      "Classification: Positive<|im_end|>\n",
      "----1----\n",
      "<|im_start|>system\n",
      "You are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n",
      "<|im_start|>user\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "Sentence: 'This is such a cool lecture!'<|im_end|>\n",
      "<|im_start|>assistant\n",
      "Classification: Positive<|im_end|>\n",
      "<|im_start|>user\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "Sentence: 'I really don't like the last scene.'<|im_end|>\n",
      "<|im_start|>assistant\n",
      "Classification: Negative<|im_end|>\n",
      "<|im_start|>user\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "Sentence: 'I love advanced NLP!'<|im_end|>\n",
      "<|im_start|>assistant\n",
      "Classification: Positive<|im_end|>\n",
      "----2----\n",
      "<|im_start|>system\n",
      "You are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n",
      "<|im_start|>user\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "Sentence: 'This is such a cool lecture!'<|im_end|>\n",
      "<|im_start|>assistant\n",
      "Classification: Positive<|im_end|>\n",
      "<|im_start|>user\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "Sentence: 'I really don't like the last scene.'<|im_end|>\n",
      "<|im_start|>assistant\n",
      "Classification: Negative<|im_end|>\n",
      "<|im_start|>user\n",
      "Classify the sentence's sentiment as 'Positive' or 'Negative':\n",
      "Sentence: 'I love advanced NLP!'<|im_end|>\n",
      "<|im_start|>assistant\n",
      "Classification: Positive<|im_end|>\n"
     ]
    }
   ],
   "source": [
    "messages = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"Classify the sentence's sentiment as 'Positive' or 'Negative':\\nSentence: 'This is such a cool lecture!'\"\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"assistant\",\n",
    "        \"content\": \"Classification: Positive\"\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"Classify the sentence's sentiment as 'Positive' or 'Negative':\\nSentence: 'I really don't like the last scene.'\"\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"assistant\",\n",
    "        \"content\": \"Classification: Negative\"\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"Classify the sentence's sentiment as 'Positive' or 'Negative':\\nSentence: 'I love advanced NLP!'\"\n",
    "    }\n",
    "]\n",
    "\n",
    "input_text = tokenizer.apply_chat_template(messages, tokenize=False)\n",
    "inputs = tokenizer(input_text, return_tensors=\"pt\")\n",
    "outputs = model.generate(**inputs, max_new_tokens=50, do_sample=True, num_return_sequences=3)\n",
    "for i in range(len(outputs)):\n",
    "    print(f\"----{i}----\")\n",
    "    print(tokenizer.decode(outputs[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Chain-of-thought with base model\n",
    "\n",
    "Prompting the model to \"think step by step\" can elicit intermediate reasoning steps before the final answer. Even base models can exhibit this behavior when prompted appropriately, though the reasoning may be flawed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = \"HuggingFaceTB/SmolLM2-360M\"\n",
    "\n",
    "tokenizer = AutoTokenizer.from_pretrained(model)\n",
    "model = AutoModelForCausalLM.from_pretrained(model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Q: On average Joe throws 25 punches per minute. \n",
      "A fight lasts 5 rounds of 3 minutes. \n",
      "How many punches did he throw?\n",
      "A: Let's think step by step.\n",
      "First, we need to find the number of rounds.\n",
      "We know that 1 round = 3 minutes, so we can write this as:\n",
      "1 round = 3 minutes\n",
      "Now, we can find the number of minutes in a round.\n",
      "We know that 1 minute = 60 seconds, so we can write this as:\n",
      "1 minute = 60 seconds\n",
      "Now, we can find the number of seconds in a minute.\n",
      "We know that 1 second = 60 minutes, so we can write this as:\n",
      "1 second = 60 minutes\n",
      "Now, we can find the number of seconds in a round.\n",
      "We know that 1 round = 3 minutes, so we can write this as:\n",
      "1 round = 3 minutes\n",
      "Now, we can find the number of seconds in a minute.\n",
      "We know that 1 minute = 60 seconds, so we can write this as:\n",
      "1 minute = 60 seconds\n",
      "Now, we can find the number of seconds in a round.\n",
      "We know that 1 round = 3 minutes, so we can write this as:\n",
      "1 round = 3 minutes\n",
      "Now, we can find the number of seconds in a minute.\n",
      "We know that 1 minute = 60 seconds, so we can write this as:\n",
      "1 minute = 60 seconds\n",
      "Now, we can find the number of seconds in a round.\n",
      "We know that 1 round = 3 minutes, so we can write this as:\n",
      "1 round = 3 minutes\n",
      "Now, we can find the number of seconds in a minute.\n",
      "We know that 1 minute = 60 seconds, so we can write this as:\n",
      "1 minute = 60 seconds\n",
      "Now, we can find the number of seconds in a round.\n",
      "We know that 1 round = 3 minutes, so we can write this as:\n",
      "1 round = 3 minutes\n",
      "Now, we can find the number of seconds in a minute.\n",
      "We know that 1 minute = 60 seconds, so we can write this as:\n",
      "1 minute = 60 seconds\n",
      "Now, we can find the number of seconds in a round.\n",
      "We know that 1 round = 3 minutes, so we can write this as:\n",
      "1 round = 3 minutes\n",
      "Now, we can find the number of seconds in a round.\n",
      "We know that 1 round =\n",
      "====\n"
     ]
    }
   ],
   "source": [
    "prompts = [\n",
    "    \"\"\"Q: On average Joe throws 25 punches per minute. \n",
    "A fight lasts 5 rounds of 3 minutes. \n",
    "How many punches did he throw?\n",
    "A: Let's think step by step.\"\"\",\n",
    "]\n",
    "\n",
    "for prompt in prompts:\n",
    "    inputs = tokenizer(prompt, return_tensors=\"pt\")\n",
    "    outputs = model.generate(\n",
    "        **inputs, \n",
    "        pad_token_id=tokenizer.eos_token_id,\n",
    "        max_new_tokens=512,\n",
    "        temperature=0.4,\n",
    "        do_sample=True\n",
    "    )\n",
    "    print(tokenizer.decode(outputs[0]))\n",
    "    print(\"====\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Chain-of-thought with instruct model\n",
    "\n",
    "Many instruction-tuned models can solve problems step-by-step when asked, as they've typically been trained to follow such instructions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = \"HuggingFaceTB/SmolLM2-360M-Instruct\"\n",
    "\n",
    "tokenizer = AutoTokenizer.from_pretrained(model)\n",
    "model = AutoModelForCausalLM.from_pretrained(model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<|im_start|>system\n",
      "You are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n",
      "<|im_start|>user\n",
      "Solve the problem:\n",
      "On average Joe throws 25 punches per minute. \n",
      "A fight lasts 5 rounds of 3 minutes. \n",
      "How many punches did he throw?<|im_end|>\n",
      "<|im_start|>assistant\n",
      "To solve this problem, we need to calculate the total number of punches Joe threw in the fight.\n",
      "\n",
      "First, we need to find out how many rounds there are in the fight. Since the fight lasts 5 rounds of 3 minutes each, we can calculate the total number of minutes in the fight.\n",
      "\n",
      "There are 3 minutes in each round, so the total number of minutes in 5 rounds is:\n",
      "3 minutes x 5 rounds = 15 minutes\n",
      "\n",
      "Now, we can calculate the total number of punches Joe threw in the fight:\n",
      "Total number of punches = Total number of minutes in the fight\n",
      "Total number of punches = 15 minutes\n",
      "\n",
      "Therefore, Joe threw 15 punches in the fight.<|im_end|>\n"
     ]
    }
   ],
   "source": [
    "messages = [\n",
    "    {\n",
    "        \"role\": \"user\", \n",
    "        \"content\": \"\"\"Solve the problem:\n",
    "On average Joe throws 25 punches per minute. \n",
    "A fight lasts 5 rounds of 3 minutes. \n",
    "How many punches did he throw?\"\"\"\n",
    "    }\n",
    "]\n",
    "\n",
    "input_text = tokenizer.apply_chat_template(messages, tokenize=False)\n",
    "inputs = tokenizer(input_text, return_tensors=\"pt\")\n",
    "outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.4, do_sample=True)\n",
    "print(tokenizer.decode(outputs[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Program-aided reasoning\n",
    "\n",
    "Instead of natural language reasoning, we can prompt the model to solve problems by writing and executing code. This leverages the model's code generation capabilities and the code executor's accurate computations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<|im_start|>system\n",
      "You are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n",
      "<|im_start|>user\n",
      "Solve the problem by writing a Python program:\n",
      "On average Joe throws 25 punches per minute. \n",
      "A fight lasts 5 rounds of 3 minutes. \n",
      "How many punches did he throw?<|im_end|>\n",
      "<|im_start|>assistant\n",
      "Here's a Python program that solves the problem:\n",
      "\n",
      "```python\n",
      "def calculate_punches(minutes_per_punch):\n",
      "    punch_time = 5  # 5 rounds x 3 minutes per round = 15 minutes\n",
      "    punch_per_minute = 25  # Joe throws 25 punches per minute\n",
      "    total_punches = punch_time * punch_per_minute\n",
      "    return total_punches\n",
      "\n",
      "minutes_per_punch = 25\n",
      "minutes_in_round = 15\n",
      "\n",
      "total_punches = calculate_punches(minutes_per_punch)\n",
      "print(f\"Joe threw {total_punches} punches in one fight.\")\n",
      "```\n",
      "\n",
      "This program takes in the average punching time (minutes per punch) and the number of rounds in a fight, and then calculates the total number of punches Joe threw. The result is then printed out.<|im_end|>\n"
     ]
    }
   ],
   "source": [
    "messages = [\n",
    "    {\n",
    "        \"role\": \"user\", \n",
    "        \"content\": \"\"\"Solve the problem by writing a Python program:\n",
    "On average Joe throws 25 punches per minute. \n",
    "A fight lasts 5 rounds of 3 minutes. \n",
    "How many punches did he throw?\"\"\"\n",
    "    }\n",
    "]\n",
    "\n",
    "input_text = tokenizer.apply_chat_template(messages, tokenize=False)\n",
    "inputs = tokenizer(input_text, return_tensors=\"pt\")\n",
    "outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.4, do_sample=True)\n",
    "print(tokenizer.decode(outputs[0]))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "anlp",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}