ChatGPT has made ripples in the artificial intelligence community since its release, quickly amassing over 100 million users. ChatGPT’s outstanding performance and adaptability are made possible by a skillfully concealed art within its code: quick engineering.
Introduced in 2022, Generative AI’s disruptive potential was highlighted by DALL-E, MidJourney, and StableDiffusion. Later in 2022, though, it was Open AI’s ChatGPT that stole the show. And there was no letting up in sight from this forward motion.
The AI arms race has been given a huge boost by Google’s introduction of Bard and Meta’s Lamma 2 reaction to OpenAI’s ChatGPT. Inputting information into these models allows us to shape their actions and reactions. Because of this, we are all essentially trained to act as timely engineers. People in the tech sector have taken note. Startups like Vellum AI that are working on rapid engineering are receiving significant funding from venture investors. Forbes also notes that there is a booming and desirable employment market for timely engineers, with wages that may approach $300,000.
In this piece, we will explain what prompt engineering is and how it relates to ChatGPT. We’ll go into the fundamentals, investigate the methods, and look at how they’re being used in the business world.
Let’s get a feel for how ChatGPT works and the need for timely engineering before diving in headfirst. Some of these cutting-edge methods will be covered extensively in subsequent publications, so don’t miss out!
The Mechanics Behind ChatGPT Prompt Engineering
The secret to ChatGPT’s success is its capacity to comprehend and imitate the subtleties of human communication. In order to ensure generalisation over specificity, the model is trained on a wide variety of online content, but importantly, it is not aware of any particular documents or sources in its training set. This method of training enables ChatGPT to produce original replies, handle challenging conversations, and even have a sense of humour. However, it’s crucial to keep in mind that ChatGPT just produces replies based on patterns it discovered during training and neither actually understands nor has any opinions.
ChatGPTs are very flexible tools because of their capacity to comprehend and react to human language in a sophisticated and context-aware manner.
These models’ mechanics are based on the idea of “tokens,” which are discrete language chunks that can be anything from a single letter to a whole word. These models anticipate the subsequent sequence of expected tokens by processing a certain number of tokens at a time (4096 for GPT-3.5-Turbo or 8192 or 32768 for GPT-4). The models use sophisticated linear algebra to analyse the tokens and forecast the next token that is most likely to occur.
For instance, a straightforward statement of nine words may become 10 tokens. The model can produce several tokens from a single difficult word, making it easier for it to comprehend and generate language. Tokenization is used to manage even languages with complicated word structures, such as German.
Tokens are generated one at a time, starting with the first token and going all the way to the last, using GPT (Generative Pre-trained Transformer) models. The model analyses the full sequence once again after producing each token before moving on to produce the next. The production of text resembles a large for-loop since this iterative procedure continues until the last token is produced.
The quality of these replies, however, greatly depends on the stimuli it is given. Using ChatGPT takes a certain amount of ability and knowledge, much like directing a discussion with a human. Prompt engineering can help in this situation.
What is Prompt Engineering?
Prompt engineering is the skill of creating exact, usable prompts or input to direct AI (NLP/Vision) models like ChatGPT towards producing the most economically advantageous, precisely helpful, and securely produced outputs.
Prompt engineering has several uses in the field of AI and is not just limited to text production. It is used more frequently in fields like robotic process automation bots, 3D assets, scripts, robot instructions, and other forms of digital media. It offers a distinctive fusion of reasoning, code, creativity, and, occasionally, special modifiers.
While a prompt may contain natural language text, pictures, or other sorts of input data, the results produced by various AI services and tools might differ greatly. Each tool includes unique modifiers that specify the word count, writing style, viewpoint, layout, or other aspects of the intended answer.
This field is crucial for improving existing generative AI tools and developing better AI-powered services. To run a customer-facing chatbot or to perform activities like drafting industry-specific contracts, enterprise developers, for instance, frequently use prompt engineering to customise Large Language Models (LLMs) like GPT-3.
The ability of AI models to produce highly focused, practical answers in a variety of circumstances depends critically on this iterative process of fast refinement and performance measurement.
Becoming a Prompt Engineer
The ability of the GPT-4 model to properly understand complicated instructions and resolve challenging situations makes it a priceless tool. A quick engineer’s involvement in maximising efficiency and cost-effectiveness might depend on their awareness of the many ways to leverage this model’s capabilities.
GPT models like GPT-4 may be interacted with in essentially two different ways. The OpenAI API is one method, where charges are determined based on input and output tokens. In this case, the price per 1K tokens might vary depending on the size of the context. For instance, the price may increase to $0.06 for input tokens and $0.12 for output tokens in a bigger 32K context. Thus, the cost of utilisation can go up fast when servicing a huge amount of queries.
As an alternative, ChatGPT uses the GPT-4 concept and runs on a subscription-based system.
Model parameter tweaking is an essential part of prompt engineering. Adjusting these factors will change the model’s ability to predict the future. Prompt engineers can enhance the precision and quality of the model’s predictions, making them more accurate and useful in the given environment.
Despite the fact that many prompt engineers have technical backgrounds, people with a variety of professional backgrounds can pursue careers in the area because to its interdisciplinary character. Writing specialists, academics, and even artists are increasingly using their specialties to improve the performance of AI models. This transition is also evident in the employment market, where an increasing number of businesses are looking for swift engineers with a variety of backgrounds and talents.
Prompt Design and Engineering in ChatGPT
The performance of language models may be greatly improved by prompt design and engineering, which goes beyond just creating questions. These processes need a thorough grasp of the AI model as well as a highly iterative and polished approach.
Prompt Design
Fundamentally, prompt design is the art and science of developing the ideal prompt for a specific large language model (LLM), such as ChatGPT, to accomplish a specific aim. It’s a mixture of
- Understanding of the LLM: Different language models may respond variably to the same prompt. Moreover, certain language models may have unique keywords or triggers, which they interpret in a specific way.
- Domain Knowledge: Expertise in the relevant field is crucial when designing prompts. For instance, generating a prompt to infer a medical diagnosis would require medical knowledge.
- Iterative Approach and Quality Measurement: The process of creating the ideal prompt often involves trial and error. Therefore, having a way to evaluate the output quality beyond just subjective judgment is vital, particularly when using the prompt at a larger scale.
Prompt Engineering
Prompt engineering is an extended realm of prompt design that includes several critical processes:
- Design of Prompts at Scale: This process includes the design of meta prompts (prompts that generate other prompts) and prompt templates, which are parameterized prompts that can be instantiated at runtime.
- Tool Design and Integration: Prompts can sometimes include results from external tools, and integrating these tools seamlessly into the system is crucial.
- Workflow, Planning, and Prompt Management: Deploying an LLM application, like a chatbot, often requires managing libraries of prompts, planning and choosing the right prompts, and efficiently integrating various tools.
- Evaluation and Quality Assurance of Prompts: This aspect includes defining metrics and processes to evaluate the prompts both automatically and with human involvement.
- Prompt Optimization: The cost and latency of the AI model can depend on the choice of the model and the length of the prompt (number of tokens).
Many automated prompt design approaches, tools, and frameworks have been developed to manage prompts at scale. However, it’s important to understand that none of these tools can replace the nuanced understanding, judgment, and experience of a seasoned prompt engineer.
Prompt Engineering: Techniques and Best Practices
1) Prompt Placement and Description
Placing instructions at the outset of the prompt can significantly impact the AI’s understanding and response. Consider the task of summarizing a text in a bullet point format. A less effective approach would be:
User: Summarize the text below as a bullet point list of the most important points.
{text input here}
On the other hand, a more effective technique could be:
User: Summarize the text below as a bullet point list of the most important points.
Text: “””
{text input here}
“””
By clearly separating the instructions and the context with symbols such as ### or “””, the AI model can better delineate its task and the text it needs to process, thereby giving more accurate and relevant responses.
Secondly, being specific, descriptive, and detailed about the context, expected outcome, length, format, style, and so forth can significantly enhance the precision of AI responses. Suppose you need to generate a brief overview of a scientific paper in an engaging, layperson-friendly style. A vague instruction like “Summarize the scientific paper” would yield less effective results compared to a specific, detailed prompt:
User: Generate a brief (approx. 200 words), engaging summary of the following scientific paper. The summary should be understandable to someone with no scientific background, while accurately reflecting the paper’s key findings.
Paper: “””
{scientific paper text here}
“””
2) Persona Pattern
The Persona Pattern is a strategy that sets the AI to function from a certain perspective, such as a professional role or fictional character. This tactic enables the model to provide outputs related to the specific roles the persona would handle. For instance, one could instruct the AI to act as a cybersecurity expert during a code review. This pattern is particularly useful when users need assistance but are unsure about the exact details required in the output.
Take the following as an example:
User: “Assume the persona of a senior historian specializing in the French Revolution. Given that persona, explain the key events and reasons leading to the downfall of the French monarchy.”
In this scenario, the AI is expected to respond with an in-depth analysis of the French Revolution, focusing on the fall of the monarchy, as a historian would.
Introducing New Information
ChatGPT doesn’t possess knowledge beyond its training cutoff, hence it can’t provide information on recent events or updates post it’s training. However, you can introduce new information to the model via your prompts.
User: “What is cryptocurrency?”
ChatGPT: “Cryptocurrency is a type of digital or virtual currency that uses cryptography for security. The most well-known cryptocurrency is Bitcoin.”
Now, if we add a bit more context to the prompt, we can make the explanation even more intuitive:
User: “Explain the concept of cryptocurrency to a high school student studying digital finance”
ChatGPT: “Imagine you’re playing an online video game and there’s game money you can earn to buy upgrades or new characters. Cryptocurrency is kind of like that, but instead of being limited to the game, you can actually use it to buy real-world goods and services. It’s like digital money that no bank or government controls.”
3) Prompt Size Limitations
Understanding the size limitation of ChatGPT is essential as it directly impacts the volume and type of information we can input. Language models are not built to process unlimited data at once. They have an inherent constraint on the size of the prompt we can create and input. This limitation has profound implications for the design and execution of the prompts.
ChatGPT has a token limit (generally 2048 tokens), which includes both the prompt and the generated response. This means long prompts can limit the length of the response. Therefore, it’s important to keep prompts concise yet informative.
In practical usage, we must act as editors, selectively choosing the most relevant information for the task at hand. Imagine writing a paper or an article with a word or page limit – you can’t just dump random facts, but carefully select and structure information relevant to the subject.
User: “Given the token limitations of the model, summarize the key events of World War II in less than 1000 words.”
By acknowledging the model’s token limitations, this prompt directs the AI to provide a concise yet comprehensive summary of World War II.
Prompts for Repeated Use: In a conversational scenario, you can re-use prompts or refine them based on previous responses. This gives a conversational flow and maintains context in the interaction.
4) Question Refinement Pattern
ChatGPT can also help refine your questions. For example, if you ask a question, it can suggest a better-formulated question for more accurate results.
The Question Refinement Pattern involves the LLM refining the questions asked by the user. It’s particularly useful when users are not experts in a domain or are unsure of how to phrase their question.
An example prompt may look like this:
User: “Whenever I ask a question about data science, suggest a more refined question considering statistical analysis specifics and ask if I want to proceed with the refined question.”
Here, the AI is expected to refine the user’s general data science questions to more detailed questions considering statistical analysis aspects.
5) Cognitive Verifier Pattern
The Cognitive Verifier Pattern breaks down complex questions into smaller, manageable sub-questions. The answers to these sub-questions are then combined to provide a comprehensive answer to the main question. This approach is useful when initial questions are too high-level or when additional clarity is needed to provide a complete answer.
For instance:
User: “If I ask a question about climate change, break it down into three smaller questions that would help you provide a more accurate answer. Combine the answers to these sub-questions to give the final answer.”
In this scenario, the AI will break down the complex question about climate change into three manageable sub-questions. After receiving answers to these sub-questions, the AI will then combine these to provide a more comprehensive and accurate response.
This pattern involves generating multiple questions to obtain a more accurate final answer. It’s a process of verification and consolidation of information for a robust final output
6) Few-Shot Prompting Approach
Few-shot prompting is a powerful approach for teaching AI models to follow specific patterns or carry out tasks. The idea is to feed the model with a number of examples before asking the desired question. This helps to set the desired behavior and guide the model’s responses.
Consider a scenario where a user is attempting to draft a concise, compelling marketing copy for a new tech gadget:
User: “Here are some examples of tech marketing copies: ‘Experience music like never before with our cutting-edge wireless headphones.’ ‘Capture your world in stunning 4K with our ultra-sleek action camera.’ Given these examples, draft a marketing copy for our new AI-powered smartwatch.”
ChatGPT: “Empower your lifestyle with our AI-powered smartwatch, your companion for health, connectivity, and productivity.” The remarkable feature of few-shot learning is that it doesn’t necessitate a detailed description of how to perform the task. Instead, it requires demonstrations of what’s being done, which can make it a good fit for many applications.
Furthermore, this technique extends beyond simple input-output pairing. The model can also be taught to handle intermediate steps to achieve a goal.
7) Chain-of-Thought Prompting
In their recent research paper, “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models” authors Jason Wei and team aim to stimulate reasoning within large language models. Rather than using traditional, direct prompts, the team makes use of a series of interconnected prompts, forming a ‘chain-of-thought.’
For instance, a user might be interested in the potential impact of artificial intelligence on job markets:
User: “Could you briefly explain the concept of artificial intelligence?”
ChatGPT: “Artificial Intelligence, or AI, refers to the capability of a machine to imitate intelligent human behavior. It’s about designing machines that can think and learn like humans, enabling them to perform tasks such as language translation, decision-making, and problem-solving, among others.”
User: “How is AI influencing the current job market?”
ChatGPT: “AI is both creating new jobs and changing the nature of existing ones. On one hand, it automates routine tasks, leading to job displacement in certain sectors. On the other hand, it’s creating demand for roles that didn’t previously exist, such as AI ethics officers, data analysts, and machine learning engineers. Additionally, AI can augment human roles, allowing people to accomplish tasks more efficiently and accurately.”
Chain-of-thought prompting follows a progression of related prompts, maintaining a ‘conversation’ with the model that guides it towards more precise and contextually appropriate responses.
Understanding ChatGPT API and Prompt Engineering from a Developer’s Perspective
From a developer or researcher’s perspective, prompt engineering is a more complex process. It involves the design of algorithms and techniques to help the AI model understand and respond to prompts more effectively. This could include strategies for handling ambiguous prompts, identifying the context of prompts, or chaining prompts together to stimulate more complex thought processes in the AI
In the above illustration, a user interacts with a chat interface, powered by GPT-4. Their input is enhanced for clarity and contextual consistency by a specialized module before being fed to the AI model. The role of this module is to refine and enhance the user’s input for better understanding while also maintaining the context of the conversation. The AI’s response, crafted based on the refined prompt, is returned to the user through the chat interface. The interaction history is updated consistently, maintaining the conversational context. Overall, this diagram illustrates a dynamic user-AI conversation flow enabled by prompt engineering techniques.
The ChatGPT API’s interface is engineered with various hyperparameters that enable users to refine the AI’s responses to prompts, making them more effective and versatile. These include the Temperature, Maximum Length, and Stop Sequences parameters. Let’s explore how they function and can be utilized in different contexts.
1) Temperature
The temperature parameter influences the randomness of the AI’s responses. A higher temperature, such as 1.0, encourages more random output, resulting in diverse but potentially off-topic responses. In contrast, a lower temperature, like 0.2, prompts the AI to select more deterministic responses, which can be beneficial for focused and specific outputs but may lack variety.
Example: If you’re writing a creative story and want unpredictable, imaginative elements, set the temperature to a higher value. A prompt could be: “Write a short story about a time-traveling cat.” With a high temperature, you might get a wild, fantastical tale with unpredictable twists and turns.
2) Maximum Length
This parameter controls the maximum token length of the model’s output, which includes both the tokens in the message input and message output. Setting a higher limit allows for more extended responses, while a lower limit keeps the output short and concise.
Example: For brief answers to trivia questions like, “Who won the Nobel Prize in Literature in 2020?” you might want to set the maximum length to a low value, ensuring the response is concise and direct.
3) Stop Sequences
Stop sequences are specific strings of text where, when the model encounters them, it ceases generating further output. This feature can be useful for controlling the length of the output or instructing the model to stop at logical endpoints.
Example: For a formal letter, you could use “Yours sincerely,” as a stop sequence to ensure the AI does not generate any additional text beyond the proper ending of the letter.
4) Top P
The ‘Top P’ parameter, also known as nucleus sampling, is a method that provides a dynamic selection of the number of words considered at each step of the model’s predictions. A lower value, like 0.5, leads to safer, more focused outputs. A higher value, like 0.9, includes a broader selection of words, leading to more diverse outputs.
Example: If you’re creating an AI to write poems and want it to use a wide array of vocabulary and stylistic phrases, you might set ‘Top P’ to a higher value. A prompt could be: “Compose a poem about autumn’s beauty.”
5) Frequency Penalty
Frequency Penalty controls how much the model should favor less frequent words. A higher penalty (up to 1) encourages the model to use less common words, while a lower value (down to -1) encourages the model to use more common words.
Example: If you’re trying to generate a business proposal and want to avoid jargon, you might set the frequency penalty to a lower value to keep the language clear and accessible.
6) Presence Penalty
The Presence Penalty parameter affects how much the model is penalized for generating new ideas or topics that were not present in the conversation history. Higher values encourage the model to stick to the topics already mentioned, while lower values allow the model to introduce new concepts more freely.
Example: For brainstorming sessions, you might set the presence penalty to a lower value to encourage a diverse array of ideas. You could use a prompt like, “Generate innovative marketing strategies for an online educational platform.”
Industry Application of Prompt Engineering
Customer Support and FAQ Generation
In the world of customer support, the utility of prompt engineering can’t be overstated. One of the most groundbreaking applications of AI in this sector is the advent of AI-powered chatbots. For instance, consider an e-commerce platform like eBay. They utilize an advanced chatbot system that has been fine-tuned with prompt engineering to handle customer inquiries. The chatbot is capable of handling a variety of issues including, but not limited to, providing delivery updates, processing refund requests, and answering queries about product specifications.
When a customer asks, “Where is my order?”, the AI chatbot doesn’t merely respond with a generic message; instead, it taps into the order management system, fetches real-time data, and provides a personalized response like, “Your order #1234 is currently in transit and is expected to be delivered by tomorrow evening.” Such personalized and relevant responses are the direct result of well-engineered prompts, which form the core of these intelligent chatbot systems.
Content Creation and Marketing
In the domain of content creation and marketing, prompt engineering serves as the cornerstone of AI-driven innovations. Companies like Grammarly use AI-powered engines to aid users in creating engaging and grammatically correct content. These platforms work on the basis of prompts, guiding the AI model to generate suggestions or corrections that can enhance the overall quality of the content.
Consider a user aiming to create engaging content about AI advancements. They could employ a prompt like: “Explain the latest advancements in AI in a simple, engaging manner for beginners.” The AI, harnessing this prompt, can draft a concise, reader-friendly piece on complex AI concepts, thus bridging the knowledge gap and stimulating reader interest. This is just one of many ways prompt engineering can enhance content creation.
Coding Assistance and Code Interpretation
Prompt engineering, in this context, is like a two-way conversation between the developer and the AI. The developer provides the AI with a prompt – a description of the problem or functionality they need to implement, and the AI responds with a relevant piece of code or suggestion, aiding the software development process.
Additionally, AI-enhanced coding tools, like GitHub Copilot, have been developed with the expertise of OpenAI. These advanced tools have undergone extensive training to perform complex tasks such as filling out incomplete code files. They do so despite the distinct differences between the process of writing code and the task of completing it.
Education and Personalized Learning
The integration of AI into the education sector has seen a notable surge in recent years, with prompt engineering playing a pivotal role. AI tutors have been developed to provide personalized learning experiences, adapting to each student’s learning pace and style.
Take, for example, an AI-based learning platform like Duolingo. It uses well-engineered prompts to assess a student’s proficiency in a new language and provide suitable exercises. If a student is learning French and struggles with conjugation, the AI tutor can introduce tailored exercises like, “Conjugate the verb ‘manger’ (to eat) in the present tense.” The student’s performance and progress are continually assessed, and the prompts are refined and adapted accordingly, ensuring a personalized and efficient learning experience.
The Unstoppable Rise of Prompt Engineering
As we approach the conclusion of our deep dive into prompt engineering, it’s crucial to underscore how truly nascent this field is. We are at the very precipice of an era where artificial intelligence goes beyond responding to pre-programmed commands, evolving to process and execute carefully engineered prompts that yield highly specific results.
The numerous applications we’ve explored, from customer support to content creation, data analysis, and personalized learning, are just the tip of the iceberg. As research in this arena intensifies, we can look forward to even more sophisticated and nuanced uses of prompt engineering. The convergence of human creativity and AI ingenuity is propelling us towards a future where artificial intelligence will not just assist but transform various aspects of our lives.
