5 skills needed to become a prompt engineer

What is prompt engineering?

In the broadest sense, prompt engineering is the art of figuring out how to ask the best questions to elicit the most meaningful responses from AI. This typically involves strong language skills to use nouns, verbs and other vocabulary arts to query the AI.

Although any generative AI user will effectively perform some of the tasks required from a prompt engineer by simply asking questions of the AI, the engineer delves into the nuances of language input and observes how the AI output responds. This allows the prompt engineer to help refine development of the large language model (LLM) and look for AI limitations, errors and defects that AI developers can address. Prompt engineers can also work to train the AI by helping it to understand how to interpret and deal with various prompts. Consequently, the role of prompt engineer has been described as a mix of programming, instructing and teaching.

To understand the importance of prompts, consider the simple example of a user querying AI about a simple real-world topic such as awards in the entertainment industry. A user asks the following question:

"Who won the movie award?"

This is an ineffective prompt because it's far too broad. The AI might respond by returning every winner of every movie-related award for every year where data is available. It would be an unmanageable list that a user would need to manually parse.

However, the user could ask a more specific question, such as the following:

"What movie won the Academy Award for Best Picture in 2019?"

The AI with adequate data and training could effectively respond with a specific answer. In this case, the movie Green Book won the Academy Award for Best Picture in 2019.

The prompt engineer must know that the AI's answers to questions are correct. If the results are absent, incomplete, unpredictable or unintended, the prompt engineer can train the AI so that it knows the correct answers -- or report issues to the development team for remediation.

For the Academy Award example, a prompt engineer might test the AI by asking the following question:

"What movie won the Academy Award for Best Picture in 1917?"

Since the first Academy Award ceremony was held in 1929, the prompt engineer would pay particular attention to the way in which the AI responds and what explanations it might provide. This is because there is certainly no data to meet the query as the Academy Awards did not exist in 1917.

What does a prompt engineer do?

From an employment perspective, prompt engineering is an abstract form of UI engineering. For example, a traditional UI engineer is responsible for ensuring that the UI is intuitively designed, easy to navigate and provides clear responses or behaviors for users. Generative AI poses different UI challenges because users simply ask the AI for what they want. The UI is the prompt mechanism itself for large language models like GPT. The prompt engineer must understand how the AI works, recognize how the AI will respond given specific prompts and ensure that the AI will deliver meaningful output or responses for any input prompt.

Today's prompt engineer serves a cross-disciplinary role that has the following three major components:

Develop, test and refine AI prompts

The core role of any prompt engineer is direct work with AI platforms to develop new prompts. They test the AI behaviors and outputs resulting from those prompts and work to improve prompts or help to impose AI guardrails to maintain safe or predictable AI output given new or unexpected prompts.

For example, a prompt engineer working with a specific AI platform such as healthcare AI might develop prompts to inquire about patient diagnoses and test how the AI responds when input such as patient names or medical terminology is misspelled. Then, they might make recommendations or changes to refine acceptable prompts or AI responses. In many cases, prompt engineers may be involved in ongoing AI training and refinement.

Collaborate with cross-disciplinary teams

A prompt engineer is typically part of the development team, but serves more of a consulting and quality control role. The prompt engineer will work with developers on the product team designing and coding the AI platform. They'll also be part of the data team, which establishes the data set and trains the AI platform, as well as the business team comprised of project stakeholders.

A prompt engineer is usually tasked with aligning prompts to company goals and user needs. For the healthcare AI example, a prompt engineer might focus work on physician and other caregiver needs and expectations from AI, such as how medical terms are used and resulting patient data and diagnoses are presented.

Analyze and report

Finally, a prompt engineer must understand analytics. They need to monitor and correlate inputs and outputs and establish meaningful metrics to measure the behaviors and performance of the AI platform. These analytics are useful to developers, data scientists and the business team.

GIGO (garbage in, garbage out) is one of the oldest axioms of computer science, but never has it been more relevant than in the AI field. For example, a prompt engineer might analyze AI responses to certain prompt sets and alert the data science team to signs of data bias that could require more training or a review of data content. Similarly, AI prompt responses that indicate data gaps or cause unpredictable results might signal the need for a data review and additional training.

5 skills needed to become a prompt engineer

The skill set needed to qualify for a prompt engineer job is not long, but it can be deceptively broad. This is typical for emerging roles that are still being defined by a rapidly developing industry. Current prompt engineering roles generally ask for a set of skills that include the following five major elements:

• Strong verbal and written communication skills. Unlike many IT roles, prompt engineers must communicate with AI systems through words and phrases. Detailed prompts can be quite complex and involve hundreds, even thousands, of carefully chosen words. In addition, the cross-disciplinary nature of prompt engineering roles makes communication and collaboration critically important.
• Programming proficiency. Although prompt engineering is not exactly programming, it is common for prompt engineers to have some involvement in coding -- whether they participate in development of the AI platform itself, or use the programming skills to automate testing and other functions. This often requires several years of proven experience with well-accepted languages such as Python or its peers. It also helps to have a strong knowledge of APIs, operating systems and CLIs. Precise requirements will depend on the company and the AI platform.
• Prior prompt experience. Given that prompt engineering is a new role, it's difficult to pinpoint a minimum level of prior experience, since the traditional three-to-five-year benchmark just doesn't apply. Still, most employers will look for prompt engineers with some demonstrated experience in building and testing AI prompts, along with experience working with major models such as GPT and platforms such as ChatGPT.
• AI technology knowledge. Prompt engineers rely on language skills, but they still require a comprehensive understanding of natural language processing, LLMs, machine learning and some amount of AI-generated content development. This is particularly important if the prompt engineer will have any hands-on involvement in coding or other AI platform development responsibilities.
• Data analysis experience. Prompt engineers must understand the data provided to an AI platform, the data used in prompts and the data generated or provided by the AI in response. This requires strong knowledge of data analytics techniques and tools. For example, a potential employer may look for several years of experience analyzing structured and unstructured data sources. This knowledge can be essential when looking for data bias or other undesirable data, and will serve to objectively understand the quality of the AI output.

In addition, prompt engineers will require soft skills, such as excellent problem-solving and analytical skills, along with the ability to collaborate effectively with cross-functional teams. Although formal education and certification is not yet widely available, many employers will seek some traditional type of formal education such as a BS in Computer Science, Engineering or another related field.

The future of prompt engineering

The future of prompt engineering is hard to predict, but it's reasonable to expect that prompt engineering will follow a familiar IT industry maturity pattern of diversification, specialization and standardization.

For example, practical or commercial AI such as ChatGPT is still in its infancy, and new LLMs are proliferating quickly. GPT-3.5 has only just been supplanted by GPT-4, and Bing AI has appeared from Microsoft. Other models and platforms will inevitably appear and diversify in the coming years. Prompt engineers will work in the near term to understand and master each model.

Soon, there may be too many models to master, and a few major models will likely dominate. One might expect that prompt engineers may face a period of specialization where the bulk of their experience and expertise rests in just a few of the models likely to appear -- similar to the way that cloud architects might focus on AWS, Google or Azure rather than attempt to master all three.

If this familiar industry pattern holds true for AI, models may eventually experience a period of consolidation as the industry pushes for some forms of standardization. Today, this kind of pattern can be seen in cloud initiatives such as OpenStack, where industry leaders begin collaborating and using open source to create more common and interoperable ways of doing things. Other standardization efforts may involve activities such as AI testing strategies and even a stable of third-party test generation tools. Each step along the way will change the role -- and demand -- for prompt engineers.