knowledge engineering
What is knowledge engineering?
Knowledge engineering is a field of artificial intelligence (AI) that tries to emulate the judgment and behavior of a human expert in a given field.
Knowledge engineering is the technology behind the creation of expert systems to assist with issues related to their programmed field of knowledge. Expert systems involve a large, expandable knowledge base integrated with a rules engine that specifies how to apply information to each particular situation.
Knowledge engineers may create a system that incorporates machine learning so that it can learn from experience in the same way humans do. Expert systems are used in various fields, including healthcare, customer service, financial services, manufacturing and law.
How does knowledge engineering work?
Knowledge engineering is a process designed to solve complex problems. The process uses algorithms to emulate the thought process and patterns of a human subject matter expert. It handles questions and issues as a human expert would, making decisions in the same way human decisions are made.
A knowledge engineering system looks at the structure of a task done or decision made by a human. It studies how the conclusion is reached and resolves the issue or question using a library of problem-solving methods and a body of collateral knowledge. Collateral knowledge is information that is not central to the given issue but is still needed to make judgments.
Knowledge engineering systems can assist with the following tasks:
- troubleshooting
- solving issues
- assisting a human
- acting as a virtual agent
These systems require an extensive amount of collateral knowledge to enable analogous reasoning and nonlinear thought. They use a modeling approach, drawing on different sets of knowledge and processes to reach the same conclusion for a given question or issue. Eventually, it is expected that knowledge engineering systems will have virtual expertise that surpasses the abilities of its human counterparts.
Knowledge engineering generally involves these five steps:
- Knowledge is gathered from various sources, such as text, human experts, big data repositories and sensors.
- Knowledge is verified using test cases that human experts run to ensure they're correct.
- Knowledge is organized, encoded and provided in a knowledge base.
- Software makes inferences based on the organized and encoded knowledge.
- An explanation is devised to explain the basis for a certain conclusion.
The most important step in the process is to ensure that the knowledge base is accurate and timely. Another important aspect of the knowledge engineering process is to have a human agent who ensures the system is doing its job. The engineer will develop rules for the system so it functions like a human and reaches the same conclusions as a human expert.
Why is knowledge engineering important?
Knowledge engineering is important for several reasons, including the following four:
- Speeds decision-making. The AI capabilities of these systems speeds up information processing and decision-making. A knowledge engineering system can identify a task and work toward a logical conclusion using its expertise.
- Handles large data sets. As organizations deal with increasingly large quantities of data, they need a way to efficiently process it and make decisions, which knowledge engineering provides.
- Develops expert systems. Knowledge engineering results in the expert systems used in many industries, including medicine, engineering and finance. For example, financial expert systems provide advice on investment decisions, portfolio management and risk assessment.
- Creates decision-support systems. Knowledge engineering provides the information and tools people need to make better decisions. Decision-support systems are used in various fields, including healthcare, manufacturing and retail.
Knowledge engineering career path
Knowledge engineers design and maintain expert systems through knowledge engineering. Jobs in this field require a Bachelor of Science in a field such as computer science or mathematics.
A knowledge engineer must have expertise in AI language and knowledge representation. They also must be a domain expert in a particular problem area. For example, a knowledge engineer designing an expert system in the medical field should have prior experience in medicine or healthcare. An engineer building an expert system for a financial application should have work experience and expertise in finance and banking.
Many knowledge engineers hold master's degrees and have experience in the field of the domain they specialize in.
Relevant skill sets for a knowledge engineer include the following:
- AI
- customer experience
- data analysis
- data classification
- information management
- natural language processing
- machine learning
- search engines
- software programming
- systems design
Knowledge engineers also need to have good oral and written communication skills. The job requires contributing to business decisions related to the projects the knowledge engineer is working on and helping make decisions on product development in a continuous improvement framework.
Evolution of knowledge engineering
Scientists doing early work on knowledge engineering used the transfer approach, where they tried to transfer human knowledge and decision-making processes into a program. Those programs were designed to follow those processes to reach the same conclusion as a human expert would in the same situation. However, early developers found that the expertise a specialist required to answer questions or respond to issues needed too much collateral knowledge.
These Knowledge-based systems also could not properly emulate the intuition and gut feeling that humans use to make decisions. These sorts of high-level reasoning tasks are currently impossible for machines to recreate in virtual form.
Modern knowledge engineering seeks to reach the conclusions that human experts would reach but it doesn't necessarily follow the same process as a person. It aims to reach these conclusions as or more efficiently than a human would be able to.
One challenge that remains is to enable knowledge engineering systems to adapt to unpredictability the way human experts naturally can. The process that modern knowledge engineering systems use doesn't include as much adapting to predictability and nonlinear thinking as a human expert would. Instead, they focus on the linear, algorithmic, explainable method of problem solving and "thinking" at which computers excel.
Despite AI's limitations, engineers strive to make it capable of the higher-level thought processes only humans currently possess. Learn how neuro-symbolic AI pushes the field forward to achieve a crucial milestone, combining symbolic reasoning with machine learning technology.
