Large language models can augment data analysis, crunching more information and identifying deeper insights than data professionals can on their own. But LLMs are unreliable tools for non-analysts, and even trained analysts should use them cautiously.
Off-the-shelf LLMs are not ready to perform the role of a data analytics tool. They can't accurately or consistently answer detailed questions about the meanings of data sets. Automated LLM functions require training on the correct data sets to generate the most accurate results; it's up to analysts to ensure that outputs are secure, accurate and ethically sound.
How LLMs improve data analysis
LLMs can analyze structured numerical data. They can calculate statistics, look for trends and identify anomalies. But data analysts should only use LLMs for this type of data if they limit the tool to look only at data in specific files and confine answers to material within those files.
Text analysis
An even better use of LLMs is to take advantage of their facility with language. Data analysts can use an LLM to accelerate text analysis and -- if it is multimodal and can interpret spoken language -- oral inputs. LLMs can transcribe spoken word inputs, translate languages and analyze the results by doing the following:
Highlighting categories of words.
Looking for commonalities among comments such as similar language, or references to the same people or things.
Providing semantic scoring of inputs based on defined categories of words. For example, being angry, despairing, engaged or credulous.
Pointing out contextual information associated with use of specific words or images.
Visual media analysis
If an LLM is trained with the ability to parse visual media, it can analyze the content of pictures, charts and videos. LLMs can follow straightforward prompts, such as looking for a specific kind of object -- say, how many hats are in a picture. They can also identify subtle elements, such as the prevalence of different color palettes across TikTok videos with a given hashtag.
LLMs can help analysts combine inherently unstructured data with structured data sets by converting free text, audio or video media into specific numerical data. Trained multimedia LLMs can also generate visualizations of data sets, ranging from traditional line or bar charts to word clouds and heat maps.
Predictive analytics. LLMs can allow analysts to analyze non-textual data, and, critically, to integrate those results with the analysis of standard numerical data. The combination broadens the reach of predictive analytics by enabling it to spot more trends. For example, an LLM would be able to identify patterns across media platforms more easily than human analysts could.
LLMs can tease out words in the data, understand the context of words and assign sets of words to themes. Using that information, analysts can then adjust the LLM model training for subsequent predictive analytics operations.
Examples of data analysis with LLMs
Analysts can use LLMs to provide insights and improve business operations in several ways.
Identify actions based on customer feedback
LLMs can help analysts understand patterns and trends in customer sentiment about specific products, services, store locations or staff. Analysts can use LLMs to analyze data from fixed-choice surveys, emails to sales teams and support desks, support and sales chat logs, social media postings and content from podcasts or product review videos. Companies can use the results to improve their operations and address shortcomings in their service or product portfolios.
Data analysts will not see LLMs replace their jobs anytime soon.
Identify business development opportunities
Analysts using LLMs can harvest data from sources including competitor websites, relevant social media channels and their own support channels. The information can help identify new product or service opportunities, open new stores or websites, move into new locations or form new partnerships with other companies.
Identify potential threats
Governments and corporations can analyze the content of material posted in public. The information can help augment data from other sources to spot potential security threats.
Data professionals must learn more about what LLMs can do and how to keep the models honest. In the meantime, LLMs will be able to assist analysts, but not replace them. Organizations need analysts to craft prompts carefully and verify the accuracy of all outputs -- that is, until developers can consistently produce LLMs that don't make up data, present made-up data as real or make up conclusions not driven by the data.
John Burke is CTO and principal research analyst with Nemertes Research. With nearly two decades of technology experience, he has worked at all levels of IT, including end-user support specialist, programmer, system administrator, database specialist, network administrator, network architect and systems architect. His focus areas include AI, cloud, networking, infrastructure, automation and cybersecurity.
