Big data analytics

Kafka at center of new event processing infrastructure

Events are as important as data in emerging applications underlying many e-commerce efforts. Streams of events tell a company what motivates customers to use online products. Continue Reading

By

• Jack Vaughan

Data-rich organizations turn focus to ethical data mining

As data analytics has increasingly become a core component of organizations' strategies, concerns have arisen around how data is mined. Experts offer tips. Continue Reading

By

• Beth Stackpole

How modern business intelligence shapes up to big data

Business intelligence and analytics strategies have evolved in recent years in the face of big data. Here, we get a supplier CEO perspective, some analyst counsel, and testimony from Nationwide, Schroders and King Continue Reading

By

• Brian McKenna, Senior Analyst, Business Applications

Equifax and FICO unite to move into alternative data market

With the Equifax-FICO alliance, the credit giants are offering a combined Data Decisions Cloud suite they hope can solidify their place in the alternative data market. Continue Reading

By

• Brian Holak

Tools fill gaps in predictive modeling and machine learning

Machine learning platforms can bring the lone data scientist into the overall workflow. Updates to tooling also let that data scientist use familiar development interfaces. Continue Reading

By

• Jack Vaughan

Qlik Research head talks Associative Engine, NLP and Data Swarm

Elif Tutuk, research head at Qlik, discusses projects her team is working on -- including a smarter Associative Engine, multi-attribute visualizations and NLP. Continue Reading

By

• Brian Holak

Facebook alumni forge own paths to big data analytics tools

Startups Interana and Rockset differ in their approaches to providing new query capabilities on fast-arriving big data. Both are led by technologists who started at Facebook. Continue Reading

By

• Jack Vaughan

Machine learning approaches gain critical mass for data pros

Machine learning will bring change to analytics and data management, said data luminary Michael Stonebraker. Others agree managing such change will take special effort. Continue Reading

By

• Jack Vaughan

Aerospike database garners Spark, Kafka connectors

Apache Kafka and Apache Spark connectors ease use of the Aerospike NoSQL data store in high-speed applications such as analytics that are becoming more broadly supported. Continue Reading

By

• Jack Vaughan

Users find data preparation tools vital to BI strategies

Data preparation isn't the sexiest topic, but it's critically important to IT and business users, according to a study by Dresner Advisory Services. Continue Reading

By

• Brian Holak

Augmented analytics tools, NLP search, graph are trending

Augmented analytics, NLP and graph analytics are changing the data and analytics market, according to Gartner -- enabling users to more easily get at and share data. Continue Reading

By

• Brian Holak

Hazelcast grid tunes for data scalability tradeoffs

An in-memory data grid (IMDG) from Hazelcast lets designers tune subsystems to support consistency over availability, or the reverse, depending on what designers want. Continue Reading

By

• Jack Vaughan

Explore Hadoop distributions to manage big data

Discover the uses of Hadoop distributions and the first steps in evaluating these products, as well as how the merger of rivals Cloudera and Hortonworks affects the market. Continue Reading

By

• David Loshin, Knowledge Integrity Inc.

What to consider when choosing big data file formats

While JSON may be the go-to data format for developers, Parquet, ORC or other options may be better for analytics apps. Learn more about big data file formats. Continue Reading

By

• George Lawton

Originators form group to boost Presto SQL query engine

The Presto engine arose as an alternative to Hive for big data queries. Now, the Presto Software Foundation has formed to promote the SQL query software's virtues. Continue Reading

By

• Jack Vaughan

5 best practices for managing real-time data integration

Real-time data integration isn't like traditional data integration -- "it's moving, it's dirty and it's temporal," cautions one data pro. Experts offer up some best practices. Continue Reading

By

• George Lawton

Expert urges data pros to hone data science skills

IT expert William McKnight shares job tips for data professionals looking to prosper in a changing enterprise. His first piece of advice: continually foster data science skills. Continue Reading

By

• Brian Holak

Demand for data scientists is booming and will only increase

Fueled by big data and AI, demand for data science skills is growing exponentially, according to job sites. The supply of skilled applicants, however, is growing at a slower pace. Continue Reading

By

• Brian Holak

IT managers look to deploy BI, analytics in the cloud

More organizations plan to use cloud-based BI and analytics and focus on data governance and predictive analytics, TechTarget's 2019 IT Priorities Survey found. Continue Reading

By

• Brian Holak

Advantages of graph databases: Easier data modeling, analytics

Graph databases are finding a place in analytics applications at organizations that need to be able to map and understand the connections in large and varied data sets. Continue Reading

By

• Jack Vaughan

Tableau CEO Adam Selipsky talks strategy

Tableau’s CEO, Adam Selipsky, talks strategy and business intelligence trends with Computer Weekly. He asserts enterprise relevance for Tableau, predicts BI tools consolidation and a ubiquitous user base for analytics Continue Reading

By

• Brian McKenna, Senior Analyst, Business Applications

Southern Water’s centralised data team geared for silo busting

Southern Water has centralised its data specialists and overhauled its data management and business intelligence technology to support business decision-making at scale Continue Reading

By

• Brian McKenna, Senior Analyst, Business Applications

Data management trends for 2019: Governance, DataOps, cloud

Better data governance, increased cloud use and wider DataOps adoption head the list of trends for data management teams to plan for in 2019, IT analysts say. Continue Reading

By

• Jack Vaughan

5 tips for enabling citizen data scientists

Self-service analytics tools are enabling citizen data scientists to dig deeper into BI than ever before. Experts offer insights into how to empower data democratization. Continue Reading

By

• Ericka Chickowski

Why data literacy skills still matter with augmented analytics

Democratizing data analytics gives everyone access to tools and information, but data literacy is still required for analyzing data and delivering successful outcomes. Continue Reading

By

• George Lawton

FaunaDB distributed cloud database hunts transactional NoSQL

Startup Fauna has built a "relational NoSQL" database for distributed transaction processing. Data consistency and speedy builds attracted early user ShiftX. Continue Reading

By

• Jack Vaughan

What you need to know about network analytics software

Reality check: Are network analytics tools overhyped? This guide explores the key features of these tools, how they can help and their effect on networking's future. Continue Reading

By

• Michaela Goss, Senior Site Editor

Deep learning in big data analytics with Sisense Hunch

Sisense Hunch, a new Sisense BI tool, enables big data analytics for IoT devices by using deep learning to automatically extract insights from big data. Continue Reading

By

• Mark Labbe

How Morgan Stanley scaled up Tableau self-service analytics

Morgan Stanley wanted to empower users to make data-driven decisions. A Tableau analytics platform was the answer. The problem: introducing it to 30,000 users. Continue Reading

By

• Ericka Chickowski

What data management challenges do analytics programs face?

Expert Anne Marie Smith shares five reasons why organizations' analytics programs might fail and how a data management framework and other programs can help. Continue Reading

By

• Anne Marie Smith, Ph.D., Alabama Yankee Systems, LLC

Analytics for IoT requires a sound strategy for success

IoT analytics projects can supply important insights to aid the business bottom line. But first you must understand the varieties of data and pick the best tool for the job. Continue Reading

By

• Linda Rosencrance

IoT data analytics holds the key to putting data to use

Applying IoT data analysis to info gathered from connected devices is key to extracting the data's potential value. Knowing how to get from potential to realized, though, is the challenge. Continue Reading

By

• Sharon Shea, Executive Editor

10 features to look for in visualization tools for big data

Big data is meaningless if it isn't understandable. Experts explain why users need data visualization tools that offer embeddability, actionability and more. Continue Reading

By

• George Lawton

6 big data visualization project ideas and tools

These data visualization project examples and tools illustrate how enterprises are expanding the use of "data viz" tools to get a better look at big data. Continue Reading

By

• George Lawton

10 tips for implementing visualization for big data projects

Organizations need to keep users and design at the forefront when launching data visualization efforts, according to experts. Find out why colors and sizing matter. Continue Reading

By

• George Lawton

Neo4j graph database targets AI use cases, performance

The Neo4j graph database is poised for use in AI applications, in which understanding data can stymie efforts. Recent updates target AI needs and performance issues. Continue Reading

By

• Jack Vaughan

Cloud buoys data microservices -- for on-premises systems, too

Data in a microservices architecture is percolating anew. This news analysis looks at IBM Cloud Private for Data and other means to harmonize data in public and private locations. Continue Reading

By

• Jack Vaughan

Cloudera-Hortonworks merger narrows Hadoop users' options

Hadoop users will have fewer choices as big data rivals Cloudera and Hortonworks unite. But the new company may be more competitive with AWS and Google. Continue Reading

By

• Jack Vaughan

UCSD graduates to advanced analytics with SAP HANA platform

The University of California, San Diego implemented an SAP HANA platform that uses advanced analytics and machine learning to run operations and help students graduate on time. Continue Reading

By

• Jim O'Donnell, News Writer

Data science teams use business ties to boost data knowledge

To ensure that advanced analytics applications are relevant to business operations, data scientists are collaborating with workers who are experts on business data. Continue Reading

By

• Craig Stedman, Industry Editor

AI for analytics augments and bolsters business intelligence

 Continue Reading

Augmented analytics reveals the hidden side of things

The rise of augmented analytics empowers businesses to find correlations in data that might very well be the critical difference in operational efficiency and cost savings. Continue Reading

By

• Bridget Botelho, Editorial Director, News

Big data platform broadens place in analytics architecture

Big data platforms stumbled a bit getting out of the prototyping stage. But a view from the Strata Data Conference in New York sees broader use in the offing. Continue Reading

By

• Jack Vaughan

Mining equipment maker uses BI on Hadoop to dig for data

BI on Hadoop is still new, but moving BI to data is trending. A data scientist working with IoT data at Komatsu sees the importance of getting big data to the right people. Continue Reading

By

• Jack Vaughan

5 trends driving the big data evolution

The speedy evolution of big data technologies is connected to five trends, including practical applications of machine learning and cheap, abundantly available compute resources. Continue Reading

By

• Mike Matchett, Small World Big Data

Containers key for Hortonworks alliance on big data hybrid

Hortonworks is joining with Red Hat and IBM to work together on a hybrid big data architecture format that will run using containers both in the cloud and on premises. Continue Reading

By

• Jack Vaughan

Using visualizations and analytics in media content

By using the best mass media BI practices, from data visualization to measuring metrics, companies like Condé Nast and Comcast are attracting consumers and raising revenues. Continue Reading

By

• Mark Labbe

Graph processing gives credit analysis firms an edge

A graph database startup's parallel loading, processing and querying combine to deliver real-time data for fintech firms that offer fast credit evaluations online. Continue Reading

By

• Jack Vaughan

Reddit and the aspiring data scientist

For amateur data scientists, Reddit provides the opportunity to post about personal projects, chat with like-minded people and get free advice from a large community. Continue Reading

By

• Mark Labbe

Confluent Platform 5.0 aims to mainstream Kafka streaming

Confluent Platform updates seek to bring data streaming with Apache Kafka to a wider audience. A new GUI and user-defined functions are part of the 5.0 release. Continue Reading

By

• Jack Vaughan

BigQuery ML moves machine learning into Google BigQuery

Google is enabling BigQuery users to build SQL-based machine learning models inside the cloud data warehouse via a BigQuery ML technology now out in beta. Continue Reading

By

• Craig Stedman, Industry Editor

Qlik-Podium acquisition aims to boost BI data management

With its acquisition of Podium Data, Qlik seeks to amplify its enterprise BI data management capabilities and raise the level of competition with Tableau and Power BI. Continue Reading

By

• Shaun Sutner, News Director

Data analytics in government efforts lack structure

Data analytics in government agencies lack organization, focusing on immediate problems instead of attacking underlying causes without a unified strategy or system. Continue Reading

By

• Mark Labbe

Reskilling the analytics team: Math, science and creativity

Technical skills are a must for data scientists. But to make analytics teams successful, they also need to think creatively, work in harmony and be good communicators. Continue Reading

By

• Nipa Basu

Chief data officer role: Searching for consensus

The chief data officer role is about many things -- regulations, innovation, AI and more. Consultant Randy Bean discussed the matter ahead of an MIT symposium on the topic. Continue Reading

By

• Jack Vaughan

Labeled data brings machine learning applications to life

The types of data being collected for analytics use are increasing, but traditional structured data is a good match for machine learning. Gartner's Svetlana Sicular explains why. Continue Reading

By

• Maxine Giza

GPU cloud tools take complexity out of machine learning infrastructure

While talk of AI on GPUs is abuzz, actually building a machine learning infrastructure remains a dark art. A startup's PaaS is looking to automate parts of the process. Continue Reading

By

• Jack Vaughan

Hadoop data governance services surface in wake of GDPR

GDPR influence is touching a Hadoop big data world that was immune to many privacy considerations until now. This podcast features the rise of Hadoop data governance for data lakes. Continue Reading

By

• Jack Vaughan

MongoDB 4.0, Stitch aim to grow NoSQL apps in cloud, on-prem

NoSQL vendor MongoDB upgraded its database software with ACID support, while also releasing a serverless platform intended to simplify application development. Continue Reading

By

• Craig Stedman, Industry Editor

Hadoop data lake architecture tests IT on data integration

Hortonworks users talk about building Hadoop data lakes to support new applications -- and the challenges their teams face on ingesting and refining data for end users. Continue Reading

By

• Jack Vaughan

Hortonworks cloud options grow via Google, Microsoft, IBM

Hortonworks now supports Google Cloud Storage and has also broadened cloud deals with Microsoft and IBM, aiming to increase cloud uses of its big data platform. Continue Reading

By

• Jack Vaughan

Google Cloud data lake fuels cloud payment processing flow

To create a cloud payment processing system, Global Payments first had to deploy a data lake in the Google Cloud. Getting quick user feedback was another early step. Continue Reading

By

• Jack Vaughan

Starburst finds new worlds to conquer with SQL query engine

Relational databases may have hit a wall of late, but the SQL query engine seems poised for wider growth. Starburst, a retro startup of sorts, is among those looking to take it wider still. Continue Reading

By

• Jack Vaughan

New data science platforms aim to be workflow, collaboration hubs

Oracle's acquisition of DataScience.com is shining a spotlight on workbench-style platforms designed to centralize advanced analytics work by teams of data scientists. Continue Reading

By

• Craig Stedman, Industry Editor

AI elevates predictive maintenance for Kone and ThyssenKrupp

Kone and ThyssenKrupp are using artificial intelligence to improve predictive maintenance of their lifts and escalators Continue Reading

By

• Lindsay Clark

GPU databases bring greater parallelism to big data processing

GPU databases offer a new way to process data. 451 Research analyst James Curtis discusses where they fit in big data applications, particularly for parallel processing. Continue Reading

By

• Jack Vaughan

Qlik readies new Qlik Sense features for cloud, AI, big data

Self-service BI vendor Qlik is beginning to add a promised set of advanced functionality to its Qlik Sense software, but the features will be delivered in stages. Continue Reading

By

• Craig Stedman, Industry Editor

Data scientist vs. business analyst: What's the difference?

Data science and business analyst roles differ in that data scientists must deep dive into data and come up with unique business solutions -- but the distinctions don't end there. Continue Reading

By

• Matt Mueller and Ed Burns

Teradata applies time series analytics tools to IoT data

Data warehouse pioneer Teradata looks to ease IoT data analysis with capabilities that address skills gaps on time series analytics techniques, which are coming more to the fore. Continue Reading

By

• Jack Vaughan

IT teams take big data security issues into their own hands

Data security needs to be addressed upfront in deployments of big data systems -- and users are likely to find they have to build some security capabilities themselves. Continue Reading

By

• Craig Stedman, Industry Editor

Information architecture applied to big data streaming, AI

New technologies challenge data professionals, but taking a step back helps with hurdles. In this interview, consultant William McKnight takes a measured look at data streaming, GDPR and AI. Continue Reading

By

• Jack Vaughan

Cloud workloads, data lakes challenge information architecture

Data management options are expanding; cloud workloads are an example. That means changing your approach to information architecture, says data management expert William McKnight. Continue Reading

By

• Jack Vaughan

Nvidia, Pure Storage FlashBlade lift 'AIRI' infrastructure

Pure Storage AIRI is AI-ready infrastructure that integrates Pure's all-flash FlashBlade NAND storage blades and four Nvidia DGX-1 artificial intelligence supercomputers. Continue Reading

By

• Garry Kranz

Oracle Autonomous Database debut heralds new cloud hopes

It wasn't the first in the cloud computing space, but Oracle is all in now. Key to the effort is a new Autonomous Database service, initially available for data warehousing uses. Continue Reading

By

• Jack Vaughan

Kubernetes container orchestration gets big data star turn

The new thing in big data is Kubernetes container orchestration. While it's still early, there are signs of activity, which are cited in this edition of the Talking Data podcast. Continue Reading

By

• Jack Vaughan

Humans and AI tools go hand in hand in analytics applications

Companies are keeping data analysts and other workers in the loop with AI applications to check the results generated by automated algorithms for accuracy, relevance and missing info. Continue Reading

By

• Craig Stedman, Industry Editor

£5m Ada Lovelace Institute launched to look into data ethics

Nuffield Foundation announces institute to examine ethical and social issues arising from the use of data, algorithms and artificial intelligence Continue Reading

By

• Brian McKenna, Senior Analyst, Business Applications

The Cambridge Analytica case: What’s a data scientist to do?

The Cambridge Analytica controversy has highlighted data ethics issues especially dear to early career stage data scientists Continue Reading

By

• Brian McKenna, Senior Analyst, Business Applications

Cambridge Analytica-Facebook case shows need for ethical data mining

Ethical data collection practices are becoming even more important, as cases like Cambridge Analytica's misuse of Facebook data challenge consumer trust in enterprise analytics. Continue Reading

By

• Ed Burns

Kubernetes gains momentum in big data implementation process

Big data vendors and users are looking to Kubernetes-managed containers to help accelerate deployments and enable more flexible use of computing resources. Continue Reading

By

• Craig Stedman, Industry Editor

IT, others focus efforts as GDPR implementation date looms

The EU's General Data Protection Regulation is meant to bring better data privacy to bear in the age of big data. How GDPR implementation will proceed remains to be seen. Continue Reading

By

• Jack Vaughan

Silicon Valley startups home in on pressing value from data

A slew of California-based startup and early-stage data analytics companies are promising CIOs better ways to get value from their investments in databases and analytics Continue Reading

By

• Brian McKenna, Senior Analyst, Business Applications

Streaming tool from StreamSets eyes data in motion for GDPR

StreamSets software for inspecting big data brings governance to data in motion. Such capabilities may find more use as the European Union's GDPR deadline looms. Continue Reading

By

• Jack Vaughan

Rethinking analytics processes spurs enterprise innovation

By taking a fresh look at the makeup of their analytics organizations, enterprises can innovate their business models and take advantage of digital disruption. Continue Reading

By

• Ed Burns

Diversified data sets for analytics deliver top results

Analytics teams should focus on data diversity to ensure that their projects deliver the most meaningful insights -- but they must be wary of some stumbling blocks. Continue Reading

By

• Ed Burns

U-SQL

U-SQL is a Microsoft query language that combines a declarative SQL-like syntax with C# programming, enabling it to be used to process both structured and unstructured data in big data environments. Continue Reading

By

• Lindsay Moore

Newest AI technology set to disrupt tech sector, job markets

Emerging AI technology will lead to the creation of influential new software companies and disrupt job markets, says a Goldman Sachs researcher -- predictions that are discussed here. Continue Reading

By

• Ed Burns

Google TPUs open up on cloud; LinkedIn intros Hadoop Dynamometer

In big data news, we find Google TPUs, or Tensor Processing Units, offered as a cloud service, while LinkedIn is open sourcing a Hadoop test simulator called Dynamometer. Continue Reading

By

• Jack Vaughan

TensorFlow

TensorFlow is an open source framework developed by Google researchers to run machine learning, deep learning and other statistical and predictive analytics workloads. Continue Reading

By

• Jack Vaughan

MongoDB 4.0 takes ACID transactions to multi-document level

MongoDB is taking a deeper step into SQL-style processing waters with a 4.0 update that brings increased support for ACID-compliant transactions to its NoSQL database. Continue Reading

By

• Jack Vaughan

Data Lab CEO Gillian Docherty on how data can boost Scottish economy

Gillian Docherty, CEO of The Data Lab, outlines the organisation’s mission to turn analytics and artificial intelligence to benefit Scotland’s economy Continue Reading

By

• Brian McKenna, Senior Analyst, Business Applications

Slow to gain traction, AI apps on the verge of explosion

From chatbots ("Can I help you?") to killer bots ("I'll be back."), artificial intelligence runs the gamut of applications and emotions like no other technology. It's been nearly 70 years since AI first came into consciousness with humankind, yet only recently has it made serious inroads into business and consumer markets, mainly riding the coattails of its offshoot, machine learning.

AI's other offspring -- deep learning, cognitive computing, image recognition and natural language processing -- show plenty of promise but have barely left the womb. Yet market research and industry foot soldiers tell us AI is on the cusp of explosive growth in practically every major industry.

Along those lines, the February issue of Business Information opens with our editor's note, which provides a good dose of common sense and advice for businesses not to throw caution to the whirlwind of hyperbole surrounding AI. Companies anxious to apply AI, particularly machine learning, to their operations for fear they'll fall behind the competition, must first do their homework and separate fact from fiction, all the while keeping in mind that implementing AI apps is difficult and requires time, expense and the right kind of data.

Our cover story takes the issue of good data one step further and examines the very real dangers that bias in machine learning data sets can create. Data scientists have their work cut out for them identifying and pinpointing bias, especially since nearly all data inherently contains bias. In another feature, we look at the usefulness of today's AI tools, which, except for some niche AI apps, are not very useful, but they come with a whole lot of promise as a soon-to-be transformative force in business operations.

Also in this issue, a major components manufacturer uses AI apps to transform its assembly line process, cognitive computing can help improve a doctor's bedside manner, the benefits of AI and machine learning entice healthcare organizations to move to the cloud and humankind's range of reactions to AI's unlimited potential parallel those of the atomic age.

 Continue Reading

Guide to using advanced analytics and AI in business applications

AI hype is becoming reality as the technology gains ground in businesses. Get tips on machine learning, cognitive computing and other advanced analytics initiatives in this guide. Continue Reading

Big data throws bias in machine learning data sets

AI holds massive potential for good, but it also amplifies negative outcomes if data scientists don't recognize data biases and correct them in machine learning data sets. Continue Reading

By

• Bridget Botelho, Editorial Director, News

Cognitive computing in healthcare mends doctor-patient gaps

Cognitive computing and healthcare data aggregation may prove to be important bedside companions to doctors treating their patients in the era of medical specialization. Continue Reading

By

• Scott Robinson, New Era Technology

IT analysts look for clarity on IBM analytics tools

IBM's analytics software is strong technically but disjointed and hard to understand, say analysts who think simplifying things should be a priority at the company's Think 2018 conference. Continue Reading

By

• Ed Burns

Insurer's machine learning use case: Changing driver behavior

Machine learning tools can be put to use for more than targeted marketing and product recommendations. Auto insurer HiRoad is using them to help create safer drivers. Continue Reading

By

• Ed Burns

Data catalog software takes open data initiative to the streets

Data catalog software from Alation helped San Diego's chief data officer open up the city's data to residents, as well as city planners and other government workers. Continue Reading

By

• Jack Vaughan

Data managers should study up on GPU deep learning

As GPU deep learning becomes more common, data managers will have to navigate several new layers of complexity in their quest to build or buy suitable data infrastructure. Continue Reading

By

• Jack Vaughan

Three ways to turn old files into Hadoop data sets in a data lake

Hadoop data lakes offer a new home for legacy data that still has analytical value. But there are different ways to convert the data for use in Hadoop depending on your analytics needs. Continue Reading

By

• David Loshin, Knowledge Integrity Inc.