data splitting

What is data splitting?

Data splitting is when data is divided into two or more subsets. Typically, with a two-part split, one part is used to evaluate or test the data and the other to train the model.

Data splitting is an important aspect of data science, particularly for creating models based on data. This technique helps ensure the creation of data models and processes that use data models -- such as machine learning -- are accurate.

How data splitting works

In a basic two-part data split, the training data set is used to train and develop models. Training sets are commonly used to estimate different parameters or to compare different model performance.

The testing data set is used after the training is done. The training and test data are compared to check that the final model works correctly. With machine learning, data is commonly split into three or more sets. With three sets, the additional set is the dev set, which is used to change learning process parameters.

There is no set guideline or metric for how the data should be split; it can depend on the size of the original data pool or the number of predictors in a predictive model. Organizations and data modelers might choose to separate split data based on data sampling methods, such as the following three methods:

1. Random sampling. This data sampling method protects the data modeling process from bias toward different possible data characteristics. However, random splitting can have issues regarding the uneven distribution of data.
2. Stratified random sampling. This method selects data samples at random within specific parameters. It ensures the data is correctly distributed in training and test sets.
3. Nonrandom sampling. This approach is typically used when data modelers want the most recent data as the test set.

With data splitting, organizations don't have to choose between using the data for analytics versus statistical analysis, since the same data can be used in the different processes.

Common data splitting uses

Ways that data splitting is used include the following:

• Data modeling. This uses data splitting to train models. An example of this is in regression testing modeling, where a developer uses a model to predict a system's response when operated with made-up values. Using this set of values, the developer would select a portion of that data to act as the training data. Then, they would compare those results against the test data put through the regression model. This gives the developer a sense that the model is accurate.
• Machine learning. This also uses data splitting to train models. Training data is added to the model to update its training phase parameters. After the training phase is finished, the data from the test set is measured against how the model handles new observations.
  
  

• Cryptographic splitting. This is a different process from the uses of data splitting mentioned above. It is a technique used to secure data over a computer network. Cryptographic splitting is meant to protect systems from security breaches and involves encrypting data, splitting the encrypted data into smaller pieces and storing those pieces in different storage locations. The data is further encrypted when stored in its new location.

Data splitting in machine learning

In machine learning, data splitting is typically done to avoid overfitting. That is an instance where a machine learning model fits its training data too well and fails to reliably fit additional data.

The original data in a machine learning model is typically taken and split into three or four sets. The three sets commonly used are the training set, the dev set and the testing set:

1. The training set is the portion of data used to train the model. The model should observe and learn from the training set, optimizing any of its parameters.
2. The dev set is a data set of examples used to change learning process parameters. It is also called the cross-validation or model validation set. This set of data has the goal of ranking the model's accuracy and can help with model selection.
3. The testing set is the portion of data that is tested in the final model and is compared against the previous sets of data. The testing set acts as an evaluation of the final mode and algorithm.

Data should be split so that data sets can have a high amount of training data. For example, data might be split at an 80-20 or a 70-30 ratio of training vs. testing data. The exact ratio depends on the data, but a 70-20-10 ratio for training, dev and test splits is optimal for small data sets.

Learn more about building machine learning models and the seven steps involved in the process here.

Editor's note: This article was remediated in June 2024 to improve the reader experience.