Precautions When Using Secondary Data

In the previous chapter, "Sources of Secondary Data in India", we explored the massive goldmines of information published by the government, international bodies, and private institutions. It is tempting to look at a beautifully formatted government report or a thick corporate dossier, grab the numbers you need, and plug them straight into your own analysis. It saves time, it saves money, and the data is already sitting right there.

But doing this blindly is one of the most dangerous mistakes a statistician can make.

Secondary data is, by definition, data that was collected by someone else, for their own purpose, using their own methods, at a specific point in time. Using secondary data is like wearing someone else's prescription glasses. They might help you see a little better than having nothing at all, but if the prescription doesn't match your exact needs, they will eventually give you a massive headache and cause you to stumble.

Before you use any secondary data, you must put it through a rigorous vetting process. Famous statistician Arthur Bowley once warned that it is never safe to take published statistics at their face value without knowing their meaning and limitations. In this chapter, we will learn exactly how to interrogate existing data to ensure it will not ruin your study.

Why Secondary Data Cannot Be Used Blindly

When you collect primary data yourself, you are in total control. You know exactly how the questions were phrased, you know if the respondents were confused, and you know the exact boundaries of your study.

With secondary data, you lose all that context. You are only looking at the final numbers, not the messy reality of how those numbers were gathered. If you use secondary data without precautions, you expose your study to several fatal risks:

1. 1The "Different Purpose" Trap: The original researcher might have been trying to prove a completely different point. Their definition of terms might not match yours.
2. 2The "Hidden Bias" Trap: The agency that collected the data might have had a vested interest in making the numbers look a certain way (for example, a corporation exaggerating its market share).
3. 3The "Outdated Reality" Trap: A dataset from 2019 might look perfectly structured, but if you are studying consumer behavior, the world changed drastically in 2020. Pre-pandemic data might be entirely useless for a post-pandemic study.

To protect your research from these traps, statisticians rely on a strict three-pillar framework. Before accepting any secondary data, it must pass the tests of Reliability, Suitability, and Adequacy.

Pillar 1: Reliability of the Data

Reliability asks a simple question: Can I trust these numbers, and can I trust the people who gathered them?

If the data is not reliable, nothing else matters. It doesn't matter if it perfectly matches your topic or if there is a massive amount of it; if it is built on lies, errors, or sloppy methodology, your final conclusions will be wrong.

To check reliability, you must act like a detective investigating the original study. You need to ask the following questions:

Who collected the data? (Credentials of the Agency)

Not all data sources are created equal. Data published by the Reserve Bank of India (RBI) or the National Statistical Office (NSO) carries a high degree of reliability because these are apex institutions with massive resources, strict legal mandates, and decades of expertise. On the other hand, data published by a small, unknown private advocacy group on their blog should be treated with extreme suspicion. You must ask: Does the collecting agency have the experience, the funding, and the trained staff to pull off a study of this size?

How did they collect it? (Methodology)

You need to find the "methodology" section of the original report. Did they use a proper random sampling method, or did they just ask their friends? Did they use trained enumerators, or did they just mail out a survey and hope for the best? If the original method was flawed, the resulting data is unreliable.

Did they have a bias?

Consider the source's motivation. If a tobacco company publishes a statistical report on the health effects of smoking, you must be highly skeptical of the reliability of that data. Always look for independent, neutral sources that have no financial or political stake in the outcome of the numbers.

What was the level of accuracy desired and achieved?

In statistics, 100% accuracy is rarely possible. But you need to know what margin of error the original researchers accepted. If they accepted a 10% margin of error, and your study requires pinpoint precision (like a 1% margin of error), their data is not reliable enough for your specific needs.

Pillar 2: Suitability of the Data

Suitability asks: Even if this data is perfectly accurate and trustworthy, does it actually fit the specific puzzle I am trying to solve?

Data can be 100% reliable but completely unsuitable for your study. This is where many researchers make mistakes. They find a highly credible government report, but they fail to realize that the government's definitions do not match their own.

To determine suitability, you must carefully compare the original study's framework with your own.

Checking the Definition of Units

This is the most common trap. Suppose you are conducting a study on "Agricultural Laborers" in Punjab. You find a highly reliable dataset from ten years ago about agricultural laborers. However, upon reading the fine print, you discover that the original study defined an "agricultural laborer" as someone who works on a farm for at least 180 days a year. Your study, however, defines it as someone who works on a farm for at least 100 days a year.

Because the definitions of the basic unit do not match, the data is unsuitable. You cannot compare their numbers with yours, because you are counting two different groups of people.

Checking the Scope and Object

If you want to study the cost of living for middle-class families in urban areas, but the secondary data you found was collected to measure the cost of living for industrial workers in slum areas, the data is unsuitable. The object of the original inquiry was different, meaning the basket of goods they measured (like basic rations vs. electronics and fuel) will be completely different from what you need.

Checking the Time Period and Conditions

Context is everything in statistics. Data is a snapshot of a specific moment in time. You must ask: Were the conditions during the original data collection similar to the conditions I am studying now?

For example, imagine you are studying normal household savings rates. You find a brilliant, reliable dataset from the year 2020. Is it suitable? No. The year 2020 was marked by a global pandemic, severe lockdowns, and massive economic disruption. People's saving and spending habits during that year were highly abnormal. Using 2020 data to project normal future savings trends would be disastrous. Data collected during times of war, famine, economic depression, or pandemics is usually unsuitable for studying normal periods.

Pillar 3: Adequacy of the Data

Adequacy asks: Is there enough of this data to give me a complete picture?

You might find data that is highly reliable (collected by a great agency) and perfectly suitable (the definitions match yours exactly). But if it doesn't cover enough ground, it is inadequate.

Geographical Coverage

Suppose you are tasked with estimating the average income of apple farmers in the entire Union Territory of Jammu & Kashmir. You find a recent, reliable, and suitable study on apple farmers' incomes. However, you notice that the original researchers only collected data from the Shopian district, completely ignoring Baramulla, Kupwara, and Anantnag.

While the data is great for Shopian, it is inadequate for a study claiming to represent all of J&K. The geographical scope is too narrow.

Sample Size

If the original study only interviewed 50 people to represent a population of 5 million, the sample size is too small. Even if those 50 people were interviewed perfectly, the data is inadequate for making broad statistical claims.

The Vetting Process: How to Actually Do It

When you are sitting at your desk with a secondary dataset in front of you, how do you actually apply these precautions? You don't just guess; you follow a systematic flowchart.

Worked Example: Evaluating a Dataset

Let’s put this into practice with a concrete scenario.

Your Goal: You are an economist working for the government. You need to write a report on the "Current average daily wage of skilled construction workers in urban India for the year 2025."

The Data You Found: A 2022 report titled "Wages in the Building Sector," published by the "National Association of Real Estate Builders" (a private lobbying group for construction companies). The report surveyed 500 workers across three major cities (Mumbai, Delhi, and Bengaluru). It defines a "skilled worker" as anyone who has worked on a site for more than 30 days.

Let's evaluate this data using our three precautions:

1. 1Reliability Check:

• Agency: It is published by a private lobbying group representing builders. Builders have a vested interest in showing that they pay high wages (to look good) or low wages (to argue for government subsidies). There is a high risk of bias.
• Verdict: Fails reliability.

1. 1Suitability Check:

• Definitions: The report defines a "skilled worker" as anyone with 30 days of experience. The official Indian government definition of a skilled worker requires formal vocational training (like an ITI certificate) or years of specific apprenticeship.
• Time Period: The data is from 2022. You need data for 2025. Inflation and economic shifts over three years make the old wage numbers unsuitable for current estimates.
• Verdict: Fails suitability.

1. 1Adequacy Check:

• Coverage: The report only covers three mega-cities. Urban India includes thousands of smaller cities and towns where wages are vastly different.
• Sample Size: 500 workers is a microscopic sample size for a country as large as India.
• Verdict: Fails adequacy.

Conclusion: As an economist, you must completely reject this secondary data. Using it would result in a deeply flawed report. You must keep searching for a source that passes the RSA checks (such as the Periodic Labour Force Survey published by the NSO).

A Note on "Adjusting" Secondary Data

Sometimes, you will find data that is reliable and adequate, but slightly unsuitable because of a minor difference in definitions or base years. In advanced statistics, you don't always have to throw the data away.

If you know exactly how the original data was calculated, you can sometimes apply mathematical adjustments to make it suitable. For example, if you have reliable income data from 2015, and you want to use it for a 2020 study, you might adjust the 2015 figures using the official Consumer Price Index (CPI) to account for inflation.

However, this requires a high level of statistical skill. The golden rule remains: If you do not fully understand how the secondary data was collected and what it represents, do not use it.

Quick Revision

• Secondary data is risky: It was collected by someone else for their own reasons. Using it blindly can ruin your study.
• The Three Pillars (RSA): Always check Reliability, Suitability, and Adequacy before using existing data.
• Reliability (Trust): Depends on the credentials of the collecting agency, the scientific nature of their methods, and the absence of bias. Government apex bodies are generally highly reliable.
• Suitability (Fit): The original study's definitions, objectives, and context must match your current study. Data collected during abnormal times (wars, pandemics) is usually unsuitable for normal studies.
• Adequacy (Volume/Coverage): The data must cover a large enough geographical area and have a sufficient sample size to represent the population you are studying.
• Methodology is key: Always read the methodology section of a secondary report to understand the definitions and margins of error used by the original investigators.