One morning, sales analyst is presenting their querterly sales performance dashboard that is indicating the company’s revenue has increased this querter by 35%. The management team has expressed its satisfaction, charts appear to be convincing and the results are outstanding from their business perspective. But There's one thing, though. Something is off. What is it ? Can you guess? Well, it is that no one has determined if there are any anomalies or incorrectness in the data.

The time it takes to find the error could be when the purchasing department has raised its inventory budget; when marketing has revised its objectives; and when management has made decisions based on a growth trend that was nonexistent. It is not a problem of the dashboard design. It's an issue in the data that feeds it.

Companies spend lots of time and resources on analytics, reporting and forecasting processes, with the expectation of making better decisions. But the quality of those decisions rests upon something far less tangible:

the source of the data and how it gets from the source to the analyst.

The Problem Often Begins Before Analysis

Data analysis is usually associated with SQL queries, Excel formulas, Python scripts, and interactive dashboards. These tools are important, but they only work with the information they receive.

Consider a retail business that aggregates data from Shopify, its accounting system, and a warehouse management system. Every platform logs different aspects of the business. One keeps track of orders, one records payments and the third keeps track of stock movements. These usually are working well on their own. The challenge that arises is when their records need to be merged.

An order may show up in the Shopify before the payment is confirmed. A refund could be recorded in the accounting system a day later. In the meantime, the warehouse might log an inventory adjustment that doesn't have a direct link with the original order ID.

When these records are merged together without knowing the differences, then the resulting data may have inaccurate totals, incorrect dates or missing associations.

A 2024 article [1] by McKinsey on how data drives business performance found that around 60% of the technology executives surveyed pointed to the lack of quality data as one of the biggest challenges preventing them from scaling their data solutions. The same article noted that roughly 45% of data scientists' time is spent preparing the data, instead of creating and enhancing analytical models.

This is a fair amount of expert effort in preparing information to be used.

It is this simple lesson that the students do not seem to have been taught: analytical work does not start when a person opens a spreadsheet. It starts with the understanding of the information that will go into it.

Your Data Source Determines What You Can Trust

Every analytical result has an origin. It might be a sales platform, a database, a third-party API, a CSV export, or a collection of operational spreadsheets.

Knowing more than where the data came from: understanding the origin. It involves knowing what each field is, what records are in it, how often information is updated, and where the source has limitations.

For instance, let's say you have an online shop and you want to work out the monthly income based on the total values of orders. What about cancelled orders? What about refunds that are made after the initial sale? Are shipping charges included? When is revenue recognized: at the order or at the receipt of payment?

There are no right or wrong answers to all the questions. The right definition will depend on the business purpose and the accounting rules in question. If the definitions are ambiguous, however, two analysts can analyze the same source and get different revenue numbers. Either of the calculations may be mathematically correct based on the formulas used, but only one calculation may be appropriate for the business question posed. Before being able to analyze the data reliably, one must understand the meaning of the data and not just the names of the columns.

In addition, if information is provided through multiple systems, source selection is important. Pre-combination analysis requires the analysts to determine the authoritative source on each business concept. Order information might be more suitable on the sales platform, and settled payments might be better off in the accounting system.

These functions limit the number of conflicting numbers and streamline reporting.

A Good Data Pipeline Does More Than Move Information

Once the source is understood, the next challenge is delivering its information to the analytical environment accurately and consistently.

This is where the data pipeline becomes essential.

A typical pipeline extracts information from source systems, validates it, transforms it into a usable structure, and loads it into a database, warehouse, or reporting tool. Depending on the project, it may also handle incremental updates, error recovery, and automated monitoring.

The important point is that successful data transfer does not automatically mean successful data preparation.

Imagine a business that imports its daily sales transactions into a central database. The scheduled job runs every morning, and the dashboard refreshes without any visible errors. However, the extraction process retrieves the previous day's transactions again because it does not track which records have already been processed.

After several weeks, some orders have been counted multiple times. The pipeline is running, but the results are wrong. A dependable system should therefore address several practical concerns:

• Duplicate prevention: Ensure repeated imports do not inflate totals.

• Data validation: Check required fields, expected formats, and valid values.

• Consistent transformations: Apply the same business rules every time.

• Incremental loading: Retrieve new or changed records without unnecessarily reprocessing everything.

• Failure handling: Record errors and allow incomplete jobs to be recovered safely.

• Monitoring: Detect unexpected changes in record counts, missing data, and stale reports.

These safeguards do not need to be complicated. Even a small Python script or scheduled SQL process can benefit from basic checks and clear error reporting.

The objective is not to build the most sophisticated pipeline possible. It is to build one that can deliver dependable results and make problems visible when something goes wrong.

The Business Cost of Poor Data Quality

The impact of bad data goes beyond the analytics department.

According to the data quality research by Gartner, as per 2020 research, on average, organizations lose at least $12.9 million due to poor data quality each year. This is an estimate of the cost of those organizations studied and is not a blanket cost for all businesses but is indicative of the size of the problem. [2]

Recently, in 2025 IBM's Institute for Business Value found that 43% of CIOs said that improving data quality was their top data priority. IBM also found that over a quarter of the organizations surveyed said that poor data quality costs them over $5 million a year. [3]

These results suggest a crucial difference: not all data is bad data. It can generate extra labor, consume more resources, postpone decision making and negatively affect the confidence in reporting.

Consider inventory planning. Over estimating the number of products sold due to duplicate transactions can lead a business to purchasing more inventory than is needed. This would incur extra inventory carrying costs and less space for stocking other items in the warehouse, and possibly an increase in storage expenses.

However, underestimating consumption will leave the company with too little of popular products, unable to capitalise on sales and customers let down. Both times, the analytical calculation may be totally accurate, depending on the data. Information going into the calculation did not reflect business activity and the failure occurred as a result.

Data quality should be treated as an operational concern rather than a final cleaning task.

Why More Sophisticated Analysis Cannot Fix Everything

The impulse to fix the problem of unreliable reporting is to simply add more complicated formulas, machine learning models or advanced visualizations. At times, it is important to have those tools. However, complexity is not a good substitute for missing or incorrect input.

Assume a retailer is interested in predicting the demand for the next three months. Two years of past sales are used to create the forecasting model by the analyst. The model shows the seasonal variation and generates a reasonable prediction.

There have been, however, times in history where a product was not available. In such times recorded sales were naturally not high because customer demand for products was not as strong, but because the business did not have products to sell.

If the model assumes that a decline in sales equals a decline in demand, it could underestimate future demands. Without the relevant stock availability data, and its proper application within the algorithm, a sophisticated algorithm will not be able to reliably differentiate between these situations.

Customer segmentation, marketing attribution, financial reporting, and operational forecasting are just some of the areas in which the same issue arises.

Data Value, according to McKinley's research, is all about the integrity of the entire process of identifying data that matters, gathering it, strengthening it, processing it and using it to make decisions: [4]

Better models matter. Better dashboards matter. However, their worth is determined by the quality of the information, along with the business environment that backs them.

A Trustworthy Data Pipeline

A practical improvement does not require rebuilding an entire data infrastructure. Most businesses can start by strengthening a few essential practices.

1. Understand the source before designing the workflow.

Record the sources of information, field definitions, what the information represents, and which systems are reliable for each measurement.

2. Validate data at the point of entry.

Look for missing identifiers, invalid dates, negative data, and duplicated data. Avoid entering error into the system where ever possible, rather correct it later.

3. Separate raw data from transformed data.

Have an easy to access copy of the original data and use transformations with a separate layer. This simplifies investigation of discrepancies and reproducing results in the event that the business rules change.

4. Reconcile important metrics.

Verify processed totals with trusted totals. So, if there are 12,450 valid orders in the source and 12,980 in the analytical table, the difference should be investigated prior to the dashboard being published.

5. Monitor the pipeline after deployment.

A workflow that worked a month ago just might not work today due to a change in API, a new product type that comes into existence, or a change in the data structure of a source system. Observe technical problems and any changes in the data set that is produced.

6. Make data ownership clear.

Someone must be tasked with deciding what metrics are important, monitoring the changes, and making decisions regarding what to do when validation fails. When there is no ownership, errors may not be resolved since it is assumed that another team is responsible.

These practices enable the development of a pipeline, which moves beyond a mere transfer mechanism to one that can be controlled and is conducive to reliable reporting.

Bring Confidence in Every Number

A useful analytical system should not only generate results, it should generate results that are useful. It ought to provide people with the reasonable confidence in those results.

If a manager queries on why revenue has gone up, the analyst should be able to go back to the transactions and be able to comment on the calculation. If inventory forecasts are not met when they are unexpected, the team should be able to figure out if this is because of real demand, source system problem or transformation error.

It is based on clear definition, reliable data flows, appropriate validation and understanding of business operation, which are the reasons for such confidence.

It also changes the analyst's role. The team will be able to focus more on recognizing opportunities, communicating performance, and assisting the business in making informed decisions rather than spending each reporting cycle on probing discrepancies and fixing common mistakes.

Conclusion

A dashboard can be attractive, a model can be sophisticated, and an automated report can arrive on time every morning. All of these accomplishments do not ensure the reliability of the information. Important aspects of data analysis are linking reliable sources, sound analytical methods, reliable pipelines, and reality based decisions.

For businesses, this is not about the reports, but the processes leading to these reports. For analysts, it is about learning how to look into what information is coming from, how it's changing, and whether what they are getting is true or not. It is the one a business can trust enough to act on.