Friday, October 28, 2022

The problem with using data for comparison: Free Press Journal 29th October 2022

 

Global comparisons even for basic economic indicators are always hard to make. The dominance of an unorganised sector, and the tracking system used, makes it hard to get the right numbers

The Global Hunger Report evoked mixed emotions in India. Those who believe India is the best performing economy in the world have taken the view that the report is disparaging and needs to be junked. Those on the other side feel that this is a wake-up call for the government to do something as growth which does not address the needs of the poorest sections is not one to take pride in. How are we to read these global reports in general?

The interesting thing is that the Global Hunger Report has been coming out every year for the last two decades and our rank has always been very low in the region of 100. Therefore, those who find this objectionable should remember that the methodology followed has been something we have not had a problem with since 2000. In fact in the past this data was used to push the case for more grass root reforms by various governments.

As to the concept of hunger, the critics have said that using 4 parameters such as malnutrition and children being under weight, short and dying early (infant mortality) is not really capturing hunger. This can be debated as even within the country we find it hard to measure poverty numbers based on calories and have reverted to an income based approach.

But this is a challenge for any ranking system, which can be of countries or management institutes or companies where the critics can always argue that the concept is not fair. It is hence necessary to understand that when any such cross-country comparison is made, the organisation upfront gives the approach which is standardised across all nations. This is the only way to make the concept homogenous for comparability. Therefore, theoretically this is the right way of going about the task.

The second issue is that when a certain standard is laid down for say ranking countries on competitiveness, or doing business or hunger or governance, it has to be based on a sample which is relevant. Choosing the right sample is critical; and generally the organiser looks at the peer level persons/institutes for responses. Hence when ‘doing business’ was assessed by the World Bank, it is but natural that a set of companies or associations are contacted. There will be a bias for sure as it may not cover the entire cross section of say the SMEs and hence the results can be tilted. Besides this could not be done across the country and hence the parameters were assessed for two regions. For the Global Hunger Report, the sample size was 3000 while for the Doing Business Report (which showed India in very good light), the sample was less than 100. Hence, one should read such reports as being at best indicative as it is possible that if the scope or concept was changed or the sample enlarged, the results could be different.

Global comparisons even for basic economic indicators are always hard to make. For the most basic concept used like GDP, every country has its approach which is not comparable. The dominance of an unorganised sector, and the tracking system used, makes it hard to get the right numbers. There is use of proxies when calculating these numbers. In our case we use corporate results to calculate value added which then is used for various sector growth rates. But such data is not available for the smaller companies and comes with lags and is a limitation. This holds even in other countries too depending on the availability of data.

Therefore, when the domestic authority calculates the numbers, it is hoped that they are honest in their effort and do not overstate numbers. This has been a constant complaint with economic numbers of China. One may recollect that even in our case when the new set of GDP numbers were calculated, the direction and quantum of change varied from the previous series causing considerable controversy. But for sure, international agencies like IMF and World Bank have to necessarily rely on domestic agencies for most of the data. Hence their final growth numbers tend to converge with the official numbers and it is only their forecasts that would be different.

The same problem surfaces when it comes to forecasts which vary across agencies. For example in India’s case for FY23 there are forecasts from 6.5% to 7.5% by multilateral agencies and credit rating firms. Clearly their assumptions are different and can at best be taken to be indicative. The final numbers will be very different and in general 80% of the forecasters get the number incorrect (if a margin of 0.2% on either side is permitted). In fact, even the NSO which collects and computes official data has significant variations in the advance estimate to final estimate. For a normal year like FY19, the advance estimate of 7.2% came in at 6.5% ultimately while for FY20 5% came down to 3.7%.

Therefore, data is as a rule is tricky and can at best be taken to be indicative. The reason is the data flow system has limitations given the size of the country. And when comparing the same across a set of 150 nations it becomes even more challenging. This is why all global ranking systems keep the concept simple with few parameters so that then probability of error comes down.

No comments: