How scale and ownership are related to financial performance? A productivity analysis of the Chinese banking sector

Abstract

This paper analyzes productivity with a directional distance function model by investigating convexity and non-convexity in the Chinese banking sector. Two primary findings are summarized. First, there is a distinct change in productivity between large versus small banks and public versus nationwide joint-stock banks; the economies of scale and public orientation of the Chinese economy affect the performance of nationwide joint-stock banks. Second, the number of banks that experience local versus global technological change is small, indicating that productivity changes are driven by national rather than regional dynamics, although minor regional dynamics exist in the data.

1 Background

The People’s Bank of China (PBC), now the Central Bank of China, was established in 1948 and functioned as a central bank as well as a commercial bank; in other words, it can be characterized as a monobank (García-Herrero et al., 2006). During the period from 1979 to 1984, central authorities decided to reestablish (or establish) four specialized banks (SBs), i.e., the Agricultural Bank of China (ABC), the Bank of China (BOC), the China Construction Bank (CCB), and the Industrial and Commercial Bank of China (ICBC). All of the aforementioned banks were separated from the PBC (except for the CCB from the Ministry of Finance) and operated in the agricultural area, foreign exchange area, fixed-asset investment area, and industrial and commercial areas, respectively. However, they were not entirely profit oriented and had to conduct policy-related business while being hampered by government intervention. As a result, nonperforming loans arose and accumulated in the following years.

When favorable market economy reforms became more widespread, the government began to take comprehensive measures to improve China’s banking system. In 1994, the government founded three policy banks to take over the policy-related business from the four SBs to transform the latter into commercial banks, which are characterized by market-functioning and profitable institutions (García-Herrero et al., 2009). Additionally, beginning in 1986, authorities approved the opening of a number of joint-equity commercial banks, including national joint-equity commercial banks and regional commercial banks, to decrease the monopoly power of the four wholly state-owned banks (WSOBs) in the financial market. The four WSOBs were castigated by the outside world for their high nonperforming loan ratios and their technical bankruptcies after the Asian financial crisis (Whalley, 2003). Therefore, the Chinese government instituted a package of measures to address the threatening situation, including issuing 270 billion Yuan, drastically reducing top-heavy employment and closing branches, reducing the tax burden, and establishing four asset management companies (AMCs) to remove a large sum of bad loans from the four WSOBs (Bonin and Huang, 2001).

When China joined the World Trade Organization (WTO) in 2001, there were new challenges brought to the financial sector before they would fully open up to the world 5 years later. Considering the loss of market shares and even bankruptcy, the government favored enhancing the efficiency and competitiveness of domestic commercial banks, particularly the four state-owned banks (SOBs). It launched a series of reforms to improve the banking industry so that it could compete with foreign counterparts. First, a special banking supervisory organization—the Chinese Banking Regulatory Commission (CBRC)—was established in 2003. Second, the government utilized the exchange rate reserve to replenish the capital funds of Chinese commercial banks through a sovereign wealth fund company, i.e., the Central Huijin Company. Third, the government transformed the four SOBs into joint-stock commercial banks and listed them on the stock exchange so that they would be supervised by market forces. Table 1 shows the 14 key commercial banks in China.

Table 1 The 14 key commercial banks in China

2 Literature review and objective

Most previous research on Chinese banks has focused on the efficiency differences between the four large banks and other banks, comparing the effects of bank scale and ownership. Chen et al. (2005) compared the cost efficiencies of the four largest banks and two smaller size classes for the 1993–2000 period, concluding that the big four and smaller joint-equity banks are all cost efficient relative to medium-sized joint-equity banks. Furthermore, financial deregulation in the mid-1990s had a significant effect on efficiency.

Kumbhakar and Wang (2005) utilized an input distance function and concluded that the four large banks were less efficient than the joint-equity banks for the 1993–2002 period. Moreover, deregulation had no significant effect on efficiency improvement. Fu and Heffernan (2007, 2009) validated the results of Kumbhakar and Wang (2005) with a cost frontier model for the 1985–2002 period, finding that the X-efficiency of Chinese banks significantly declined; they concluded that the reforms had little impact on the structure of China’s banking sector. Lin and Zhang (2009) used regression models to analyze the performance of Chinese banks from 1997 to 2004, concluding that the four large state banks were less profitable, less efficient, and displayed worse asset quality than other types of banks. Berger et al. (2009) analyzed the profit efficiency of Chinese banks from 1994 to 2003 and concluded that the four large state banks were the least efficient and that foreign banks were the most efficient; they also discovered that minority foreign ownership was associated with significantly improved efficiency.

Several studies have utilized parametric estimations to clarify the factors affecting efficiency. Ariff and Can (2008) analyzed the cost and profit efficiencies of 29 Chinese commercial banks from 1995 to 2004 with a two-stage data envelopment analysis (DEA) model, using a Tobit model in the second stage, and concluded that profit efficiency levels were well below cost efficiency and that the inefficiencies were related to revenue. Furthermore, state-owned banks were less efficient, as joint-stock banks and medium-sized banks were significantly more efficient than small and large banks. Jia (2009) analyzed the relationship between bank ownership and performance with regression models, concluding that lending by state-owned banks has been less prudent than lending by joint-equity banks. Jiang et al. (2009) analyzed the technical efficiency of Chinese banks from 1995 to 2005 with a stochastic distance function and concluded that bank efficiency and profitability improved with joint-stock ownership versus state ownership. These results demonstrate that state-owned banks are less efficient than joint-stock or foreign banks, though more recent results do show improvement in the former.

There are many existing studies that apply DEA. Laurenceson and Yong (2008) analyzed the efficiency of the 11 most prominent banks 5 years after China’s entry into the WTO with a DEA for 2005, 2006, and 2007, concluding that the differences between levels were insignificant. Barros et al. (2011) analyzed Chinese bank efficiency with a B-convex DEA model and concluded that economies of scale were not translating into efficiency, though efficiency did increase during the study period. Avkiran and Morita (2010) analyzed Chinese banks with a nonradial DEA model that demonstrated that different inputs and outputs could capture five key stakeholder perspectives. They concluded that profits relative to total assets and return on equity presented a high correlation with efficiency.

However, there is no prior study focusing on the structure of productive technology change using both convexity and nonconvexity production frontier line of the Chinese banking sector. Our objective is to clarify the differences of the results between convex and nonconvex assumptions in the Chinese banking sector. This study is the first to focus on determining the existence of (non)convexity in banking sector production.

This paper analyzes the efficiency in Chinese banks from 1993 to 2010, focusing on both convex production technology (CPT) and nonconvex production technology (NCPT) in the Chinese banking sector. Although research on banking efficiency is a well-established field as we explain above, research on Chinese banks focusing on the NCPT is more limited. In this context, the relationship between production technological change and firm characteristics (e.g., firm scale and ownership) is questionable.

This research is motivated by the following criteria. First, there is a systematic conclusion that Chinese banks are inefficient based on various alternative methodologies of efficiency research (Berger et al., 2009). Reasons for this systematic result vary, but they are typically attributed to bank management procedures in which the government and Communist Party are closely entwined with the bank managers. While this situation may appeal to state planners, it is inefficient for individual banks (Economist, 2010) and justifies further analyses of Chinese bank productivity to discern undetected aspects of Chinese banking activity. Second, China has three of the four largest banks in the world today based on market capitalization (Economist, 2010), signifying the global importance of Chinese banks and justifying the present research. Third, efficiency and productivity are two separate aspects of overall performance, and to date, the analysis of Chinese banks has focused on efficiency. Therefore, this paper proposes an innovative approach in this context and presents an updated and clearer view of the extent of Chinese bank productivity, complementing research in this field and identifying local and global technological changes.

The remainder of this paper is organized as follows. The “Methods” section details the methodology. Then, the “Data and results” section presents the data and discusses the results, and the “Conclusions” offers conclusions.

3 Methods

3.1 Directional distance function

We apply a directional distance function (DDF) approach to evaluate the productive inefficiency of the Chinese banking sector. One advantage point of DDF is that productive inefficiency can be evaluated using multiple input and output variables. There is a growing literature of productivity analysis studies using DDF (Fujii et al., 2010; Tanaka and Managi, 2013; Fujii et al., 2014; Kumar et al., 2015).

Let $x\in {\mathrm{\Re}}_{+}^L,\ y\in {\mathrm{\Re}}_{+}^M$ be vectors of inputs and outputs, respectively, and then define the production technology as follows:

$$ P(x)=\left\{\left(x,y\right):x\ \mathrm{can}\ \mathrm{produce}\ (y)\right\} $$

(1)

Productive inefficiency score D(x, y| g _x , g _y ) of production units in P(x) for each firm in this study is defined with the distance β from the production frontier consisting of efficient production units as follows:

$$ \overrightarrow{D}\left(x,y\Big|\ {g}_x, {g}_y\right)=\mathrm{S}\mathrm{u}\mathrm{p}\left\{\upbeta :\left(y+\beta {g}_y\right)\right\}\in P\left(x-\beta {g}_x\right) $$

(2)

where g _x and g _y denote nonnegative directional vectors of the input and outputs, respectively. From the above definition, Eq. (2) can be computed for province k by solving the following optimization problem under the assumption of CPT:

$$ {\overrightarrow{D}}_{\mathrm{CPT}}\left({x}_k^l,{y}_k^m\Big|{g}_{x^l},{g}_{y^m}\right)=\mathrm{Maximize}\ {\beta}_k^{\mathrm{CPT}} $$

(3)

$$ \mathrm{s}.\mathrm{t}.\kern1.5em {\displaystyle \sum_{i=1}^N}{\lambda}_i{x}_i^l\le {x}_k^l-{\beta}_k^{\mathrm{CPT}}{g}_{x^l} $$

(4)

$$ {\displaystyle \sum_{i=1}^N}{\lambda}_i{y}_i^m\ge {y}_k^m+{\beta}_k^{\mathrm{CPT}}{g}_{y^m} $$

(5)

$$ {\displaystyle \sum_{i=1}^N}{\lambda}_i=1 $$

(6)

$$ {\lambda}_i\ge 0\kern7.5em \left(i=1,\cdots, N\right) $$

(7)

where l and m represent the name of the input and desirable output, respectively. x is an input matrix with dimensions L × N and y is a desirable-output matrix with dimensions M × N. Furthermore, g _x is the directional vector of the input matrix, g _y is the directional vector of the desirable-output matrix, β _k is the inefficiency score of the firm k, and λ _i is the weight variable. To estimate the inefficiency score of all firms, the model needs to be applied independently to each of the N firms. The right-hand side of Eqs. (4) and (5) represents the frontier line, and λ _i is the parameter to decide the reference point. The directional vector specifies for inefficient firms the way to improve productivity towards the frontier production line. Equation (6) sets that production function is under the various return to scale assumptions.¹

In a similar manner, our NCPT formulation calculates the distance function by solving the following optimization problem:

$$ {\overrightarrow{D}}_{\mathrm{NCPT}}\left({x}_k^l,{y}_k^m\Big|{g}_{x^l},{g}_{y^m}\right)=\mathrm{Maximize}\ {\beta}_k^{\mathrm{NCPT}} $$

(8)

$$ \mathrm{s}.\mathrm{t}.\ {\displaystyle \sum_{i=1}^N}{\lambda}_i{x}_i^l\le {x}_k^l-{\beta}_k^{\mathrm{NCPT}}{g}_{x^l} $$

(9)

$$ {\displaystyle \sum_{i=1}^N}{\lambda}_i{y}_i^m\ge {y}_k^m+{\beta}_k^{\mathrm{NCPT}}{g}_{y^m} $$

(10)

$$ {\displaystyle \sum_{i=1}^N}{\lambda}_i=1 $$

(11)

$$ {\lambda}_i\in \left\{0,1\right\}\ \forall i $$

(12)

$$ {\lambda}_i\ge 0\kern7.9em \left(i=1,\cdots, N\right) $$

(13)

In the DDF model under the NCPT assumption is added Eq. (12) in the DDF model under the CPT assumption. Equation (12) has the role of the integrality constraint. By setting Eq. (12), we can assume a nonconvexity production frontier line (Kerstens and Managi (2012)). Figure 1 represents the production frontier line shape under two assumptions which are a nonconvex and convex technology.

Fig. 1

Production frontier line under nonconvex and convex technologies. Source: Kerstens and Managi (2012)

3.2 Luenberger productivity indicator and structure of technical change

The total factor productivity (TFP) was computed with the results of the distance function model and is derived as follows (Chambers et al., 1998):

$$ {\mathrm{TFP}}_t^{t+1}={\mathrm{TECHCH}}_t^{t+1}+{\mathrm{EFFCH}}_t^{t+1} $$

(14)

$$ {\mathrm{TECHCH}}_t^{t+1}=\frac{1}{2}\left\{{\overrightarrow{D}}^{t+1}\left({x}_t,{y}_t\right)+{\overrightarrow{D}}^{t+1}\left({x}_{t+1},{y}_{t+1}\right)-{\overrightarrow{D}}^t\left({x}_t,{y}_t\right) - {\overrightarrow{D}}^t\left({x}_{t+1},{y}_{t+1}\right)\right\} $$

(15)

$$ {\mathrm{EFFCH}}_t^{t+1}={\overrightarrow{D}}^t\left({x}_t,{y}_t\right)-{\overrightarrow{D}}^{t+1}\left({x}_{t+1},{y}_{t+1}\right) $$

(16)

where x _t represents the input for year t; x _t + 1 is the input for year t + 1; y _t is the desirable output for year t; and y _t + 1 is the desirable output for year t + 1. ${\overrightarrow{D}}^t\left({x}_t,{y}_t\right)$ is the inefficiency score of year t based on the frontier curve in year t. Similarly, ${\overrightarrow{D}}^{t+1}\left({x}_t,{y}_t\right)$ is the inefficiency of year t + 1 based on the frontier curve in year t + 1.

The TFP score indicates the productivity change compared to the benchmark year. We estimate two types of TFP indicator. One is TFP_CPT which is estimated under CPT assumption and the other one is TFP_NCPT which is estimated under NCPT assumption. The TFP includes all categories of productivity change, delineated as technical change (TECHCH) and efficiency change (EFFCH). TECHCH shows the shift in the production frontier, and EFFCH measures the change in a production unit’s position relative to the frontier (i.e., catching up).

Kerstens and Managi (2012) developed an identification methodology for global technological change (GTC) and local technological change (LTC) through productivity indicators using both TFP results estimated under CPT and NCPT assumptions. Figure 2 shows the graphical image of GTC and LTC. The performances of six firms in the t year and t + 1 year are found in Fig. 2. Here, we consider two cases. The first case is that all six firms shift to the upper left direction which shows a more efficient production (less input, more output). This frontier line shift represents the global technical change. The second case is that only firm D shifts to the upper left direction in the t + 1 year, and the other firms stay at the same point in the t year. In this case, the frontier line shape is changed partially from the t year to t + 1 year. This partial frontier line shift is called the local technical change (Fujii et al., 2015).

Fig. 2

Global technical change (GTC) and local technical change (LTC). Source: Kerstens and Managi (2012)

If we define global technical progress as resulting from efficient observations at two time periods that experience positive TECHCH under CPT assumption (TECHCH_CPT) between the periods t and t + 1 relative to CPT, then:

$$ \mathrm{G}\mathrm{T}\mathrm{C}\left(\left({x}_t,{y}_t\right)\left({x}_{t+1},{y}_{t+1}\right)\right)=\left\{{D}_{\mathrm{CPT}}^t\left({x}_t,{y}_t\right)=0{\displaystyle \cap }{D}_{\mathrm{CPT}}^{t+1}\left({x}_{t+1},{y}_{t+1}\right)=0{\displaystyle \cap }{\mathrm{TECHCH}}_{\mathrm{CPT}}^{t,t+1}>0\right\} $$

(17)

whereby the arguments of the proportional distance function are suppressed to condense notation. First condition shows bank is evaluated as efficient under CPT assumption in t year. Second condition represents bank is evaluated as efficient under CPT assumption in t+1 year. Third condition means TECHCH indicator estimated under CPT assumption is positive, which shows production frontier line is shifted to more efficient direction from t year to t+1 year. GTC is achieved if above three conditions are satisfied simultaneously. In contrast, if we define the local technical progress as resulting from efficient observations at two time periods in terms of NCPT while being inefficient with respect to CPT that experiences positive TECHCH under NCPT assumption (TECHCH_NCPT) in terms of NCPT between the two time periods, then:

$$ \mathrm{L}\mathrm{T}\mathrm{C}\left(\left({x}_t,{y}_t\right)\left({x}_{t+1},{y}_{t+1}\right)\right)=\left\{\begin{array}{c}\hfill \left[{D}_{\mathrm{NCPT}}^t\left({x}_t,{y}_t\right)=0{\displaystyle \cap }{D}_{\mathrm{CPT}}^t\left({x}_t,{y}_t\right)>0\right]\kern1.75em \hfill \\ {}\hfill {\displaystyle \cap}\left[{D}_{\mathrm{NCPT}}^{t+1}\left({x}_{t+1},{y}_{t+1}\right)=0{\displaystyle \cap }{D}_{\mathrm{NCPT}}^{t+1}\left({x}_{t+1},{y}_{t+1}\right)>0\right]\hfill \\ {}\hfill {\displaystyle \cap }{\mathrm{TECHCH}}_{\mathrm{NCPT}}^{t,t+1}>0\kern12em \hfill \end{array}\right\} $$

(18)

The condition of LTC in Eq. (18) is more relax than condition of GTC in Eq. (17) (Fujii et al., 2015). It should be noted that there is no case to be identified as both GTC and LTC simultaneously. In Fig. 1, firms B, D, and F satisfy Eq. (17) and firms C and E satisfy Eq. (18). Thus, firms B, D, and F are observed as GTC, and firms C and E are observed as LTC from t year to t + 1 year. Firm A does not achieve both GTC and LTC because TECHCH = 0 under the CP and NCP model (see Fig. 1).

4 Data and results

4.1 Data

We utilize data from 14 Chinese banking firms from 1993 to 2010. The financial statements of the Chinese banks were sourced from their websites (1998–2008) and the Almanac of Chinese Finance and Banking (2000–2011). Table 2 provides a description of the data samples. All financial data were deflated to 2005 prices. Banks are assumed to provide two outputs, loans (100 million Yuan) and securities (100 million Yuan), while utilizing two inputs, number of employees (persons) and deposits (100 million Yuan). The Humphrey (1985) intermediation approach was adopted, which views banks as collecting deposits that are then “intermediated” into loans and securities. Berger and Humphrey (1997) suggested that the intermediation approach is best suited for analyzing bank-level efficiency.

Table 2 Data description from 1993 to 2010 (average score per bank)

To further analyze the results, we separated the firms by company scale and ownership. First, we divided the sample into two groups based on the number of employees: large and small. Companies with more than 10,000 employees were categorized as large, while firms with less than 10,000 employees were considered small. Based on this criterion, there were eight large firms (ABC, BC, BOC, CCB, CITIC, CMB, GDD, and ICBC) and six small firms (CEB, HX, IB, MSB, SPD, and SZD). Second, we divided the sample into state-owned commercial banks and nationwide joint-stock commercial banks to delineate ownership characteristics. Using the banks listed in Table 1, there were 4 state-owned bank firms and 10 nationwide banks.

4.2 Results and discussion

Figures 3 and 4 display the TFP change of each scale and ownership groups from 1993 to 2010 by graphically comparing the nonconvex and convex results. In calculation of the productive inefficiency score using DDF, we set the directional vector as (g _y , g _x ) = (y, −x). This type of directional vector assumes that an inefficient firm can improve productivity while increasing desirable outputs and decreasing inputs in proportion to the initial combination of actual inputs and outputs.

Fig. 3

Productivity change by ownership and scale by the CPT model

Fig. 4

Productivity change by ownership and scale by the NCPT model

Figure 3 verifies that the TFP of both state-owned banks and large banks increased during the period, while it decreased for nationwide and small banks. This observation is a previously recognized result in the literature, where scale is of paramount importance (Berger and Mester, 1997). In fact, small banks do not typically have national coverage and therefore are not used by the central bank to distribute the monetary base. Though this activity is traditionally a starting point in commercial banking, the small banks are effectively banned from it. The negative evolution of nationwide joint-stock banks is smaller and may have been due to the public orientation of the Chinese economy during this period that favored national public and large banks over regional banks.

Figure 4 shows that the TFP evolves as in Fig. 3 but with minor changes. Based on this result, it is verified that the CPT and NCPT models display a similar visual evolution. However, the CPT model highlights the changes in the series, presenting more decreases or increases than the nonconvex model.

From Figs. 3 and 4, we guess there is significantly different TFP change by scale and ownership. To confirm the differences of the TFP change by scale and ownership, this study used nonparametric methods to obtain the TFP indices. Thus, we were not able to determine the distribution of scores in advance; therefore, we needed to apply the nonparametric test for equality densities introduced by Li et al. (2009). This nonparametric equality test is well suited to analyze the difference of distribution when analyzing TFP scores. This equality testing employs bootstrap resampling to obtain the finite-sample distribution of the statistics. In this paper, we set the bootstrap algorithm to be repeated 1000 times.

Table 3 presents the results of the nonparametric test of small versus large banks and state-owned versus nationwide banks. Based on Table 3, the TFP_CPT and TECHCH_CPT are statistically significant at a 5 % level in separating small from large banks, but the inefficiency scores and EFFCH_CPT are not. Thus, we can say the trends of TFP_CPT and TECHCH_CPT are significantly different between large-scale firms and small-scale firms. These results imply that characteristics of firm scale affect financial performance of Chinese banking firms through production frontier shift effect. In other words, large-scale banking firms tended to achieve financial performance improvement with innovative activities from 1993 to 2010.

Table 3 Mean value and nonparametric test of productivity indicator by scale and owner

Meanwhile, inefficiency score and EFFCH_CPT are statistically significant between a state-owned firm and nationwide firm. The EFFCH indicator represents how much an inefficient firm catches up with efficient firms. Thus, we consider that a nationwide firm tended to achieve increased financial performance by steady efforts.

Next, we discuss the results under NCPT in Table 3. The difference of TFP_NCPT and TECHCH_NCPT between small versus big banks is statistically significant at a 10 % level, which is a similar trend with the results under the CPT assumption. Meanwhile, the difference between state-owned versus nationwide joint-stock banks is not significant in all indicators. These results imply that the differences of ownership do not have a strong effect into the financial performance indicator if we assume nonconvexity.

Finally, Table 4 presents the number of firms identified as GTC and LTC from 1993 to 2010 (17 periods). From Table 4, the total number of GTC is 72 which is higher than the total number of LTC. This observation signifies that GTC is more common in China, implying that because Chinese banks are centrally regulated, they behave globally. Next, we discuss the differences of results focusing on each scale and ownership group. Before we compare the results of each group, we need to adjust the effect of sample size differences among groups. Because the GTC and LTC indicator is a count data, a large sample group tends to observe the many times of GTC and LTC. To adjust the number of firm sample differences among groups, we use the share of GTC and LTC identification.

Table 4 Number of observed GTC and LTC

From Table 4, the share of GTC is more than double in large-scale firms and state-owned firms than small and nationwide firms. This result shows that innovative financial performance improvement in the Chinese banking sector is mainly by large or state-owned bank from 1993 to 2010. Meanwhile, the share of LTC is almost the same between a state-owned firm and nationwide firm, even though the large-scale-firm group has a double score of share comparing with the small-scale-firm group. This result implies that LTC is the more popular financial performance improvement in nationwide banking firms in China. This is another evidence to consider that nationwide banks improve their financial performance due to steady effort.

5 Conclusions

This paper analyzes the productive inefficiency and total factor productivity in Chinese banks from 1993 to 2010, focusing on both convex production technology (CPT) and nonconvex production technology (NCPT) in the Chinese banking sector. By applying both CPT and NCPT assumption, we identified the global technical change (GTC) and local technical change (LTC).

The conclusions of the present research are as follows. First, productivity changes are distinct between large versus small banks and public versus nationwide joint-stock banks, meaning that economies of scale and the public orientation of the Chinese economy affect the performance of the nationwide joint-stock banks. Second, the number of banks that experience LTC versus GTC is small, signifying that productivity changes are driven by national and not regional dynamics, although there are some minor regional dynamics in the data.

What are the policy implications of this paper? Chinese bank performance should be continually monitored using alternative methods that can shed light on unknown research aspects of banking activity. Benchmarking enables the banks to follow the performing leaders and therefore upgrade their activity, becoming more competitive each year. Recognizing the both convexity and nonconvexity of the data is of paramount importance in reaching meaningful conclusions.

How does this paper compare with previously published works on Chinese banks? By utilizing an alternative DEA model, this paper distinguishes between GTC and LTC, introducing a new view of Chinese bank efficiency. Avkiran (2011), who used a slack-based DEA, and Matthews (2013), who presents a Chinese bank risk analysis with a network DEA, are both convex models, so no clear comparison can be made with the present research. More research is required to confirm the presented results.