Impact of Project Management Investments in Project Success 1 Project Management and Project Performance: a Longitudinal Study
Vol.4, No.2, pp.23-94,
IMPACT OF PROJECT MANAGEMENT INVESTMENTS IN PROJECT SUCCESS PROJECT MANAGEMENT AND PROJECT PERFORMANCE: A LONGITUDINAL STUDY :
LEANDRO ALVES PATAH, PHD UNIVERSIDADE NOVE DE JULHO – SÃO PAULO, BRASIL
Abstract: The aim of this study is to investigate the relationship between the adoption of project management methodologies and project success. The methodological approach involved a longitudinal field survey conducted in three countries, Argentina, Brazil and Chile. Data were obtained for 1387 projects from different sectors. The results provide evidence that the degree of use of project management methods have statistically positive impact on operating results of the projects. The number of project managers certified as PMP in the division that conducts the project, as well as investments in training in project management has a statistically positive impact on operating results of the projects. It was also possible to observe that the complexity of the project affects the operating results of the projects. Other conclusions are that it is easier to explain project success than failure in projects, complex projects have less chance of success in relation to lower complex projects, investments in project management training increase the chances of operational project success and it is possible to measure success in project management. Keywords Performance, project management models, complexity.
1. Introduction Various methods and techniques have been developed and encapsulated in bodies of knowledge institutes and professional associations of project management (PMI, 2013, IPMA, 2006). Nevertheless, the project management remains a challenge highly problematic, since a lot of projects exceed their budgets, delay or fail to meet its objectives, as evidenced by several studies (DAI; WELLS, 2004, THE STANDISH GROUP, 2009, WHITE, FORTUNE, 2002). In recent years many companies have spent significant amounts of resources in project management. The PMI (2009) presents a number of $ 12 trillion, one-fifth of world GDP, as the amount to be invested in projects in each of the years of this decade. Despite this, there is a lot of research about the return on investment in project management methods. Thus, despite the high degree of investment in projects and increasing numbers of skilled project managers, some questions about the results and benefits come with these investments. In general, work on the revision of the theory of present projects, among others, the following questions (IKA, 2009; SODERLUND, 2004): • How to prove that spending money on training in methods and management projects has value and what the function or value added by the project management? • How to make senior executives committed to the methods of project management without showing the financial results generated from the investments? • What determines the success or failure of a project? The few existing quantitative evidence presents a picture less than rosy. The The Standish Group (2009), based on a survey of 280,000 information technology projects conducted in 2008 shows that only 32% of the projects may be considered a success. Close to half of the projects studied, ie 44%, presents problems of timing or costs and 24% can no longer be recovered and projects were considered failures. In the same survey, projects that spend more than the predicted value, burst its budget on average by 45%. How to complete the project on time, the survey data are also not far from encouraging, on average, the schedule is increased by 63% of its original, and only 67% of the required characteristics and features are typically delivered to customers designs. Projects fail and this will not change unless companies start to measure where the projects fail and why (BUCHANAN, 2008)
hodologies (BESNER; HOBBS, 2013; HONG et al., 2011; CHOU; YANG, 2012; ALA-RISKU; KARKKAINEN, 2006). Although the volume of literature in the area, there is scant empirical evidence that relates implementation of project management to the results obtained. On the other hand, executives are also seeking evidence that their investments are working effectively and producing the expected value of profits at the end of projects. In short the question of quantifying the value of Project Management has not yet been satisfactorily answered (IKA, 2009, THOMAS, MULLALY, 2008). Aiming to contribute to the evaluation of the benefits of project management in organizations, this paper seeks to assess the relationship between investments and results of projects. Investments in the implementation of project management involve the development and use of tools and methods, training of project managers, and administrative and organizational support. As a proxy to the pay off it will be used the operating performance achieved by the projects. The methodological approach involves a longitudinal study with a hybrid approach, qualitative and quantitative. This paper is divided into five sections. Section 2 presents a summary of the theoretical framework, followed by Section 3 which presents the methodological approach. The results and discussions are presented in section 4. Finally, section 5 presents the conclusions and recommendations for future work.
2. Review of Literature
2.1. Methods of Project Management Projects can be defined as organizations “forgettable”, which arise through the routine, being composed of individuals who are unlikely to work together again. A strategy to encode the organization’s capabilities in project management should be defined at the enterprise level to repeat successful approaches in future projects. Using a well-structured and implemented, these capabilities can be stored and transferred over time, space and context. Additionally, through the creation of an external memory to individuals, this type of coding knowledge can make organizations less vulnerable to loss the tacit knowledge stored in people (IBERT, 2004). A systematic project can consist of methods, toolkits and design models. Thus, project management can be viewed as the sequential application of structured processes, continuous and repeated that, when used by an organization in a gradual and safe mode for your business, lets take steps toward the institutionalization of standardized practices. In addition, systematic team needs help in the planning and delivery of projects, considering the whole cycle of life, consistent and efficient, always business-oriented and customer satisfaction (SILVEIRA, 2008). And for that, there are some characteristics of systematic project management organizations highlighted by the author as being in line with ISO 9000 standards or other official institutes of project management. It should also have a flexible and comprehensive set of processes, tools and techniques that support the activities, audited periodically by a PMO (Project Management Office). It is worth noting the need for documentation, measuring instruments and control of projects during the life cycle and communication of results to stakeholders. Some studies have been tried to understand the impact of these PM standards implementation (MCHUGH; HOGAN, 2011; BESNER; HOBBS, 2013; CHOU; YANG, 2012) and suggest relation between PM maturity and success. There are currently several sets of models of project management methods available for use by professionals and organizations to better manage their projects. The most widespread methods currently are available by institutes and associations dedicated to the study as presented in Table 1.
2.2. Success in Projects Several studies have been conducted over the last decade, seeking to analyze how project success can be measured (BELOUT; GAUVREAU, 2004; BESNER; HOBBS, 2006; BIZAN, 2003; DVIR; RAZ; SHENHAR, 2003; GRAY, 2001; KENDRA; TAPLIN, 2004; LIPOVETSKY et al., 2005; RAZ; SHENHAR; DVIR, 2002; REPISO; SETCHI; SALMERON, 2007). The project success is usually defined as meeting the objectives of time, cost and quality in which meets the project’s stakeholders. Nevertheless, research conducted over the past year, and since the 1980s, which have been investigating the dimensions of project success, led to re-write the above formula with the general agreement that project success is multi-dimensional and that different people measure the success of projects in different ways at different times (BARBER, 2004; BRYDE, 2003; IKA, 2009; JUGDEV; MULLER, 2005). In order to measure the success of projects you can create criteria and metrics as proposed by Ling (2004), which is a division refining the project success in achieving success in the product of the project by meeting quality standards and achieve success in the process by meeting the goals of time and budget. For this, the author uses four metrics to evaluate the performance of projects: cost, time, quality and service customer satisfaction. Since Larson and Gobeli (1989) present some factors that may affect the success of projects such as project structure, project manager’s competence and size of the project, using the same indicators proposed by Ling (2004). Another way to measure success is folding it in two different criteria. One, the very success, which according to Cooke-Davies (2002), cannot be measured until the project is finished, and another, the performance of projects, which can be measured at runtime. According to the author, no system of metrics in projects can be considered complete without a package of measures (performance and success) and should seek a method of connecting them, as a means of assessing the accuracy with which the performance of projects predict the success of the organization. Thamhain (2004) in a study conducted between 2000 and 2003, with 76 project teams of 27 companies, seeking, in turn, associate with the environment of project teams with the same performance. According to the author, the main variables related to the project teams that influence success, are the environment of the team and team performance. The benefits to the client (DVIR et al., 1998), adaptability and ability to cooperate with the project in other areas of the organization (KATZ; ALLEN, 1985) and service quality standards and safety (WHITE; FORTUNE, 2002) should also be included in systems of performance measurement projects systems. Financial criteria are also being used to measure performance on projects for some time. The criteria may include economic return, cost / benefit analysis (ARCHER; GHASEMZADEH, 1999), contribution of improved financial measures such as profits, market share and value of new projects obtained (THOMAS, DELISLE; JUGDEV, 2002). Another direct way of assessing the benefits of project management is to analyze the margins of a firm’s current projects. You can compare the scope of a project when the company sells, with the value obtained when it is completed. The difference can be partly explained by the methodology of project management. It may be noted that the metrics for performance measurement used in most projects are those related to obtaining, at the end of the project, initially planned values of time and cost (GRAY, 2001; KATZ; ALLEN, 1985; LARSON; GOBELI, 1989; LING, 2004; WHITE, FORTUNE, 2002), and consensus is the financial issue involved (ARCHER; GHASEMZADEH, 1999; THOMAS, DELISLE; JUGDEV, 2002), which was expected. Some of these authors, however, include other concepts such as risk (ARCHER; GHASEMZADEH, 1999), satisfaction of stakeholders, obtained new projects and team performance (THOMAS; DELISLE; JUGDEV, 2002). And others, provide a slightly different by including the value as a function of the processes that organizations generate (IBBS; REGINATO, 2002).
3. Methodology and Hypotheses :
Our methodological approach involved a longitudinal study with a multi-methods research approach, merging qualitative and quantitative approach. It is an increasing interesting in applying multimethodological research (SINGHAL; SINGHAL, 2012a,b), by using different sources of data, or by using different subsets of the same data. In this study, several data collection methods were combined in order to achieve triangulation, such as structured and unstructured interviews of key participants (PMO coordinators and project managers) and PMO projects’ archival data (see Appendix). Discrepancies among these sources of evidence were noted and discussed. To test research hypothesis and develop the research model, the logistic regression method was selected. Several factors can lead to high-performance designs, as seen previously. This study emphasizes the use of project management methods (KESSLER; WINKELLHOFER, 2002; THE STANDISH GROUP, 2009; WHITE; FORTUNE, 2002) and training of project teams (COOKE-DAVIES, 2002; DAI; WELLS, 2004). Several authors outline the benefits of using project management methods (IBERT, 2004; KERZNER, 2001; KESSLER; WINKELLHOFER, 2002; THE STANDISH GROUP, 2009; WHITE; FORTUNE, 2002). KERZNER (2001) presents a list of benefits, including: improving the performance of activities in relation to care plans and objectives of the project. Thus emerge the following research hypotheses: • H01: The level of utilization methods of project management does not affect the operational results of the projects. Archibald (2003) presents the project manager himself as one of the critical factors of project success. Additionally, Cooke-Davies (2002) includes in its list of critical success factors for a project, adequate training at all levels of the organization in project management concepts. Other authors also include issues of training and education as key success factors in project management (DAI; WELLS, 2004). To assess whether the investment in training in project management improves their performance, were established hypotheses H02a and H03b. • H02a: The investment in training in project management does not influence the projects operational results • H02b: The number of project managers certified as PMP in the division that conducted the project does not influence the projects operational results. Finally, The Standish Group (2009) presents the size of a project, its duration and size of the team as metrics that can influence the success of a project. According to the institute, about the size of the project, the smaller the project, the higher the probability of project success. Regarding the duration of the project, the institute argues that smaller projects with durations are more likely to succeed. The same happens with the team size. Cooke-Davies (2002) confirms these arguments when presented as a critical success factor for the suggestion to keep the durations of the projects in three years. Crawford, Hobbs and Turner (2004); Larson and Gobeli (1989) and White and Fortune (2002) are other authors who include criteria of complexity as factors leading to an outstanding performance on projects. In turn, Shenhar, Dvir (1996) and Raz, Shenhar, Dvir (2002) classified into four levels of complexity ranging from low to super high-tech. In order to determine whether and how the complexity of the project influences the performance of it, it was proposed to the third hypothesis of this thesis: • H03: The complexity of the project does not affect the operational results of projects.
3.2. Data Collection This paper describes independent and dependent variables according to the concept of Marconi and Lakatos (2003). As the independent variable (X) is the one that influences, determines or affects another variable, according to the hypotheses, the independent variables chosen were use of project management methods and effort in training in project management. Since the dependent variable (Y) consists of those values (phenomena or factors) to be explained or discovered, by virtue of being influenced or affected by certain independent variable, the choice of compliance costs, deadlines and financial performance of the project, were based on key indicators for measuring performance of selected theory, as discussed above. Moderating variable (M) is one factor, phenomenon or property that also impacts the dependent variable, but to a lesser extent, influencing the relationship between the independent and dependent variables (MARCONI; LAKATIS, 2003). Thus, from the critical success factors presented above and the criteria used for selection of control, the complexity of the project was selected as the only moderating variable to assess its influence on the creation of better cost, schedule and financial performance of the project according to the use of methods and effort in training the project team.
4. Discussion and Analysis of Results
4.1. Data Obtained:
The organization studied is a multinational company with several divisions operating in different markets; it was possible to obtain data from a large number of projects over a long period of analysis. This company has 60% of their gross sales from projects. The company produces and installs a wide variety of equipment, mostly delivered to customers through specific projects tailored to the needs of each. The rates of product innovation are extremely high, and the products currently sold are developed in no more than three years ago. Basically, the company sells customized solutions for its customers with low volume of units produced and a great variety. The company has several business units, and for sizing the sample of projects examined, it was adopted the rule of thumb proposed by Hair et al. (1998), that when it comes to the analysis of dependence or interdependence among variables, one should obtain at least 20 samples for each variable, for a total of 120 samples of projects to be obtained during the stage of data collection. 2851 data were obtained from projects conducted from January 2005 to June 2008. However; several of these projects did not have complete information as the final costs of implementation, the categorization assigned and others. After a detailed analysis, it was selected 1387 projects with complete data, or 48.65% of the total, able to be analyzed to try to prove the assumptions set out in section 3. These projects were conducted between July 2006 and June 2008. For all projects were obtained from the information contained in appendix. In relation to the performance segment of the project, the area with more projects obtained is the area of energy, with 713 projects (51.41% of total), followed by the area of medicine with 218 projects. Regarding the country of realization, most of the projects was conducted in Brazil with 823 projects (59.34% of total), followed by Argentina with 294 projects (21.20%) and Chile, with 270 projects (19, 47%). The independent variable Use of Project Management Methods (X1) is concentrated between 70.83% and 83.33% and has a low relative dispersion (coefficient of variation = 4%) and standard deviation of 3.29%. The projects have an average of 77.18% of degree of implementation of project management methodologies, with a median of 75.98% and 75.00% fashion. The independent variable Effort in Training in Project Management (Financial Investment) (X2) has values between R$ 0.00 and R$ 615,000.00. In the business units that were developed the projects studied, the average investment is R$ 238,609.00, with a median of R$ 85,000.00, and fashion of R$ 615,000.00. The relative dispersion is 113% (coefficient of variation), with a standard deviation of R$ 270,344.00. For the independent variable X2 – Effort in Training in Project Management (number of PMPs), the projects have, on average, seven certified employees and a median of three employees certified. The relative dispersion is 116% (CV) and standard deviation of 8. The dependent variable Y1 – Budget Meeting distributes between a minimum of -100% (final value greater than the estimated value) and 716.6% (final value less than the estimated value). Additionally, almost half of the projects tie the final cost to plan and few projects (12.98%) have their final cost above planned levels. The projects have an annual average cost of 9.51% below the estimate, with median and mode equal to zero. The relative dispersion is high, with a coefficient of variation of 495% and standard deviation of 47.07%. The dependent variable Y2 – Compliance Deadline distribute a range between -10% (late project) and +10% (early design). Most projects (70.23%) are completed within the planned date. The projects deadline have, on average 4.66% lower than expected, median and mode were 9.8% lower than initially estimated. The relative dispersion is high with a coefficient of variation of 159% and standard deviation of 7.42%. The dependent variable Y3 – Financial Performance is distributed in a range between -664% (projects indicating loss) and 502% (indicating projects gain). The average is negative (-2.68%), with median and mode equal to zero. The projects have a high variability, with standard deviation of 84.40%. 4.2. Analysis of Results As the non-parametric test of Kolmogorov-Smirnov test rejected the hypothesis of normal distribution for the three dependent variables (p on the bank of projects with external clients is to obtain contractual amendments, discussed in this perspective does not work once it comes to a point external to the organization, more related to market and selling price of the project. So let it be a single variable related to the financial aspect to the project, along with the variable of time, to compose the new dependent variable of success. The independent variables considered in logistic regression analysis were as follows: Use of Project Management Methods (X1); Effort in Training of Project Management (Financial Investment) (X2); Effort in Training of Project Management (Number of PMPs) (X2). The moderating variable considered in the logistic regression analysis is the complexity of the project (M1). The new dependent variable, called result, considered in the logistic regression analysis results from the definition of the operating results of the project and was established by two response categories: Success (1) and Failure (0). It was found that the variable X2 (Financial Investment) and X2 (Number of PMPs) are highly correlated, therefore, unnecessary and unwise, the entry of these two variables in the estimation of the model. Thus, it was kept only the variable X2 (Financial Investment) in the sequence of the analysis.
All the hypotheses established in this work were denied. The proposed objectives, the first was to measure the influence of investment in project management, through the development, implementation and use of methods of project management and training of project managers on the operational performance of the same. The goal regarding the use of project management methods and their relation to the operational performance of the projects was achieved using the technique of multivariate data analysis type logistic regression to reject the hypothesis H01, demonstrated that the use of methods have statistically positive impact on operating results of the projects, measured through the achievement of project budget and schedule. Another objective of this work addresses the analysis of the relationship between the effort in training of project managers and the results achieved with the project. This goal was also achieved for rejecting hypotheses H02a and H02b, respectively established to evaluate the investment in project management training by the units of analysis and the amount of project managers certified as PMPs on them. For both metrics, the statistical analysis showed that the effort in training in project management has a statistically positive impact on projects operational results. The ultimate goal proposed for this work addresses the analysis of the influences of the complexity of projects, where lower complex projects were expected to deliver better operational results, while more complex projects should be more difficult to get your results. This was achieved with the rejection of the H03 and the results presented. Through statistical analysis conducted was found that the complexity of the project has positive statistically impact on project operational results. Thus, as seen in the theory of critical success factors on which this discussion is more to success than the failure of projects for the consolidated model presented in this paper is easier to explain project success (effectiveness of 91.40%) than failure in projects (20.11% efficiency). It was observed that all variables positively influence the likelihood of success of the operational results of the project. Since complexity is the variable that most influenced the overall success of a project, with a weight greater than the use of methods and training. It was found that complex projects have less chance of success, around 55% in relation to projects of lower complexity that have a chance of success of 70%. The results also showed that when the project is less complex, the chance of operational success of it nearly doubles, increasing 80%. Given the two models presented (for the most complex projects and for projects less complex), the model with the highest percentage of hits refers to less complex projects, but can only correctly predict every project a success. It is like noted: projects less complex signal operational success. When the analysis is make using specific models for less and more complex projects, the chance of success of the model is larger than when using a single consolidated model for the two cases. This demonstrates the increasing importance of addressing the application of management concepts in organizations in a manner customized to the needs of the company and especially the needs of projects. The consolidated analysis developed in this work showed that for highly complex projects, success is not influenced by training in project management and, less complex projects are not influenced by use of management methods. However, it is worth investing in training for less complex projects. As shown, each R$ 100,000.00 increase in investment in training project management, the chances of operational success increase at 28%. It was also found that for projects with failure, the mean level of implementation, investment in training and number of PMPs is very close to the most complex projects and less complex. As for successful projects, investment in training and the number of PMPs are much higher than for less complex projects. That is, less complex projects require a higher operational effort to succeed. It is possible to say that for simple projects, structural aspects of project management still have their influence felt, but for more complex projects, they no longer influence both the practice and experience applied. Nevertheless, the results obtained together with the overall effectiveness of the model in predicting operating results of the projects, the order of 64%, suggest that there may be additional variables to those studied in this paper to explain the success or failure of a project. Finally, as observed in this work, it is possible to measure success in project management and the values obtained may be very close to reality, as was the case of the contingency model developed here, which showed an efficiency of 91% to predict the success of a project.
(Please contact editor chief of Apa for references)