Download Classification of Problems in Requirements Elicitation for Software Development and more Lecture notes Information Technology in PDF only on Docsity!
Issues in Informing Science and Information Technology Volume 12, 2015
Cite as: Davey, B., & Parker, K. (2015). Requirements elicitation problems: A literature analysis. Issues in Informing Science and Information Technology, 12, 71-82. Retrieved from http://iisit.org/Vol12/IISITv12p071- 082Davey1929.pdf
Editor: Eli Cohen
Requirements Elicitation Problems:
A Literature Analysis
Bill Davey
RMIT University
Melbourne, Australia
Kevin R. Parker
Idaho State University
Pocatello, Idaho, USA
Bill.Davey@RMIT.edu.au parkerkr@isu.edu
Abstract
Requirements elicitation is the process through which analysts determine the software require- ments of stakeholders. Requirements elicitation is seldom well done, and an inaccurate or in- complete understanding of user requirements has led to the downfall of many software projects. This paper proposes a classification of problem types that occur in requirements elicitation. The classification has been derived from a literature analysis. Papers reporting on techniques for im- proving requirements elicitation practice were examined for the problem the technique was de- signed to address. In each classification the most recent or prominent techniques for ameliorating the problems are presented. The classification allows the requirements engineer to be sensitive to problems as they arise and the educator to structure delivery of requirements elicitation training.
Keywords : requirements elicitation, requirements gathering, software development, systems de- velopment, software failure.
Introduction
Although often overlooked or assumed to be a straightforward task (Kitapci & Boehm, 2007), requirements gathering for software development projects is the most critical phase of any soft- ware development methodology. Determining software requirements is the cornerstone to any successful project. Requirements are critical for defining, estimating, and managing any project (Raisinghani, Riggen, & Ryan, 2014). Quality requirements are essential to the success of any software development project, regardless of whether it is based on agile or traditional methodolo- gies. The requirements gathering process followed in a traditional frameworks and agile ap- proaches may differ, but the criteria for quality requirements remain constant (Nee, 2009).
An important part of requirements gathering is obtaining requirements from people: requirements elicitation (RE). Without an accurate understanding of what the stakeholders really want and need, projects cannot develop what the stakeholders desire. Thus, requirements elicitation is a crucial first step in the software development process (Kitapci & Boehm, 2007). Failure of information systems is common, and effective re- quirements elicitation is an important factor in avoiding system failure. The most efficient and well-engineered sys- tem must be useful to end users and that
Material published as part of this publication, either on-line or in print, is copyrighted by the Informing Science Institute. Permission to make digital or paper copy of part or all of these works for personal or classroom use is granted without fee provided that the copies are not made or distributed for profit or commercial advantage AND that copies 1) bear this notice in full and 2) give the full citation on the first page. It is per- missible to abstract these works so long as credit is given. To copy in all other cases or to republish or to post on a server or to redistribute to lists requires specific permission and payment of a fee. Contact Publisher@InformingScience.org to request redistribution permission.
Requirements Elicitation Problems
is contingent on the right specifications being obtained in the first instance. The purpose of this paper is to present a view of what can go wrong in requirements elicitation based on a study of published research designed to improve RE. First some justification will be made for the conten- tions that systems failure is common and that good RE is important. Then the results of the litera- ture analysis will be explained.
Failure and Abandonment Rates
The industry would like to be able to use the term ‘engineering’ when talking about professional activities. While civil engineering projects are abandoned or sometimes fail, collapsing bridges or buildings tend to make the international news. This is also the case with aeronautical engineering problems. Using these other engineering areas as a benchmark, information systems figures for failure are abysmal.
The Standish Group has conducted large-scale surveys of project managers from 1994 until 2008 and has reported the findings in the ‘Chaos reports’. These surveys classified project outcomes as ‘success’, ‘challenged’ or ‘failed’. A summary by Eveleens and Verhoef (2010) of the first five Standish surveys is shown in Table 1 and indicates that outright success of information systems projects is rare.
Table 1: Success rates of IS projects taken from Eveleens and Verhoef (2010, p. 2) YEAR SUCCESS CHALLENGED FAILED
1994^16 %^53 %^31 % 1996 27 % 33 % 40 % 1998 26 % 46 % 28 %
2000 28 % 49 % 23 % 2004 29 % 53 % 18 %
The Standish surveys have been replicated by others in the United States with similar outcomes (Emam & Koru, 2008). These empirical results are supported by authors of studies of information systems failure (Anbari, 2003; Basili & Boehm, 2001; Briggs & Gruenbacher, 2002; Dalcher & Drevin, 2003; Eberlein & Leite, 2002). Studies have found very similar problems in other coun- tries. For example, Sauer looked at the UK. In 565 projects they found 5% abandoned and 55% over budget and 20% of projects delivered with less than 80% of specifications (Sauer & Cuth- bertson, 2002). Also in the UK a study in 2000 found that “Out of 1,027 projects covered only 130, or 12.7%, were successful” (Taylor, 2000, p3). These examples are consistent with several investigations of failed projects in Australia (Brouwer, 2011; Pearson, 2012), the Netherlands (Tut, 2009) and China (Xiangnan, Hong, & Weijie, 2010).
These gloomy figures for information systems project failure, although consistent, should be read with some note of warning. The most common definition of failure in this context is when a pro- ject goes over time, over budget or does not meet its objectives in any way. There is some evi- dence (Glass, 2002, 2005; Savolainen, Ahonen, & Richardson, 2012) that ‘failed’ projects are sometimes delivered and go on to meet company needs. While this definition of project failure is seen by some authors to be inappropriate there seems to be no commonly accepted definition that distinguishes between outright failure and poor estimation of cost or time. Some authors make a distinction between the success of the project management and the success of the project, for ex- ample, Savolainen, Ahonen, and Richardson (2012). Of course this does not apply to the case of abandonment of the project where nothing is delivered.
Requirements Elicitation Problems
Even in business, health care and education, it occasionally goes wrong. Com- putable put 22 remarkable Dutch ICT failures in 2009 among them (Tut, 2009) (translated from the Dutch original).
Other studies showing the persistence of failure in the UK, Australia, China and the United States include those by Brouwer (2011), Pearson (2012), Shepherd, Patzelt, and Wolfe (2011), Taylor (2000), and Xiangnan, Hong, and Weijie (2010).
This persistence is remarkable as significant work has been done in addressing failure and it would be expected that this work would impact professional practice and hence performance. Sto- ica and Brouse capture this apparent anomaly as an issue in saying:
In the field of Information Technology (IT) there is an observable trend toward project failure. Although multiple actions have attempted to address this failure trend, they have not impacted the extent of the trend (Stoica & Brouse, 2013, p728).
It seems clear from these studies over a period of time, and including very recent work, that in- formation systems project failure continues to be a significant problem.
The Role of RE in Information Systems Project Failure
The next proposition to be examined is that RE is an important aspect of failure. There is no sug- gestion here that RE is the only or the critical factor in failure, but that, as an individual factor it is significant when compared with others.
The contention can be broken into two propositions:
- The greatest contribution to system failure comes from poor RE
- The cost of fixing RE problems is higher than other sources of error.
There is a general agreement that poor RE is an important and potentially damaging part of build- ing a system.
The Standish reports are used to justify a claim that more than half of overruns and failures occur due to poor RE (Briggs & Gruenbacher, 2002; Eberlein & Leite, 2002; Lamsweerde, 2000). This finding is replicated in other studies in Europe (Lamsweerde, 2000) and in studies by Boehm (2000) and Moløkken-Østvold and Jørgensen (2003). Over a similar period various attempts have been made to measure the effect of poor requirements elicitation. Variously these studies have resulted in figures ranging from 12 per cent to 71 per cent of failure being attributed to poor re- quirements elicitation:
…accurately capturing system requirements is the major factor in the failure of 90% of large software projects” (Davis et al., 2006), “Poor requirements man- agement can be attributed to 71 percent of software projects that fail; greater than bad technology, missed deadlines, and change management issues. (Lindquist, 2005, p. 54)
There is also a general agreement amongst commentators that fixing the results of poor RE is more expensive than for other mistakes.
Significant research into information systems project failure has been conducted using the general framework of project management. In this domain the single most common work is the project management core body of knowledge (PMBoK) (IEEE Computer Society, 2008). This document points to the importance of RE as it happens at the start of the project and the cost of rectifying changes is most affected by this stage.
Davey & Parker
It seems clear from this general agreement that RE is critical to project success. In the next sec- tion evidence is examined as to the relevance of the interview of clients to good RE.
Method
A search for literature was conducted using the databases: ACM: Association for Computing Ma- chinery Digital Library, Expanded Academic ASAP (Gale), IEEE Xplore, ProQuest, CiteSeerX, Science Citation Index Expanded, and ScienceDirect (Elsevier). The search terms [requirements, elicitation, communication, systems investigation, systems analysis, business analysis, specifica- tion, requirements definition, negotiation, system failure, and cognitive difficulty] were initially used and additional searches generated from citations within the found articles. Articles before 2000 were omitted unless cited by several more recent papers.
Papers (1680) were filtered by removing those with no mention of improvements or identification of problems in RE resulting in 420 final papers. Papers reporting empirical research and meta analyses were given more weight. Papers were then assigned a number of potential key concepts in terms of the problem solved by the treatment proposed or problems identified. These key con- cepts were then aggregated into the problem types. For each problem type a study was selected that the authors considered to be a useful approach to amelioration of the problem type.
A Classification of Problems in RE
Analysis of the literature generated a very long list of problems identified as leading to poor re- quirements elicitation. This list comes from both those who have summarised the literature of RE and those writers who use their own judgement as to some underlying causes of poor RE and pro- ceed to investigate some solution to those causes. It is clear from the wide variety of sources and range of reported problems that RE is a complex and difficult task. It is unlikely that a simple “solution” exists to such a complex problem.
Meta-analysis of the literature of RE has been conducted by many authors including Hansen, Berente, and Lyytinen (2009), Dieste and Juristo (2011), Appan and Browne (2012), Zave (1997), Harris (2006), Davis, Dieste, Hickey, Juristo, and Moreno (2006), Hickey and Davis (2004), Reichental (2006), Marakas and Elam (1998), Pan (2005), Finkelstein (1994), Davidson (1996), Majchrzak, Beath, Lim, and Chin (2005), Zowghi and Coulin (2005), Pitts and Browne (2007) and in a less formal way by Jain, Vitharana, and Zahedi (2003).
Many other authors merely state a type of problem as an assumption underlying their study of a ‘cure.’
The identified problems with RE can be summarised in nine categories:
- There are human aspects of RE that preclude simple communication between consultant and client
- The language of humans is not always suitable for technological solution
- Requirements change as the project proceeds
- Clients will sometimes ask for requirements that the organization does not need
- The client cannot say what the business needs
- Some clients do not want to help you with the project
- RE failed because it was not done properly
- Symptoms that are not problems are often reported
- RE is not deterministic
Each of these will be discussed in detail in the following sections.
Davey & Parker
- Some consultants find the process difficult and rely upon their knowledge of previous so- lutions (Zowghi & Coulin, 2005).
Requirements Change as the Project Proceeds
A distinct type of reported problem arises independent of the accuracy of an initial specification. This can be seen as one of three types of change:
- Clients learn what is possible during the project (Briggs & Gruenbacher, 2002; Jain et al., 2003; Majchrzak et al., 2005; Robertson, 2001; Saiedian & Dale, 2000; Zowghi & Coulin, 2005).
- Business is essentially dynamic and so requirements change during the lifetime of the project (Boehm & Turner, 2004; Davis, Nurmuliani, Park & Zowghi, 2008; Larman, 2004; Pitts & Browne, 2007).
- People change their mind about what they want (Tsumaki & Tamai, 2006).
Clients will Sometimes Ask for Requirements that the
Organization Does Not Need
Requirements can be specified directly from client conversations. This is not always sensible, for instance:
- The client asks for something that is not really needed (Davis et al., 2008; Tsumaki & Tamai, 2006).
- The client asks for something they are not committed to (Tsumaki & Tamai, 2006).
The Client Cannot Say What the Business Needs
Problems can arise when the assumption the clients understand their business needs turns out to be unfounded:
- Some requirements are tacit. That is, understood by the client, but not stated by them as it forms part of their tacit knowledge (Hudlicka, 1999; Robertson, 2001; Tsumaki & Tamai, 2006; Zowghi & Coulin, 2005).
- Some clients only know about a single section of the business that needs to be fixed (Arthur & Groner, 2005).
Some Clients Do Not Want To Help You with the Project
Requirements can be made difficult to elicit if the client representatives are not committed to the project. This can take several forms:
- A client representative has interests that conflict with others in the project or with the aims of the project (Alvarez, 2002; Briggs & Gruenbacher, 2002; Davidson, 1996; Gior- gini, Massacci, Mylopoulos & Zannone, 2006; Zowghi & Coulin, 2005).
- Some people will use resistance tactics to avoid a conclusion to RE (Saiedian & Dale, 2000; Tsumaki & Tamai, 2006; Zowghi & Coulin, 2005).
- Clients see the new system as a part of power struggles in the organisation (Alvarez, 2002; Davidson, 1996; Zowghi & Coulin, 2005).
Requirements Elicitation Problems
RE Failed Because It Was Not Done Properly (e.g., User Input
Not Allowed)
Another distinct type of problem is lack of professional training or behaviour in the consultant or analyst:
- No user input was obtained (Hansen et al., 2009).
- Theory of RE was not used in practice (Hansen et al., 2009; Harris, 2006).
Symptoms That Are Not Problems Are Often Reported
Statements are often made using the words “sources of problem” when the description is of a symptom, e.g., “the three leading sources of project difficulty – i.e., lack of user input, incomplete requirements, and changing specifications” (Hansen et al., 2009, p.46).
The most common of these symptoms reported is the outcome of RE being an incomplete or in- correct set of requirements. This can take a few forms:
- Incomplete requirements (Arthur & Groner, 2005; Browne & Ramesh, 2002; Browne & Rogich, 2001;Harris, 2006; Lee & Rine, 2004; Marakas & Elam, 1998; Tsumaki & Tamai, 2006; Zave, 1997; Zowghi & Coulin, 2005).
- Changing specifications (Hansen et al., 2009).
- Finished system does not include all requirements asked for (Harris, 2006; Marakas & Elam, 1998).
- Incorrect requirements (Marakas & Elam, 1998).
RE Is Not Deterministic
In his PhD thesis, Harris (2006) found an “elliptical misalignment” between the theoretical and empirical worlds of RE. He maintains that this indicates that there are no causal connections be- tween RE activities and eventual requirements. This type of thinking has been reflected in a number of post-modern approaches to studying RE. Examples include the case study of failure by Mitev (2000), the HSO process proposed by Andreou (2003), a framing model developed from social cognitive theory in a PhD by Davidson (1996) and the comparison of RE techniques by Coughlan and Macredie (2002) using socially oriented methodologies.
Conclusion
All of the papers studied that identified problems with RE were examined and nine types of prob- lem were identified that incorporate all those sources. The categorization is particularly useful as each problem category is the result of work done to find methods of addressing the problem type. A software engineer finding a problem type occurring can use the category to identify how to ap- proach the job. An educator can use the categorization to structure lessons. This is important as there is some evidence that requirements elicitation is not emphasised in CS and IS courses. In particular there is evidence that graduates are especially weak at requirements gathering.
References
Alvarez, R. (2002). Confessions of an information worker: A critical analysis of information requirements discourse. Information and Organization, 12 (2), 85-107.
Anbari, F. T. (2003). Earned value project management methods and extensions. Project Management Journal, 34 (4), 24-33.
Andreou, A. S. (2003). Promoting software quality through a human, social and organisational require- ments elicitation process. Requirements Engineering, 8, 85-101.
Requirements Elicitation Problems
Hansen, S., Berente, N., & Lyytinen, K. (2009). Requirements in the 21st Century: Current Practice and Emerging Trends. In K. Lyytinen (Ed.), Design requirements engineering: A ten-year perspective (44- 87). Berlin, Heidelberg: Springer.
Harris, H. J. (2006). Requirements dilemma. PhD, Brunel Univeristy.
Hickey, A. M., & Davis, A. M. (2004). A unified model of requirements elicitation. Journal of Manage- ment Information Systems, 20 (4), 65-84.
Hudlicka, E. (1999). Knowledge elicitation in complex military environments. In Engineering of Comput- er-Based Systems, 1999. Proceedings. ECBS'99. IEEE Conference and Workshop on (pp. 328-334). IEEE.
IEEE Computer Society. (2008). IEEE guide adoption of the Project Management Institute (PMI®) Stand- ard. New York: IEEE.
Jain, H., Vitharana, P., & Zahedi, F. M. (2003). An assessment model for requirements identification in component-based software development. ACM SIGMIS Database, 34 (4).
Kitapci, H., & Boehm, B.W. (2007). Formalizing informal stakeholder decisions--A hybrid method ap- proach. HICSS 2007. 40th Annual Hawaii International Conference on System Sciences
KPMG. (2005). Global IT project management survey.
Larman, C. (2004). Agile and iterative development. Boston, MA, USA: Addison-Wesley.
Lamsweerde, A. V. (2000). Requirements engineering in the year 00: A research perspective. Paper pre- sented at the 22nd International Conference on Software Engineering, Limerick, Ireland.
Lee, S. W., & Rine, D. C. (2004). Missing requirements and relationship discovery through proxy view- points model. 2004 ACM Symposium on Applied Computing.
Lindquist, C. (2005). Required: Fixing the requirements mess; The requirements process, literally, deciding what should be included in software, is destroying projects in ways that aren’t evident until its too late. Some CIOs are stepping in to rewrite the rules. CIO, 19 , 53-60.
Maiden, N. A. M., & Rugg, G. (1996). ACRE: Selecting methods for requirements acquisition. Software Engineering Journal, 11 , 183-192.
Majchrzak, A., Beath, C. M., Lim, R. A., & Chin, W. W. (2005). Managing client dialogues during infor- mation systems design to facilitate client learning. MIS Quarterly, 29 (4), 653.
Marakas, G., & Elam, J. (1998). Semantic structuring in analyst acquisition and representation of facts in requirements analysis. Information Systems Research, 9 , 37-63.
Miller, R. A., & Luse, D. W. (2004). Advancing the IS curricula: The identification of important communi- cation skills needed by IS staff during systems development. Journal of Information Technology Edu- cation, 3 , 117-131. Retrieved from http://www.jite.org/documents/Vol3/v3p117-131-107.pdf
Mitev, N. (2000). Toward social constructivist understandings of IS success and failure: Introducing a new computerized reservation system. Proceedings of the Twenty First International Conference on Infor- mation Systems , Brisbane, Queensland, Australia ACM.
Moløkken-Østvold, K., & Jørgensen, M. (2003). A review of software surveys on software effort estimation. Paper presented at the 2003 International Symposium on Empirical Software Engineering, 2003. ISESE 2003.
Nee, N.Y. (2009). Ensuring requirements gathering success in an agile environment. ESI Horizons, July
- Retrieved from http://www.esi-intl.co.uk/horizons/publication/2009/200907_agile.asp
Nguyen, L., Armarego, J., & Swatman, P. (2002). Understanding requirements engineering: A challenge for practice and education. School Working Papers Series 2002. Faculty of Business and Law, Deakin University.
Davey & Parker
Pan, G. S. C. (2005). Information system project abandonment: A stakeholder analysis. International Jour- nal of Information Management, 25, 174.
Pearson, D. D. R. (2012). Learning technologies in government schools. Victorian Government Auditor General.
Pitts, M. G., & Browne, G. J. (2004). Stopping behavior of systems analysts during information require- ments elicitation. Journal of Management Information Systems, 21 , 213.
Pitts, M. G., & Browne, G. J. (2007). Improving requirements elicitation: An empirical investigation of procedural prompts. Information Systems Journal, 17 , 89-110.
Raisinghani, R., Riggen, R., & Ryan, K. (2014). Adopting an agile methodology requirements-gathering and delivery. PricewaterhouseCoopers. Retrieved from https://www.pwc.com/en_US/us/insurance/publications/assets/pwc-adopting-agile-methodology.pdf
Reichental, J. (2006). An evaluation of the effectiveness of interview techniques in the elicitation of tacit knowledge for requirements engineering in small software projects. Nova Southeastern University.
Robertson, S. (2001). Requirements trawling: Techniques for discovering requirements. International Journal of Human-Computer Studies, 55 , 405-421.
Saiedian, H., & Dale, R. (2000). Requirements engineering: Making the connection between the software developer and customer. Information and Software Technology, 42 (6), 419-428.
Sauer, C., Southon, G., & Dampney, C. N. G. (1997). Fit, failure, and the house of horrors: Toward a con- figurational theory of IS project failure. Paper presented at the Eighteenth International Conference on Information Systems, Atlanta Georgia.
Sauer, C., & Cuthbertson, C. (2002). UK project management is healthier than supposed, CW360.com sur- vey suggests. Computerworld.
Savolainen, P., Ahonen, J. J., & Richardson, I. (2012). Software development project success and failure from the supplier's perspective: A systematic literature review. International Journal of Project Man- agement, 30 (4), 458-469. doi: http://dx.doi.org/10.1016/j.ijproman.2011.07.
Shepherd, D. A., Patzelt, H., & Wolfe, M. (2011). Moving forward from project failure: Negative emo- tions, affective commitment, and learning from the experience. Academy of Management Journal, 54 (6), 1229-1259. doi: 10.5465/amj.2010.
Stoica, R., & Brouse, P. (2013). IT project failure: A proposed four-phased adaptive multi-method ap- proach. Procedia Computer Science, 16 , 728-736. doi: http://dx.doi.org/10.1016/j.procs.2013.01.
Taylor, A. (2000). Professional practice: IT projects: Sink or swim? The Computer Bulletin. Retrieved from http://archive.bcs.org/bulletin/jan00/article1.htm
Tsumaki, T., & T. Tamai (2006). Framework for matching requirements elicitation techniques to project characteristics. Software Process Improvement and Practice, 11 , 505-519.
Tut, D. (2009). 2009: 22 high-profile IT failures. Retrieved 20 March, 2013, from http://www.computable.nl/artikel/nieuws/ictbranche/3203753/2379258/2009-22-spraakmakende- ictfiascos.html
Urquhart, C. (1999). Themes in early requirements gathering: The case of the analyst, the client and the student assistance scheme. Information Technology and People, 12 (1), 44.
Xiangnan, L., Hong, L., & Weijie, Y. (2010). Analysis failure factors for small and medium software pro- jects based on PLS method. Paper presented at the Information Management and Engineering (ICIME), 2010 The 2nd IEEE International Conference on 16-18 April 2010.
Zave, P. (1997). Classficiation of research efforts in requirements engineering. ACM Computing Surveys, 4 (29), 315-321.
