Issues in the Development and Application of Reuse Metrics in a Corporate Environment

in a Corporate Environment

Jeffrey S. Poulin

International Business Machines Corporation

Abstract transitions in status from “novel” to “expected.”What

we reuse today we may simply “use” three years from

This paper describes the issues addressed when devel-now.

oping and implementing the IBM reuse metrics andreturn on investment model [25].Recognizing the poten-Criteria for reuse metrics: We must easily obtain, inter-tial benefits of reuse has proven instrumental in insertingpret, and implement the required data and metrics.Thereuse in the IBM programming process.However, manyrealities of the development process, organizationalissues arise in a large organization when implementingneeds, and available data affect what we can report [10],metrics due to the diversity in computer languages, envi-[32].Every company must consider the practical limits

ronments, and cultures.This paper presents the optionsof their environment when constructing their metricsand methods to resolve many of these issues [29].model.1.0Overview

The uniqueness of reuse metrics comes from the factthat, unlike productivity metrics, they reflect on what aprogrammer does not do.Organizationsrewardthosewho deliver the most function, thereby encouragingdevelopers to write code.In reuse we want to rewarddevelopers for not writing code.

This paper first discusses the goals and some criteriafor reuse metrics.The paper then describes what tomeasure as reuse andhow to measure reuse.This paperdefines, from the business case perspective, what shouldcount as aReusedSourceInstruction(RSI) and thendescribes the issues that arise when putting the metricsinto practice.

Organizational goals: Ultimately, an organizationdefines what to measure and report as reuse in accordwith its goals [6].Because IBM uses the metrics in aReturn on Investment (ROI) model we believe theymust reasonably quantify the level of reuse in an organ-ization.Specifically, we have the following goals:1.Reuse metrics must reflect effort saved.2.Reuse metrics must encourage reuse.

The real difficulty in reuse metrics lies in judgingwhether a programmeruses or reuses software.Further-more, as a technique becomes ingrained in a culture, it

2.0 What to Measure as Reuse

Use of base product (maintenance): Most softwarespends 80% of its life in maintenance [14].Some mightcount maintenance as reuse because most of the soft-ware from a new release already existed [4].However,when reporting maintenance metrics IBM excludes soft-ware from the product base1 and only reports on new,changed, or deleted code.Reusable components com-pletely new to the product contribute to the reuse levelfor the release, but calls to components in the base donot.

Use of Operating System (O.S.) services: Some countthe use of O.S. services as a form of reuse because theO.S. raises the level of abstraction in programming [23].However, we clearly do not expect to write a new O.S.for every new product [5], and therefore we do notcount the use of an O.S. as reuse.

Use of tools: Programmers routinely use text editors,debuggers, compilers, and library systems.We usetools for their intended purpose; we do not consider theuse of tools as reuse.

Use versus Reuse of components: Differentiatingbetween “use” and “reuse” hinges on whether it savedthe programmer from having to write code.Suppose aprogrammer retrieves a subroutine from a reuse libraryand uses it without modification.Although we count

1The product base consists of all code previously released in the product.

this as reuse, we do not count each subsequent call2 tothe subroutine as reuse.3

Use of pre-requisite products: Programmers have manyutilities available to them, such as database systems andgraphics packages.One could count the use of theseutilities as reuse [13].However, although these utilitiessave writing code, we do not count it as reuse [5].Wenot only expect the use of these utilities, reporting useof a database system as reuse does not accurately reflectreuse activity.

Application Generators: Application Generators cancreate large quantities of code from specifications givenin tables, pictures, templates, or fourth generation lan-guages [9].Some might consider the use of automaticprogram generators as a form of reuse [21].Clearly,these tools provide an enormous advantage to the pro-grammer.4 However, we simply cannot compare thisresult with that of traditional methods; to obviate thisproblem IBM reports this code separately.

Use of High Level Languages (HLLs): HLLs couldcount as reuse because they raise the level of abstractionneeded to program much as do abstract data types andclass libraries [21].However, we expect the use ofHLLs and do not count it as reuse.Code libraries: Counting reuse by source:

Use of utility libraries: The most basic libraries consistof the essential or required utilities in a given language(e.g., stdio in “C”).We expect their use and do notinclude them in RSI counts.

Use of standard libraries: Second come the locallibraries every programmer must know to contribute tohis group (e.g., memory functions in an O.S.).We con-sider these the local equivalent of utility libraries and donot generally count them as reuse.

Project libraries: Third, good design can result in well-structured programs and classes or superior use of lan-guage features such as generics and inheritance.Some“reuse” will result because the design or languageencourage it.However, we have to recognize when wereally save effort.We resolve the issue by deciding ifthe use falls outside the scope of normal good softwaredesign.

Domain-specific libraries: Fourth come the domain-specific libraries developed by organizations with exten-sive experience in an application area (e.g., flightcontrol functions in aerospace) [31].Although weexpect a programmer in a domain to use these libraries,we count them as reuse.IBM encourages the develop-ment of domain libraries because they have proven toresult in significant savings [22].

Corporate reuse libraries: Finally, reusable assets mayreside in a shared, corporate database of fully tested,documented, and maintained components.We count theuse of these parts as reuse.

White box versus Black box reuse: We find the bene-fits due to white box reuse (copying and modifyingexisting software) much less than those of black boxreuse (use of unmodified parts).Although black boxreuse costs more due to the additional effort required tomake software reusable, we quickly recover this costbecause we only have to maintain one part.At IBM weonly count black box reuse.5

Porting versus Reuse: Formal reuse includes porting.However, counting porting as “reuse” may distort per-ceived benefits because we include porting in our busi-ness plans.A further reason to exclude porting comesfrom the fact that porting normally involves minor adap-tations or simple recompilation.To include ported codein reuse metrics would result in very high reuse values;to prevent this distortion, we do not count porting asreuse.6

2In one example, a group reported 512 calls to the same part from the same module as reuse.

3Likewise, we may reuse class definitions in object-oriented languages but not the class instantiations.

4One group reported a 95% reuse level on a 104 kloc product in which approximately 99 kloc came from a program generator.5Some organizations track the amount of white box reuse in their products to emphasize the “total leverage” gained by copyingand modifying software.

6Organizations tasked with porting software separately track the amount of porting they do in their projects.

Organizational boundaries: We expect programmers touse good software engineering principles and to sharesoftware within their groups.Normally, only the groupwhich designed and maintains a component would use itin a new context or program.Other groups may neverknow about the component.Therefore, we seek toencourage reuse by counting a part as reused only whenit crosses these organizational boundaries.7

Differences in organizations: In large organizationssuch as IBM there exists a diversity of expertise withregard to any technology, including reuse.The varyinglevels of reuse process maturity [18], [20] becomes anissue.Either an organization encourages formal reuseby adopting a very strict definition of reuse, or the pro-gramming culture has so fully accepted a model of reusethat they feel they must further adapt this reuse defi-nition.Because we want everyone to adopt reuse, wefocus less on achieving high levels of reuse than onconstant improvement.

Table 1. Summary:WhattomeasureasReuseType of ReuseMaintenanceOperating SystemToolsMultiple UsesPrerequisite productsApplication generatorsHigh level languagesUtility librariesStandard librariesProject librariesDomain-specific librariesWhite boxBlack boxPorting

Measured?NoNoNoOne timeNoSeparatelyNoNoMaybeMaybeYesNoYesSeparately

Units of measurement: Numerous units in softwareexist: Lines of Code (LOC), Function Points (FPs) [2],[12], semantic tokens, equivalent lines of assembler,number of methods, number of features [24], number ofclasses, etc.However, most companies, including IBM,use LOC [7] despite their deficiencies [17].Further-more, LOC indicate overall code productivity and serveas a good secondary indicator in other phases [33].Studies also show that methods such as FPs and LOCcorrelate highly [1].8

Function written versus function used: We believeimplementation details must not effect the metrics.Forexample, the use of macros (with multiple in-line expan-sions) versus subroutines should not affect the metricvalue.Separating the portion of a product actuallywritten from the portion created by a tool makes com-parisons difficult; the standard productivity metrics donot work.

To meet this need, we developed the UsedInstruc-tion, (UI) [15].We calculate the UI from the total func-tion delivered to the customer, reported as if we wrotethe entire software product as in-line code (no macros,inheritance, 4GLS, etc.).The difference between the“function written” and the “function used” indicates howwell we program.The ratio of “function written” toRSI, or “function not written,” indicates the level ofreuse.

LOC we would not have written: Reusable parts oftencontain additional code to make the part more general.We decided that the cost of identifying these occur-rences and the relatively small amount of code involvedmade it acceptable to count what we would not havenormally written.

Definition of \"ship\": We report software metrics effec-tive as of the product “ship date.” However, althoughthis term pertains to external customers, we also use themetrics for internal products.

When to report metrics: To provide in-process feed-back to developers, we encourage metrics if identifiedas “in-process.” This way reviewers understand the

3.0 How to Measure Reuse

7Software development organizations vary, but for measuring reuse we define a typical organization as either a programmingteam, department, or functional group of about eight people or more.

8For assessing the level of reuse in object-oriented projects we base the portion of reuse on the portion of reused classes.Ourinternal OO community not only recommends this unit but we do not always have more fine-grained data (e.g., the portion ofmethods reused) [11].

number may change and we can still monitor progresson products that may take years of planning and devel-opment.

Reporting by quality of reuse: IBM defines three levelsof quality for reusable parts “as-is,” “complete,” or “cer-tified”[19] .Like IBM, companies that certify partsmay encourage reuse of these parts and identify neededparts by optionally reporting by quality level.

Availability of data: We must have easy data tocollect, normalize, and report.Where data does notexist, we must make decide either to make the invest-ment to obtain it or to use an alternative strategy.9 Wecalculate the IBM reuse metrics from the followingobservable data elements used in IBM and other compa-nies [10], [15]:•••••••

Shipped Source Instructions (SSI)Changed Source Instructions (CSI)Reused Source Instructions (RSI)

Source Instructions Reused by Others (SIRBO)Software Development Cost (Cost/LOC)

Software Development Error Rate (Errors/LOC)Software Error (Cost/Error)

not consider the relative efforts of the sub-organizations.

3.Sum SSI and RSI.IBMusesthisoptionbecauseitprovides a weighted average of the levels of reusein each of the sub-organizations.

Table 2. DerivedMetricsMetric

Reuse Percent• for products• product releases• organizationsReuse Cost Avoided

SymbolReuse%

Derivedfrom:

• SSI, RSI• CSI, RSI• SSI, RSISSI or CSI,RSI,

Cost/LOC,Errors/LOC,Cost/ErrorSSI, RSI,SIRBO

Unit ofMeasurePercent

RCADollars

Reuse Value AddedRVARatio

4.0Conclusion

This paper discusses the issues surrounding develop-ment and implementation of reuse metrics; we requiredresponsible and equitable resolution of the issues fortheir acceptance.Technical leaders demand responsiblesolutions because they insist on realistic and accuratevalues upon which to form their development decisions.Managers demand equitable solutions because they getjudged and compared based on performance and results.These issues define the rules for such comparisons.This paper has no known equivalent in the attentionit gives to the definition of RSI and in attempting topresent reuse as “real effort saved.” Although [3] differ-entiates between reuse within and external to the organ-ization, no other group provides a detailed explanationof how they count reuse.5.0 Cited References[1]

Albrecht, A.J., and J.E. Gaffney, “Software Func-tion, Source Lines of Code, and DevelopmentEffort Prediction: A Software Science Validation,”IEEETSE, SE-9, 1983, pp.639-648.

Combining the data: To develop metrics that reason-ably reflect the levels of reuse, we carefully defined“reuse.”We then asked:

1.Tell me the current level of reuse.

2.Tell me the value of the reuse, in dollars.

3.Tell me how I encourage building reusable parts.We developed three reuse metrics to answer thesequestions [26], [27], [28], [30].The first two metricsindicate the level of reuse activity in an organization asa percentage of products and by financial benefit.Thethird metric includes recognition for writing reusablecode.Table2 summarizes the three metrics.\"Rolling-up\" the numbers: Options consist of:

1.Vary organizational boundary.Higherlevelman-agers only count parts from outside their organiza-tion.Although they may have high levels of reusewithin the organization, the level may shrink whenrolled-up.

2.Average of all sub-organizations.Althougheasytocompute, this option could mislead because it does

9For example, in the case where a site collects process data by function point and not LOC, we replace LOCs in the metric withfunction points so they report reuse by “reused function points.”

[2]

Albrecht, A.J, “Measuring Application Develop-mentProductivity,” Proc. Joint IBM/SHARE/GUIDE App. Symp.,Oct.1979,pp.83-92.[3]

Banker, R.D., R.J. Kauffman, C. Wright, andD.Zweig, “Automating Output Size and Reusa-bility Metrics in an Object-Based Computer AidedSoftware Engineering (CASE) Environment,”unpublished, 25 Aug.1991.

[4]

Barns, B.H. and T.B. Bollinger, “Making ReuseCostEffective,” IEEE Software,V.8,N.1,Jan.1991, pp.13-24.

[5]

Bollinger, T.B., and Pfleeger, S.L, “Economics ofreuse:issues and alternatives,” Inf.SWTech.,V.32, N.10, Dec.1990, pp.643-52.

[6]

Basili, V.R., and D.M. Weiss, “A methodologyfor collecting valid software engineering data,”IEEETSE, SE-10, 1984, pp.728-738.

[7]Boehm, B.W., “Improving Software Productivity,”IEEEComputer, 20, 1987, pp.43-57.

[8]

Bourland, D.D. and P.D. Johnston, ed., To Be orNot: An E-Prime Anthology,Int.Soc.forGeneralSemantics, SF, CA, 1991.

[9]Dabin, “Software Reuse and CASE tools,”Proc.Int. Comp. SW&Apps. Conf.,1991.

[10]

Daskalantonakis, M.K., “A Practical View of Soft-ware Measurement and Implementation Experi-enceswithin Motorola,”IEEETSE, V.18, N.11,Nov.1992, pp.998-1010

[11]Devries, P., “Object Oriented Metrics,” IBMDoc.(draftV1.0), Dec.23 1992.

[12]Dreger, J.B.Function Point Analysis, Prentice-Hall, Englewood Cliffs, NJ, 1989.

[13]

Favaro, J., “What price reusability?A casestudy,” Ada Letters,V.11,N.3,Spring1991,pp.115-24.

[14] “SoftwareEngineeringStrategies,”StrategicAnal-ysisReport, Gartner Group, Inc., Apr.30, 1991.[15]“Corporate Programming Measurements (CPM),”V4.0, IBM Int. Doc.,1Nov.1991

[16]

Gaffney, J.E. and R.D. Cruickshank, “A GeneralEconomics Model of Software Reuse,”Proc.Int.Conf. SW Eng.,Melbourne,11-15May1992,pp.327-337.

[17]

Firesmith, D.G., “Managing Ada projects: thepeopleissues,” Proc. TRI Ada'88,Charleston,WV, 24-27 Oct.1988, pp.610-19

[18]

Humphrey, W., “Characterizing the SoftwareProcess: A Maturity Framework,” IEEESoftware,Mar.1988, pp. 73-79.[19]“IBM Reuse Methodology: Qualification Stand-ards for Reusable Components,”IBMDoc.No.Z325-0683, 2 Oct.1992.

[20]Kolton, P. and A. Hudson, “A Reuse Maturity

Model,” Pos. Paper, 5th Ann. WS on SW Reuse,Center for Inn. Tech., Herndon, VA, 18-22Nov.1991.

[21]Krueger, C.W., “Software Reuse,”Computing

Surveys, V.24, N.2, Jun.1992, pp.131-83.

[22]Margano, J., and L. Lindsey, “Software Reuse in

the Air Traffic Control Advanced AutomationSystem,” Joint Symp.&WorkShops: Improving theSW Proc. and Comp.Pos.,Alexandria,VA,29Apr-3 May 1991.

[23]McCullough, P., “Reuse: Truth or Fiction,”panel

position, OOPSLA 92, Vancouver, CAN, 18-22Oct.1992.

[24]Mukhopadhyay, T. and S. Kekre, “Software Effort

Models for Early Estimations of Process ControlApplications,”IEEETSE, V.18, N.10, Oct.1992,pp.915-924.

[25]Poulin, J.S. and W.E. Hayes, “IBM Reuse Meth-odology: Measurement Standards,”IBMCorp.Doc. No. Z325-0682,2Oct.1992.

[26]Poulin,J.S.“MeasuringReuse,” Proc. 5th Int. WS

on SW Reuse,PaloAlto,CA,26-29Oct.1992.[27]Poulin, J.S. and J.M. Caruso “A Reuse Measure-ment and Return on Investment Model,”Proc.2ndInt. WS on SW Reusability,Lucca,Italy,24-26Mar.1993, pp.152-166.

[28]Poulin, J.S. and J.M. Caruso, “Determining the

Value of a Corporate Reuse Program,”Proc.IEEECS Int. SW Metrics Symp.,Balt.,MD,21-22May.1993.

[29]Poulin, J.S., “Rationale and Criteria for the IBM

ReuseMetrics,” IBM Tech. Report TR 00.3726,15Apr.1993.

[30]Poulin, J.S., D. Hancock and J.M. Caruso, “The

Business Case for Software Reuse,” to appear,IBM Systems Journal,V.32,N.4.,1993.

[31]Prieto-Diaz, R., “Domain Analysis for

Reusability,” Proc. COMPSAC '87,1987,pp.23-29.

[32]Reifer, D.J., “Reuse Metrics and Measurement- A

Framework,” NASA/Goddard 15th Ann. SWEng.WS, 28 Nov.1990.

[33]Tausworthe, R.C, “Information models of software

productivity: limits on productivity growth,”Jour.Sys. & SW,V.19,No.2,Oct.1992,pp.185-201.