Home » IP Systems » Featured Stories » Traceability, the Meta-Metric with a Human Touch

Traceability, the Meta-Metric with a Human Touch

In a recent whitepaper, Jama Software defined traceability as:

“[…]traceability, connects all the data and the artifacts (these are your requirements, test cases, use cases, defects, etc). All the detailed information within your project that really makes up the scope of what it is you are building.  By creating trace relationships between the items, you can track the impact that one change has on the rest of the project, both upstream and downstream. This is vital for maintaining that continuous quality.”

The evolution from metric driven engineering processes to traceability is a fairly clear one.  Simple metrics like code coverage let us know what our existing simulators and/or formal tools along with our testcases were providing in terms of stimulating the design.  Functional and assertion coverage in conjunction with free-standing object oriented checkers provided metrics that indicated we had not only exercised a specific portion of a design configured a specific way, but that we had also checked the functionality of that design.  While all this coverage provided value that could be leveraged by engineers, it provided little or no coherent information to management beyond a set of completion percentages.  With details wrapped in a shroud of percentage counts, it’s perhaps no surprise that the 80/20 rule comes up often among management teams: “The last 20 percent of the project will take 80 percent of the total project time.”  Traceability tools provide the missing link from engineering details such as functional coverage groups to business details such as specific customer requested features.

From the top down, requirements management tools have existed as business software for a number of years.  Management could tell who requested a feature in the design, what its business value was, and even how much revenue they intended to gain based on that feature.  Engineering teams rarely utilized the requirements management system.  Few were even aware of the existence of such tools.  What was needed by both the management and engineering teams was a connecting layer that could leverage the value of the existing requirements tracking systems along with the existing metric driven collection systems.  By interfacing the two systems, both the business and engineering teams gain visibility into the entire chip design process from top to bottom and back to the top again.  A few examples will serve to better illustrate what traceability is and the benefits it offers.  In the first example, we’ll look at traceability as a purely mechanical process for linking different perspectives, (engineering and business), within the same project.  In the second example, we’ll look at traceability as an artifact management process allowing engineering teams to work more effectively by easily laying their hands on existing documents and metrics that are related to a given design block or group of design blocks.  In the final example, we’ll take a look at how business and engineering teams can work together to make more efficient decisions based on objective metrics including both output from their engineering tools and the recorded history of priorities and decisions made in the project, (essentially log books on steroids).

Example I, Get Everyone Talking the same Language:  Functional coverage planning, done correctly and completely, often leads to coverage groups that are cross products of register settings and transaction sequences monitored deep within the design.  While their raison d’être may have been blindingly apparent during the verification planning session, by the time the specified coverage group is passed on to the implementation engineer, there may be no apparent rhyme or reason whatsoever.  Without the context linking the design requirement to the cover group, the implementation engineer is at a loss.  The project as a whole is at a loss as well.  We lose the value the implementation engineer could have added had they been aware of the complete context.

Fig1

That was the bottom-up scenario.  From the top-down, the situation is equally deleterious.   Ideally management should use coverage completeness as one of the gauges indicating that the chip is ready to tape out. Like most indicators, coverage completeness may never read 100 percent.   Management will ultimately need to make a prioritized decision based (hopefully) on available data as to whether or not it’s time to ship the chip design out for fabrication.  When that time comes, a lack of coverage on the same deeply embedded cover group mentioned above is only significant if the management team has an easy way to relate that coverage to the original business requirement that it tracks.  Once again, context means power and efficiency.  Context also means being able to leverage the most value from the project’s contributors.

Genius through Amalgamation:  The second example begins with the definition of an “artifact”.  An artifact is any data or document that is produced by an engineering process, or left behind by project contributors as they do their jobs.  Artifacts are not an actual part of the finished product, they only serve to document its history.  Examples of artifacts include issue reports, specifications, verification plans, register set definitions, use cases, debug spreadsheets, and business requirements definitions, (marketing descriptions of planned features for example).  Traceability tools can be used to store links to artifacts, as well as define relationships between different artifacts, design blocks, and contributors.

When these feature of traceability tools are deployed, they give project contributors easy access to all the information about a given portion of the design.  An engineer can check for the date of the last revision of a given block.  They can then easily access information about the block.  For example, this information could include:

  • how many testcases passed or failed, what was covered by the simulations,
  • how the coverage changed,
  • whether or not the block has been synthesized since its last modification, and
  • the outputs of any physical design tools.

By having all this information easily at hand, engineers can deduce the cause of complex issues, or even anticipate issues based on the available metrics, getting out ahead of problems before they happen.

Fig2

Example 3, Better Decisions through Knowledge Warehousing:  In small tight-knit societies, histories are often passed down through an oral tradition.  When a member of the society needs access to this historical knowledge, they either have it already available through a culture of story-telling, or can approach elders in the society to retrieve the information.  Examples of such societies include the Navajo indians, the Freemasons, and up until recently, chip design teams.  However, with the mushrooming size of design teams, as well as their tendency to be distributed geographically, chip design teams that hope to succeed through a strictly oral culture are quickly becoming an endangered species.  It is increasingly necessary to memorialize project knowledge.  Rational criteria used to make decisions including priorities, engineering data, schedules, and  available resources should be stored in an easily searchable format so that, each subsequent decision can be better made with access to the information that went into the decisions that preceded it.

Traceability tools offer features that make it easy to capture these decision making criteria data as well as tools that make this data easier to access for subsequent decisions.  Many of the available tools contain ready-made links to documentation software such as Microsoft Office.  Furthermore, they provide a way to specify links between a given decision, any documented thoughts on the decision, and the available engineering and business data that went into the decision.  With these capabilities, the meetings, thought processes and priorities that went into making a decision can provide a head start the next time a design block or product requirement needs to be changed.  Rather than having lengthy meetings to determine where the project currently stands and what the impact of a proposed change will be, much of the data that would be produced by the meeting can be acquired immediately and more accurately by the traceability system.  As Jama’s Eric Nguyen pointed out: when large meetings are required, they’re typically used to hash out the finer details [of decisions]; the time spent communicating context to the team at large is obviated by the collaboration and availability of the metrics mentioned above.

Summary

These examples pointed out a few of the usage models and benefits of traceability.  The concept has been around for a while and the need for it is fairly obvious once pointed out.  Like any technique, traceability can be adopted in stages as time and budget permits.  The good news is there currently multiple vendors offering traceability enabling technologies.  Leveraging traceability and the contextual knowledge of your entire organization is easier than it’s ever been.

 

Great information delivered straight to your inbox

One comment

Leave a Reply

Your email address will not be published. Required fields are marked *

*