By Richard Goering on February 1, 2010.
This article is reposted from the Cadence blog.
Can verification engineers gain control over the verification process, and stop being full-time firefighters? With proper planning, communication, and organization, the answer is “yes,” according to Allison Goodman, validation program manager at Intel for client and enterprise solid state hard drives.
Goodman spoke at a Silicon Valley DVClub lunch meeting January 26 at Dave and Buster’s restaurant in Milpitas, California. DVClub is an interesting organization. With chapters in Austin, Bangalore, Boston, Dallas, Research Triangle Park, San Diego, and Silicon Valley, the club’s stated purpose is “to have fun while helping build the verification community through quarterly educational and networking events.” IC engineers can join for free, and events are free. Costs are picked up by sponsors, including Cadence.
The January 26 event brought together around 120 attendees. There were a few EDA folks, but as far as I could tell, most attendees were verification engineers. Goodman’s speech was entitled “Tales from the trenches – validation missteps making us full time firefighters.” Goodman started her speech by noting that “it’s not technical problems that cause bad things to happen. It’s usually on the people side.” She identified four “missteps” that force engineers to put out fires rather than proactively validate a product’s quality.
Misstep #1: Insufficient planning
Insufficient planning occurs when you don’t have what you need to do testing, and your test coverage falls short. It’s caused by undocumented assumptions, the increasing scope of projects, and “missed dependencies” (you need 10 prototypes but only get 5). “If you don’t plan for it, it will surprise you, and every surprise will end up as a fire.”
The solution? Put your plan in writing – including who does what, how features work, what it means to be “done,” what checkpoints will monitor progress, and criteria for success. Keeping track of assumptions may be the biggest part of the solution. Write them down!
Misstep #2: Not designing for test
Designers often think their designs won’t have any mistakes, so there’s no plan for testing and no communication with validators. This makes it difficult to find and replicate bugs, to figure out what you need to monitor, and to know when you’re done. Interpreting test results as “pass” or “failure” may be very difficult. The antidote is for validators to get involved in the earliest stages of the design process. “Ask how you’re going to test it and how you’re going to tell if it’s working.”
DVClub provides an opportunity for networking as well as speakers and lunches.
Misstep #3: Not creating and integrating feedback loops
All too often, the marketing team or the design engineers make changes to a product, and don’t communicate those changes to the verification team. Further, many companies place engineers in “silos” with little or no communication – for example, there are software engineers, hardware engineers, and firmware engineers who don’t talk to each other.
What’s needed is continuous feedback about any changes in the product, as well as problems found with the product. Tests should be monitored for effectiveness and continually improved.
Misstep #4: Lack of transparency
Lack of transparency happens when you tell your boss (or team) that everything is well when it really isn’t. Or, you skimp on tests and coverage as schedule pressure rises, and don’t let managers know. As a result, risks and coverage gaps increase. “Tell the real story, and encourage others to do the same. Don’t declare that it’s done until it’s really done.”
My takeaway
While there are tools that can help with verification planning and monitoring – such as Cadence Incisive Enterprise Manager – quality verification depends on “people” factors such as whether and how verification teams plan, how early they’re involved with the design process, how well and how honestly people communicate, and how adaptable teams are to feedback and change. Pay attention to these issues and perhaps you can put the fire extinguishers away.
Richard Goering