Debug is starting to be rethought and retooled as chips become more complex and more tightly integrated into packages or other systems, particularly in safety- and mission-critical applications where life expectancy is significantly longer.
Today, the predominant bug-finding approaches use the ubiquitous constrained random/coverage driven verification technology, or formal verification technology. But as designs become more complex, new methodologies and approaches need to be applied to ensure quality over time. This can include everything from AI/ML to simply updating methodologies to include more automation and less bug tracking in a notebook or spreadsheet.
“For constrained random, the usual approach is to create a coverage model from the verification plan and objectives, which serves as a map of where to target the stimulus to test the important design features thoroughly,” said Mike Stellfox, a Cadence fellow. “In a constrained random verification scenario, the tests are generating all kinds of random stimulus that exercise different paths through the design that you wouldn’t necessarily think of as a human, and bugs will be found. Because you have coverage goals, you know whether you exercised what you thought were the important things. What typically happens is a bug will be found, such as a certain behavior, either triggered by a checker identifying that this didn’t behave correctly, or another second-order effect. This could include not having a direct checker, a state machine hung, or another erroneous behavior occurred. Once the bug is found, a root cause analysis will be done with the designer to determine the cause, and that’s where you go through the debug process.”
Usually, multiple people are involved in finding a bug. It can involve the designer, who accidentally inserted it, and the verifier who actually discovered it. The designer fixes it, and as part of that process they may sure the problem is fixed the way it was supposed to be fixed. That’s where there is an opportunity to do things more systematically. Today, this mostly depends on the experience and intuition of the verification engineer.
“They will then look for ‘cousin bugs,’ since the bug may have been fixed under a set of certain stimulus conditions,” Stellfox said. “The nice thing about constrained random is if you keep things a little bit open, besides the original test, you can run lots of other stimulus in that area. You try to soak the area to see if you’ll find other cousin bugs. This is an area where there are opportunities to leverage a data-driven approach.”
The same thing happens in formal verification. “What’s become really popular in formal verification is bug hunting, where the goal isn’t to close on a proof,” he noted. “This is similar to what happens in a constrained random approach, but instead of using a constrained random stimulus engine to exercise a design, you’re using a formal engine to exercise it, and these use a similar concept of coverage points to look for the areas of concern. The nice thing with formal technology is the ability to explore a lot more space relatively quickly, under all cases. But when you’re in bug hunting mode, you’re typically trying to stimulate the design in some specific area to find bugs while you’re achieving your coverage goals.”
A number of industry players are investigating way to automate the debug process. As systems become more complex, and as tolerances tighten, what used to be considered third-order problems are now second- or first-order problems. That means bugs need to be hunted down in more places. Good starting points include RTL and testbenches, where variables are typically introduced or where most of the changes occur.
