We have to thank the Russians for the expression, ‘trust but verify’ (doveryai, no proveryai). Suzanne Massie, a scholar of Russian history, taught it to Ronald Reagan while he was president. He then used it to talk about nuclear disarmament discussions with the Soviet Union. Since then, it has been used extensively in many geopolitical contexts, especially those related to security, where you provide the carrot (we trust you), then the stick (no, we don’t). Long before it became an English proverb, it was being used in practice, especially in the emerging semiconductor industry. Back when I was an undergraduate doing my dissertation, I encountered several forms of it. I was designing a universal in-circuit emulator and used a pre-production version of the Hilo2 simulator to verify it. Yes, I did find bugs in the simulator, which led to it providing incorrect verification data. Tools are never perfect, and this remains a potential source of problems for every stage of the design, verification, and implementation process. This is probably mild compared to the types of errors we can expect from AI. Second, I found several errors at the boundaries of specification – a very real problem for most designs even today. In this case I misread a specification, which led to the wrong polarity on an enable signal. If that had been in a real device, I would have blown up most of my drivers. And yes, there were many design errors. I was young, inexperienced, and in a hurry. Later in my career, I encountered two more types of specification errors. The first of these was an omission in the specification. An undocumented opcode led the processor to violate the hardware/software contract, an error that had serious consequences for every safety-critical design that used that processor. Another error was discovered where the interrupt subsystem of a processor did not match the specification. It was unclear which was right, but it would have caused problems if you had designed according to the specification. When using AI, especially in the context of interpreting a specification to create a set of properties, it most certainly cannot be trusted. It is probably not as good as a junior engineer today. Plus, it is being given a totally inadequate specification to start with. But there are real fears about it cheating, as well. In multi-agentic frameworks, trust will be difficult to ascertain without looking at full provenance information — assuming that is provided and hasn’t been tampered with, as well. We already have several real examples where AI has worked out how to cheat in order to attain a goal. Unless a spec-to-properties tool is forbidden from accessing anything except the specification, it will likely create vacuous properties by peeking at the design, especially if the goal is to increase verification coverage. Even if told it cannot look at anything except the spec, it could ask another agent to do it, or ask another agent to update a specification so it becomes available. There are many ways for AI to cheat, and I don’t think anyone is treating that as a potential problem today. We not only need to use what AI produces to perform the verification task, as we do today, but we must also verify everything it produces, including its provenance. Doing that requires someone with enough skill to know if they are being conned. Many people already talk about AI being able to produce stuff that looks plausible, but which is nonsense. This is not just about verification. It’s about verification of trust. As soon as a goal is set, AI has everything it needs to cheat. We are racing ahead without putting in the necessary safeguards. This is happening everywhere AI is deployed, not just in the semiconductor development flow. We are all racing for higher productivity, faster turnaround times, but it will stop, or at least pause, when someone notices that AI has created false trust that leads to failure. Where are the Asimov (another Russian) laws of robots for AI? Where are the safeguards and rules that all AI is bound by? Nobody has stopped to define them, and that will hurt us all in some manner. But that is only the beginning of the implications of having AI do this task. Many people have told me that AI will take over the mundane tasks, allowing engineers to be more creative. That is a dream, not reality. It will indirectly create more, but different, mundane tasks, like checking everything AI has done. In an article I wrote this month, several people mentioned that AI will be able to create lots of properties almost instantaneously, but every one of those has to be manually checked. This is instead of a human creating a few high-quality properties that are used in verification. Maybe senior engineers will spend more time creating better specs and training data on which AI can learn. Then the junior engineers will do the trust-but-verify for the AI-created outputs. In that case, the creative part has been replaced by more mundane tasks. Creation was the fun part. Now they are spending more time in verification. They will spend most of their time checking up on what AI has done. Will that lead to moral issues within a team? Perhaps, but this is also looking at how to fit AI into the flows we already have. It goes against the notion of allowing the junior engineers to be the ones with the truly creative streak, not bound by the constraints of today’s methods, so long as they understand that they are junior and that means they make mistakes — not on purpose, but because of a lack of maturity and experience. Run fast, make mistakes, and learn. Engineers do that. Can AI do that? And does it have a moral compass to guide it? The post Trust, But Verify appeared first on Semiconductor Engineering.
Source: https://semiengineering.com/trust-but-verify/