Compare against real silicon behaviour

[athre0z]

Something that we had planned to look into sooner or later is diffing against the behaviour observed from actual CPUs, in a sandsifter-like fashion. Would that be beyond the scope of this project or would you be interested in adding something like that as well?

[woodruffw]

Would that be beyond the scope of this project or would you be interested in adding something like that as well?

I think it's definitely in-scope for the analysis component of mishegos! I had some thoughts re: using microx, but I'd definitely be interested in integrating sandsifter's techniques as well.

[athre0z]

That's good news!

Using microx for this would come with the rather big limitation that it appears to be relying on the information provided by XED, thus will likely be unable to execute / spot errors with instructions that are bugged in XED. The sandsifter approach doesn't require reliance on any software disassembler and would thus be the better fit IMO.

[woodruffw]

Using microx for this would come with the rather big limitation that it appears to be relying on the information provided by XED, thus will likely be unable to execute / spot errors with instructions that are bugged in XED. The sandsifter approach doesn't require reliance on any software disassembler and would thus be the better fit IMO.

Yes, that's a good point. My main concern with sandsifter was that (IIRC) it uses capstone as its ground truth during triaging, but that's completely separate from the execution process.

[sabastiaan]

I modified sandsifter a while back to use a different ground truth and output some other info. From what I recall the only time it uses capstone was during printing info. It was quite easy to completely remove and for example swap it for XED or just leave it behind all together. The code was quite complex and hard to work with though. Although you could by pass a lot of it's complexity by ignoring everything besides the inject function. All you need to do is include this file.

From the looks of it microx doesn't actually run an instruction on the CPU it self but tries to model its behavior with XED. I would say that in the context of the current project it would basically look like another worker.

My only concern with sandsifter would be that I would expect the performance to be way lower than you otherwise would have, considering that raising traps is a core mechanic of it's search. But this would be inherent to this problem.

[woodruffw]

I modified sandsifter a while back to use a different ground truth and output some other info. From what I recall the only time it uses capstone was during printing info. It was quite easy to completely remove and for example swap it for XED or just leave it behind all together. The code was quite complex and hard to work with though. Although you could by pass a lot of it's complexity by ignoring everything besides the inject function. All you need to do is include this file.

Awesome! Good to know that it's feasible.

From the looks of it microx doesn't actually run an instruction on the CPU it self but tries to model its behavior with XED. I would say that in the context of the current project it would basically look like another worker.

It does both, kind of: microx uses XED to transform the instruction and JITs it with some tweaks, then executes it in a special execution area. You can see most of the code that does that here: https://github.com/lifting-bits/microx/blob/03feed3ff843bfb3c03b2f72220703ad11a79cb2/microx/Executor.cpp#L1827

OTOH microx wasn't designed with the use case in mind at all (it expects completely valid instructions from the very beginning), so sandsifter + XED will probably be a better foundation.

My only concern with sandsifter would be that I would expect the performance to be way lower than you otherwise would have, considering that raising traps is a core mechanic of it's search. But this would be inherent to this problem.

Yep! That comes with the territory. Mishegos's architecture allows people to enable and disable workers as they please, so a particularly slow one isn't a big deal :-)