One concern with this approach is it may be too easy to convince someone that a plan is flawed… There are a few natural ways to mitigate this problem

You could also try to calibrate the standards by using a control group consisting of plans that are known by the relevant experts to be either sound or flawed. Provided you are unfamiliar with these plans yourself, you could then experiment with different compositions of teams A and B, more or less rounds of criticism, etc., and see which arrangement makes you most likely to accept the sound plans, and reject the flawed ones.