herwin
Posts: 6057
Joined: 5/28/2004 From: Sunderland, UK Status: offline
|
quote:
ORIGINAL: dwg quote:
I asked Sid and confirmed that he meant he has inferred that the code does certain things based on its behaviour. That's a valid claim--particularly from a tester--and requires no explicit code knowledge. There is an essential caveat to that, and to black-box testing in general. The completeness/validity of the assumptions you draw depends very heavily on whether you know all of the possible inputs and outputs to the piece of code in question. If you know all of the inputs then you can make definitive assertions about the behaviour, if you don't know all of the inputs, then there may be fundamental sensitivity to conditions you aren't testing. If you don't know all the outputs, there may be side effects you are completely unaware of that invalidate what you assume is identical behaviour across several different sets of inputs and the resultant outputs. (Simple example for the non-coders, imagine testing the behaviour of a piece of code by varying A and B, the two inputs you know about, but what if there is a third input, C, and the code says that if C is false, reverse the result..., your assumptions based on the behaviour observed with A and B is then only true in some cases). I don't know anything about the internal coding of AE, but what I've read in various places hints that the code is very likely riddled with conditional rules based on scores of different inputs, and not knowing what those inputs are and which are significant in which piece of code means we draw assumptions from black-box testing at very considerable peril of those assumptions being incomplete if not outright wrong. I would be very loathe to say 'X is true' about the code without one of the development team having confirmed it first. I'm in agreement. After a while, particularly if you've run sensitivity tests like el_cid has, you will have a reliable black box model of what the system does. JFD once described it as the COW effect--a lot of the details come out in the wash--even up in the end--if you have a valid model. If you've been overrun by the detail, you get another effect--see Military Modelling for Decision Making, 3rd edition, Wayne P. Hughes, Jr., ed., 1997. Starting on page 50, he writes "On Model Stricture, or Stifling Thought", where he criticises the way complicated opaque models inhibit imagination and creativity. "Too much detail for the sake of realism is confusing and self-defeating." He then gives examples of systems that had to be abandoned because they "contain coding mysteries that produce counterintuitive, unexplained results."
_____________________________
Harry Erwin "For a number to make sense in the game, someone has to calibrate it and program code. There are too many significant numbers that behave non-linearly to expect that. It's just a game. Enjoy it." herwin@btinternet.com
|