A theory of operation is a description of how a device or system should work. It is often included in documentation, especially maintenance/service documentation, or a user manual. It aids troubleshooting by providing the troubleshooter with a mental model of how the system is supposed to work. The troubleshooter can then more easily identify discrepancies, to aid diagnosis of problem.
IBM Redbooks are "Theories of operation" of their products.
References
Theory of operation Wikipedia(Text) CC BY-SA