Probing and Fault Injection of Distributed Protocol Implementations
This paper presents a technique for probing and fault injection
of distributed protocols. The proposed technique, called
script-driven probing and fault injection, can be used for
studying the behavior of distributed systems and for detecting
design and implementation errors of fault-tolerant protocols.
The focus of this work is on fault injection techniques that can
be used to demonstrate three aspects of a target protocol: i)
detection of design or implementation errors, ii) identification
of violations of protocol specifications, and iii) insight into
design decisions made by the implementors. The emphasis of our
approach is on experimental techniques intended to identify
specific ``problems'' in a protocol or its implementation rather
than the evaluation of system dependability through statistical
metrics such as fault coverage.
To demonstrate the capabilities of this technique, the paper describes
a probing and fault injection tool, called the PFI tool
(Probe/Fault Injection Tool), and several experiments that studied the
behavior of two protocols: the Transmission Control Protocol (TCP) and
the Group Membership Protocol (GMP). The tool can be used to delay,
drop, reorder, duplicate, and modify messages. Furthermore, by
selective reordering of messages and spontaneous transmission of new
messages, we were able to guide a distributed computation into hard to
reach global states without instrumenting the protocol implementation.
Back to
Publications list.
sdawson@engin.umich.edu