The TLDR: Too busy to read? Here's a quick summary of article The article compares a manually generated expert failure modes and effects analysis (FMEA) taken from SAE ARP 4761, a guideline for the safety assessment of civil aircraft systems, with an automatically generated FMEA, produced with paitron. By studying the voltage monitoring function provided by the device introduced in ARP 4761, the article highlights the lack of guidance for addressing common mode failures and the limited scope of the guideline, discussing the shortcomings of ARP 4761. The article also suggests a more proactive approach to safety assessment that involves considering potential hazards and risks throughout the entire system lifecycle, rather than just during development.
The purpose of this publication is to compare a manually generated expert failure modes and effects analysis (FMEA) with an automatically generated one, produced with paitron. The manual expert FMEA is taken from SAE ARP4761 , a document, which describes guidelines and methods for conducting a safety assessment for certification of a civil aircraft. The report provides among other analysis methods a piece-part FMEA performed on the power supply monitor of the power supply system of a braking system control unit.
In the present publication, the numerical model of the voltage monitor has been built based on the electrical drawings and simulated using LTspice®, a SPICE simulation software provided by the Linear Technology Corporation (Analog Devices) .
The preliminary study presented in this paper focuses on the voltage monitoring function provided by the device introduced in ARP4761 . The circuit-level implementation of this function is shown in Figure 1. The circuit is designed as a window comparator. The comparator ICs U1A and U1B compare the resistively divided tested voltage (Utest) against the reduced reference voltage (Uref) to detect over (U1B) and under (U1A) voltage conditions. The capacitors C1 and C2 implement first-order RC filters to effectively delay the circuit output in the presence of noise or ripple of the input signal. The comparators outputs are ANDed together so that if any voltage exceeds the trip point, high or low, the monitor output (Uvalid) is pulled low.
Within this project, an FMEA for the voltage monitor system was performed automatically with paitron. To evaluate paitron’s performance, these results are compared to a reference FMEA, which was taken from ARP 4761. The following table shows the system effects and their criticality which are tracked.
Table 1: Voltage monitor studied effects
|Monitor stuck valid||The voltage monitor always produces a high (valid) output signal, regardless of the actual input voltage||Dangerous|
|Monitor stuck tripped||The voltage monitor always produces a low (invalid) output signal||Dangerous|
|Window shift||The thresholds for the validation of the input voltage both decrease or increase||Dangerous|
|Trip window widens/tightens||The thresholds for the validation of the input voltage shift in the opposite direction, such that the monitoring function becomes less/more sensitive||Dangerous|
Failure modes and effects analysis
The FMEA was necessary to verify the design complies with the requirement of having less than 2E-7 failures per hour (200 Failures In Time (FIT)). The actual failure rate given by the standard is 1.42E-7 failures per hour, which matches the automatically generated 142 FIT.
With 12 components and considering the ARP 4761, the safety analysis of the voltage monitor system requires the study of 48 failure modes among which paitron:
- Analyzed 9 out of 12 components.
- Identified 36 possible failure modes of the system.
- Was able to propose a model of the system for 36 failure modes.
- Successfully evaluated the effects of 36 of the system’s failure modes, while 22 of them are leading to an effect. The rest is not leading to any system effect.
The analysis lasted less than 10 minutes and allowed to cover 75% of all failure modes to be considered for the entire analysis. paitron in its version 1.7 has no integrated circuits such as the comparator ICs in its failure mode database yet. Those components will be added in future paitron releases. With 75% automation, the manual work can still be drastically limited. The missing components, their failure modes, and the respective failure rates had to be set manually.
A view of the FMEA results generated by paitron is given below. A detailed overview of the achieved metrics of the automated analysis is provided in Table 3. For the automated analysis, information about the failure modes and their distributions were taken from the IEC 61709 , while the components’ FIT rates were selected from the MIL-HDBK-217F standard . The expert FMEA shown in ARP4761 relied only on the MIL-HDBK-217F standard. The differences in the final metrics are due to the different Failure Mode Distributions (e.g. Safe Failure Fraction), while the FIT rates were reproduced (e.g. Safety Function Failure Rate).
Table 2: Safety analysis of the voltage monitor generated using paitron
Table 3: Safety metrics of the voltage monitor evaluated using paitron
|Safe Failure Fraction (SFF)||5,95%|
|Safety Function Failure Rate [1/h]||142 FIT|
|Device Failure Rate [1/h]||142 FIT|
|Mean Time Between Failure (MTBF) [h]||803 years|
The results of the two analyses were compared and discrepancies were noticed. For the analysis, paitron checked an additional failure mode for the capacitors and one for the resistors. For the capacitors not only the Drift_0.5 (the capacitance drifts by -50%) was tested, but also the Drift_2 (the capacitance drifts by +100%) failure mode, while for the resistors the Short Circuit failure mode was additionally tested by paitron. The difference is documented in Table 4.
Table 4: Discrepancies between the failure modes of an expert FMEA and an automated FMEA
|Component||Reference FMEA (ARP4761)||paitron|
|Low cap. (Drift_0.5)||Low cap. (Drift_0.5)|
|–||High cap. (Drift_2)|
|Low cap. (Drift_0.5)||Low cap. (Drift_0.5)|
|High cap. (Drift_2)||High cap. (Drift_2)|
More interesting are the different results. For the increased and decreased resistance of R4, the manual and the automated analyses show different results. The discrepancies regarding the analysis results are highlighted in Table 5.
Table 5: Discrepancies between an expert and an automated FMEA
|Id||Component||Failure Mode||paitron FMEA||Reference FMEA (ARP4761)|
|1||R4||Decreased R||Monitor stuck tripped||Trip window widens|
|2||R4||Increased R||Trip window widens||Trip window tightens|
The effective comparison window is defined by the parameterization of the resistors R1 through R5: The voltage at node C (see Figure 1):
The voltage (UA) at the positive input of the U1A comparator (node A) is expressed as:
while voltage (UB) at the negative input of the U1B comparator (node B) is expressed as:
The overvoltage comparator U1B gives a valid (high) output as long as UB < UC
For a decrease of R4, the effective comparison window [Umin; Umax] becomes narrower as Umin and Umax tend to the same value
On the contrary, the larger R4 is, the wider the window [Umin; Umax] becomes since Umin tends to 0 while Umax tends to
Hence, in extreme cases, the decreased resistance (e.g. short circuit) of R4 even leads to a monitor stuck tripped. An increased resistance (e.g. open circuit) of R4 on the other hand would let the effective comparison widen instead of tightening.
The safety analysis in this article was performed to validate paitron by comparing the automated analysis results with those from the ARP4761. We noticed differences between the manual analysis, carried out by industry experts, and our automated approach. More precisely, the failure modes of one of the resistors were leading to different system effects. Those cases have been inspected by a safety expert from modelwise and were also verified by modeling those specific failure modes. The model as well as our safety expert confirmed the automatically generated FMEDA.
As safety analyses are cumbersome and error-prone, human errors are not rare. With paitron automating this task, such analyses become more accurate and trustworthy. Discovering an error in an expert FMEA raises the question: How many errors are there to be discovered?
Comments, suggestions? Brickbats, bouquets? Please send your feedback to our editor.
 ARP 4761, Guidelines and Methods for Conducting the Safety Assessment Process on Civil Airborne Systems and Equipment, SAE International, 1996
 MIL-HDBK-217F, Reliability Prediction of Electronic Equipment, Department of Defense – United States of America, 1991
 “LTspice,” LTspice Simulator | Analog Devices. [Online]. Available: https://www.analog.com/en/design-center/design-tools-and-calculators/ltspice-simulator.html [accessed 25.11.2021]
 IEC 61709, Electric Components -Reliability -Reference Conditions for Failure Rates and Stress Models for Conversion, International Electrotechnical Commission, 2017
Soft- and Hardware used in this project
- Desktop PC: Intel Core I5-4690K 4*3.5Ghz, 8GB RAM, Windows 10 Enterprise
- paitron v1.7 (v1.7.7982.40691)
- LTspice XVII (126.96.36.199)