​Where did ARP 4761 go wrong?

The TLDR: Too busy to read? Here's a quick summary of article

Comparison of manual & automated FMEA: The article compares a manually generated expert Failure Modes and Effects Analysis (FMEA) taken from SAE ARP 4761, a guideline for the safety assessment of civil aircraft systems, with an automatically generated FMEA, produced with paitron. 

Voltage monitor system as a case study: The article focuses on the voltage monitoring function of a power supply system of a braking system control unit, and shows how paitron can model and simulate the system and its failure modes.

Discrepancies between the two FMEAs: The article reveals that the manual FMEA from ARP 4761 contains errors and inconsistencies, while the automated FMEA from paitron is more accurate and trustworthy.

Limitations and suggestions of ARP 4761: The article discusses the lack of guidance for addressing common mode failures and the limited scope of the guideline, and suggests a more proactive approach to safety assessment that involves considering potential hazards and risks throughout the entire system lifecycle, rather than just during development.

Introduction

This article aims to compare a manually generated expert Failure Modes and Effects Analysis (FMEA) with an automatically generated one produced with paitron. The manual expert FMEA is taken from SAE ARP 4761 [1], a document that describes guidelines and methods for conducting a safety assessment for the certification of a civil aircraft. The report provides, among other analysis methods, a piece-part FMEA performed on the power supply monitor of the power supply system of a braking system control unit. 

In the present publication, the numerical model of the voltage monitor has been built based on the electrical drawings and simulated using LTspice®, a SPICE simulation software provided by the Linear Technology Corporation (Analog Devices) [3].

Analysis setup

The preliminary study presented in this paper focuses on the voltage monitoring function provided by the device introduced in ARP 4761 [1]. The circuit-level implementation of this function is shown in Figure 1. The circuit is designed as a window comparator. The comparator ICs U1A and U1B compare the resistively divided tested voltage (Utest) against the reduced reference voltage (Uref) to detect over (U1B) and under (U1A) voltage conditions. The capacitors C1 and C2 implement first-order RC filters to effectively delay the circuit output in the presence of noise or ripple of the input signal. The comparators outputs are ANDed together so that if any voltage exceeds the trip point, high or low, the monitor output (Uvalid) is pulled low.

Within this project, an FMEA for the voltage monitor system was performed automatically with paitron. To evaluate paitron’s performance, these results are compared to a reference FMEA, which was taken from ARP 4761. The following table shows the system effects and their criticality, which are tracked.

Table 1: Voltage monitor studied effects  

Failure Modes and Effects Analysis (FMEA)

The FMEA was necessary to verify the design complies with the requirement of having less than 2E-7 failures per hour (200 Failures In Time (FIT)). The actual failure rate given by the standard is 1.42E-7 failures per hour, which matches the automatically generated 142 FIT.

With 12 components and considering the ARP 4761, the safety analysis of the voltage monitor system requires the study of 48 failure modes, among which paitron:  

• Analyzed 9 out of 12 components.  
• Identified 36 possible failure modes of the system. 
• Was able to propose a model of the system for 36 failure modes. 
• Successfully evaluated the effects of 36 of the system’s failure modes, while 22 of them are leading to an effect. The rest is not leading to any system effect. 

The analysis lasted less than 10 minutes and allowed to cover 75% of all failure modes to be considered for the entire analysis. paitron in its version 1.7 has no integrated circuits such as the comparator ICs in its failure mode database yet. Those components will be added in future paitron releases. With 75% automation, the manual work can still be drastically limited. The missing components, their failure modes, and the respective failure rates had to be set manually.

A view of the FMEA results generated by paitron is given below. A detailed overview of the achieved metrics of the automated analysis is provided in Table 3. For the automated analysis, information about the failure modes and their distributions were taken from the IEC 61709 [4], while the components’ FIT rates were selected from the MIL-HDBK-217F standard [2]. The expert FMEA shown in ARP 4761 relied only on the MIL-HDBK-217F standard. The differences in the final metrics are due to the different Failure Mode Distributions (e.g., Safe Failure Fraction), while the FIT rates were reproduced (e.g., Safety Function Failure Rate).

Table 2: Safety analysis of the voltage monitor generated using paitron

Table 3: Safety metrics of the voltage monitor evaluated using paitron 

Comparison 

The results of the two analyses were compared, and discrepancies were noticed. For the analysis, paitron checked an additional failure mode for the capacitors and one for the resistors. For the capacitors, not only the Drift_0.5 (the capacitance drifts by -50%) was tested, but also the Drift_2 (the capacitance drifts by +100%) failure mode, while for the resistors, the Short Circuit failure mode was additionally tested by paitron. The difference is documented in Table 4.  

Table 4: Discrepancies between the failure modes of an expert FMEA and an automated FMEA 

More interesting are the different results. For the increased and decreased resistance of R4, the manual and the automated analyses show different results. The discrepancies regarding the analysis results are highlighted in Table 5. 

Table 5: Discrepancies between an expert and an automated FMEA 

The effective comparison window is defined by the parameterization of the resistors R1 through R5: The voltage at node C (see Figure 1):

The voltage (UA) at the positive input of the U1A comparator (node A) is expressed as:

while voltage (UB) at the negative input of the U1B comparator (node B) is expressed as:

The overvoltage comparator U1B gives a valid (high) output as long as U_B < U_C holds and the under-voltage comparator U1A gives a valid (high) output as long as U_A > U_C holds. Accordingly, the tested voltage monitor provides a high output signal when the tested voltage Utest belongs to a voltage window [Umin; Umax] defined so that:

For a decrease of R4, the effective comparison window [Umin; Umax] becomes narrower as Umin and Umax tend to the same value

On the contrary, the larger R4 is, the wider the window [Umin; Umax] becomes since Umin tends to 0 while Umax tends to

Hence, in extreme cases, the decreased resistance (e.g., short circuit) of R4 even leads to a monitor stuck tripped. On the other hand, an increased resistance (e.g., open circuit) of R4 would let the effective comparison widen instead of tightening.

Conclusion

The safety analysis in this article was performed to validate paitron by comparing the automated analysis results with those from the ARP 4761. We noticed differences between the manual analysis carried out by industry experts and our automated approach. More precisely, the failure modes of one of the resistors were leading to different system effects. Those cases have been inspected by a safety expert from modelwise and were also verified by modeling those specific failure modes. The model, as well as our safety expert, confirmed the automatically generated FMEDA.

As safety analyses are cumbersome and error-prone, human errors are not rare. With paitron automating this task, such analyses become more accurate and trustworthy. Discovering an error in an expert FMEA raises the question: How many errors are there to be discovered?

Comments, suggestions? Brickbats, bouquets? Please send your feedback to our editor.

Additional Resources

References 

[1] ARP 4761, Guidelines and Methods for Conducting the Safety Assessment Process on Civil Airborne Systems and Equipment, SAE International, 1996

[2] MIL-HDBK-217F, Reliability Prediction of Electronic Equipment, Department of Defense – United States of America, 1991

[3] “LTspice,” LTspice Simulator | Analog Devices. [Online]. Available: https://www.analog.com/en/design-center/design-tools-and-calculators/ltspice-simulator.html [accessed 25.11.2021]

[4] IEC 61709, Electric Components -Reliability -Reference Conditions for Failure Rates and Stress Models for Conversion, International Electrotechnical Commission, 2017

Project Toolkit

1. Desktop PC: Intel Core I5-4690K 4*3.5Ghz, 8GB RAM, Windows 10 Enterprise 
2. paitron v1.7 (v1.7.7982.40691) 
3. LTspice XVII (17.0.32.0)