Tuning Expert Systems for Cost-Sensitive Decisions

Algorithm 1

Expert system tuning algorithm.

Expert_System_Tuning (𝑓, 𝑆, 𝑟, 𝑝)

Input:

 𝑓: An expert system. Given the input vector of a problem case, 𝑓 suggests a CF value. By

 default, the expert system makes a positive classification decision if the CF is positive and a

 negative classification decision, otherwise.

 𝑆: A sample of 𝑛 solved cases 𝑆[𝑖] (𝑖=1,2,…,𝑛), each of which is a pair ⟨𝑥,𝑦⟩, where

 𝑥=⟨𝑥₁,𝑥₂,…,𝑥_𝑚⟩∈𝑋 and 𝑦∈𝑌. In addition, let 𝑆[𝑖]⋅𝐶𝐹 denote the CF value suggested

 by the expert system for the case 𝑆[𝑖].

 𝑟: Cost ratio. 𝐶𝑟=⁰¹𝐶₁₀.

 𝑝: Prior probability of the positive class.

Output: A tuned cutoff for the CF value.

BEGIN

 𝑛₁:= 𝑛₀:= 0.

 FOR 𝑖  := 1 TO 𝑛,

  Classify 𝑆[𝑖] using the expert system, that is, 𝑆[𝑖].𝐶𝐹  := 𝑓(𝑆[𝑖].𝑥).

  IF 𝑆[𝑖]⋅𝑦=1, THEN

     𝑛₁:= 𝑛₁ + 1.

  ELSE

     𝑛₀:= 𝑛₀ + 1.

 Sort the cases in 𝑆 on the CF value in ascending order.

 𝐶^∗:= ∝.

 FOR 𝑖  := 0 TO 𝑛,

  IF 𝑖=0, THEN

       𝜏₁:= 𝜏₂:= −1.

  ELSE IF   𝑖<𝑛, THEN

       𝜏₁:= 𝑆[𝑖]⋅𝐶𝐹.

       𝜏₂:= 𝑆[𝑖+1]⋅𝐶𝐹.

  ELSE

    𝜏₁:= 𝜏₂:= 1.

  IF 𝑖=0, THEN

       𝑛₁₀:= 𝑛₀; 𝑛₀₁:= 0;

  ELSE IF 𝑆[𝑖]⋅𝑦=1, THEN

       𝑛₀₁:= 𝑛₀₁ + 1.

  ELSE

    𝑛₁₀:= 𝑛₁₀– 1.

  Assuming the cutoff of (𝜏₁+𝜏₂)/2, compute the expected misclassification cost as

  𝐶=𝑟𝑝𝑛(1−𝑝)𝑛⁰¹₁+𝑛₁₀𝑛₀.

  IF 𝐶<𝐶^∗, THEN

    𝐶^∗:= 𝐶; 𝜏^∗₁:= 𝜏₁; 𝜏^∗₂:= 𝜏₂.

 RETURN (𝜏^∗₁+𝜏^∗₂)/2.

END.