Abstract
It is evident that a lot of accidents occur because of drowsiness or inattentiveness of the driver. The logical consequence is that we have to find methods to better analyze the driver. A lot of research has been spent on camera-based systems which focus on the driver's eye gaze or his head movement. But there are few systems that provide camera-free driver analyzing. This is the main goal of the work presented here which is structured in three phases, with the operational goal of having a working driver analyzer implemented in a car. The main question is: is it possible to make statements concerning the driver and his state by using vehicle data from the CAN Bus only? This paper describes the current state of driver analyzing, our overall system architecture, as well as future work. At the moment, we focus on detecting the driving style of a person.
1. Introduction
Driver analysis (DA) has been an active field of research for years. For example, [1] published an article about driver monitoring already in 2005.
Among others, DA can be divided in the following subtopics: driver monitoring, driving style analysis, and merging vehicle data to derive conclusions concerning the driver (The word driver means both, female as well as male drivers. This is also relevant for words like “his” or “him” which reflect also both, female as well as male persons.) and his environment. For our research work, we focus on the following aspects. (i)How can the state of the driver be detected without using a camera or realtime biosensor data like a electrocardiogram (ecd)? (ii)How can we support the driver, depending on his actual driving situation, based on the results of the driver state detection?
Driver monitoring is usually performed by cameras installed in the car for detecting the driver's behavior or state, mostly by using infrared cameras ([2, 3], or [1]).
There are also first results for noncamera based research on driver analysis: By analyzing analog speed graphs, Rygula [4] makes conclusions about the driving style, speed profile and, depending on driving time and course, aggressiveness of the driver. Therefore, he evaluated ten analog speed graphs for two drivers by comparing their speed profile, their profile referring to the distance, or referring to route and direction. Rygula states that “Even a brake of 45 minutes reduce aggressivity of driving style” ([4, page 79]).
A different approach is the research on context recognition in vehicles and the development of a driver model. Ferscha and Riener [5] describe this process of in-car context recognition and building a driver comodel.
2. Basic Architectural Considerations
There is no real “driver analyzing concept car” yet which offers a real-time system that works without camera or biosensors. We plan to build up a driver analyzing system during the next two years which is able to draw conclusions on the driver's condition. We have divided this task into subtopics of increasing complexity: (i)driver preferences, (ii)driving style analysis,(iii) driver state.
This structure helps us to get a better understanding of the environment and the tasks of a driver. First, we have to learn the favourite settings of the driver. Without any knowledge of the driver, it would not be possible to make a statement about his state. With the driving style analyses, we can again improve our knowledge of the driver. Knowing the driving style, for example, sporty, is also a needed input value for the next module, the driver state. The driver state is the main topic in our research on driver analyzing. While the first topics were needed to learn the driver preferences and to learn the driving style, we want to learn within this section the driver's condition, focusing on drowsiness, attentiveness, and stress. This means that we use the information gathered from the first two modules and merge it with additional input signals, for example, time between using the infotainment system, time to lane crossing, or environmental information like weather or outdoor temperature. Each module can be seen as a precondition for the next module.
Because of the fact that we use a fully equipped concept car, we do not need any external equipment (e.g. a GPS mouse for getting speed and position) and use only data available on the CAN bus-mainly on the comfort CAN bus. This decision is also advantageous for future projects, because the CAN bus is a standardized system and every vehicle platform has almost the same information on the CAN buses.
For the driver preferences, we focus on climate control settings. The climate data on the CAN bus include clear and structured data, for example, temperature, fan pace, or air distribution. We concentrate on climate settings, because changing the temperature and the air distribution can be seen as an indicator for, example given, fatigue [6]. Next, we plan to analyze the driving style. This can be done by analyzing data like, for example, speed, engine load, or brake pressure. Possible driving styles could be driving in a sporty way or environmentally friendly (see [7]). Analyzing the speed profile is necessary due to the fact that, for example, tired drivers tend to drive from very sporty through to aggressive.
Combined with the preferences from part one, we can say that driver X sets the temperature in the vehicle to 23 degrees and drives, for example, in a sportive fashion.
3. System Architecture
The architecture of our system is divided in several parts. At the beginning, we have to identify the CAN messages needed and establish a stable connection between our system and the CAN bus system of the car. Therefore, not every message is useful. What we need are data which depend somehow on the driver's interaction with the car. For example, the engine temperature is not a good indicator for drowsiness. Examples for appropriate parameters are speed, rounds per minute, acceleration and yaw rate, pedal movement, steering wheel angle, travel time, or using the multimedia or navigation system, because they reflect the driver's behavior or allow inferences on the driver's state. For a better understanding, Figure 1 shows the graph of recorded, useful speed data. Afterwards, we have to perform data fusion. This means that we must merge and evaluate the information included in the different messages. Depending on the respective module, the analysis method can change. There are several aspects which influence the selection process for the right method. Choosing the right method for each module will surely be one of the most important parts. On the one hand, we have to define learning algorithms to get an intelligent system; on the other hand, we must define rules for certain driving situations. We plan to measure the state on long-term trips with a driving duration of at least 45 minutes. Moreover, we restrict the system to only measure on highways or state roads.
Although we restrict the system to long-term trips and street type, there are some situations where the system might not be able to work correct. One factor is of course the driver himself. If he ignores the proposals, we cannot force the driver to adherence the instruction of the system. Also, environmental factors like snow, clear ice, as well as traffic jams or accidents might lead to a wrong state detection. Last but not least, unforeseen cases of emergency (like imminent childbirth or someone being injured) might lead to a failure, too.
With the result of the data fusion, we, and accordingly the system, can make a description of the current situation and provide it to a personal information agent as an input value. Based on this value, the information agent is able to start different applications or services, depending on the driving situation. This process is illustrated in Figure 2.
4. Current State of the System
Due to the fact that we use CAN data, we log the messages on the bus during test drives. After having identified all the needed messages on the CAN bus, we make first measurements on a test track from the Continental Automotive GmbH. The main reason for using this test track is that we have consistent conditions on it. This is an ongoing process, because we need more and new data over the time. Instead of implementing every new version directly in the car, we use MatLab as our basic development tool. As soon as the algorithms work correctly, we implement them in the vehicle framework and test them outside the test track in real traffic scenarios.
At the moment, we are working on the driver preferences and the driving style analyses. To verify the output of the algorithms, we have implemented a tool which allows the driver to register his driving style explicitly. Therefore, we installed a small touch-screen over the dashboard, on which the driver can enter his driving mode by clicking the respective button. He can choose between ECOLOGICAL, SPORTIVE, or COMFORT. Every click is registered and timestamped in a log file. This allows us to compare the output of the algorithms with the real driving situation very exactly.
Based on the android system, we also implemented an application for a mobile device. This application (app) is a data logger for the standardized on-board diagnostics 2 (OBD2). The app works in combination with an external connector including a bluetooth chip. This connector is plugged into the OBD2 interface of the car and after pairing the mobile device with the connector and pushing the start button, speed, rounds per minute, engine load, and throttle position are logged in a data base until the stop button is pushed. During logging, the driver has the possibility to rate his driving style as follows: ECOLOGICAL, SPORTIVE, or COMFORT (see Figure 3). Another possibility would be that the driver first sets his driving mode (e.g. sporty) and afterwards tries to drive like this for at least 3 or 4 minutes. But measurements showed that this is (a) not as exactly as the version described above and that (b) it is more dangerous, because the risk for the driver on the highway rises by forcing a sporty driving style. Due to the fact that these OBD2 data are in some kind similar to the CAN data, we can collect data sets from different persons in different cars without having access to the CAN bus.
5. Future Work
In the near future, we plan to set up a first version of module one and two in our test vehicle. Based on these experiences, we want to improve our procedures for logging, analyzing, and implementing. Moreover, we need the results of the first implementations for choosing the right analyzing methods. In the long run, we have to develop new methods for combining the modules and new algorithms for interpreting the vehicle data correctly. Driver state modeling will also be a major challenge.