Surveillance and anti-tampering are two key capabilities of many real world systems. Surveillance mechanisms are typically implemented using visual and/or infrared cameras. They are commonly utilized to monitor an asset or area under protection and to detect threats or hazardous situations through either operator visual inspection or image processing algorithms that typically require high performance processing nodes. Anti-tampering mechanisms, on the other hand, aim primarily to protect the embedded system itself by detecting any attempt for compromising system integrity and, sometimes, by applying countermeasures such as the emergency erasure of flash memories containing important data (cryptographic keys for example). Anti-tampering mechanisms are based both on traditional technologies such as secure packaging and use of seals as well as on sensors detecting the unjustified alteration of a periodically measured physical attribute, such as the electrical resistance of a protective enclosure.
In the context of nSHIELD, ISD aims to explore a novel acoustic based technology that can be used for monitoring and protecting both assets as well as the embedded system itself. Acoustic based systems have been in use for decades in underwater environments in the form of passive and active sonars. Recently, a lot of effort has targeted the exploitation of the potential of acoustic based systems as a technology that can complement camera based inspection. The main advantages of acoustic based technology, compared to camera based inspection, are the following:
Sound processing is computationally much less demanding than image processing.
There are no “out-of-view” areas due to obstacles or due to overcrowding and the data acquired are not affected by variable illumination conditions.
The reduction in cost and in processing power enables building systems with a really large number of acoustic sensors.
On the other hand acoustic processing faces many challenges mainly due to the interference of background sounds. That’s why most commercial applications target abnormal events that can be easily extracted from the ambient noise of the environment, such as gunshots . In this type of applications a geographically distributed network of sensors is used to constantly process sound data, perform a detection of a gunshot acoustic signature, and use the known relation is sensors’ placement to compute the rough area in which the event occurred .
Most research efforts in the area target the efficient detection and classification of the abnormal events of interest [3, 4]. However not much attention has been placed on key system features that can simplify processing tasks and hopefully boosts the widespread usage of acoustic based detection. Typically, a single sensor is used, or a network of sensors that are geographically distributed and interact in a loosely coupled fashion.
In the context of nSHIELD, ISD will develop a novel audio based surveillance system that aims to overcome the most important limitations of non-military grade systems currently utilized in acoustic based research providing correlated data acquisition from a large number of overlapping sensors.
More specifically, the system will be able to interface hundreds of hardware synchronized microphones and transfer the combined audio stream to memory in real time. It will be the only acoustic based system with sensors hardware synchronized, meaning there is neither time difference nor time drift among samples captured by different sensors. This feature dramatically simplifies data correlation for two reasons. First, by correlating the synchronized samples from multiple sensors placed at known locations and by taking advantage of the system’s inherent redundancy to make sure that sound sources are captured by multiple sensors, applications will be able to simplify ambient noise extraction, to perform detection of irregular acoustic events using simple peak detection and to identify the direction of any threat using triangulation. When sensors are not hardware synchronized, their sampling rates are controlled by individual crystal clocks, and they drift. This makes it difficult to accurately correlate sensor data. Moreover, at high speed motion applications (railway case) even a few milliseconds of difference in the time domain result in significant difference in the space domain, making difficult to estimate the relative positions of the microphones at their points of capture.
The data acquisition will be performed by an FPGA-based board that will collect the data samples and will deliver them to a processing unit over a standard PCI family bus. Interfacing the final product to the end application will be performed by software running on an embedded PC. For demonstration purposes, in the context of the project, a standard PC may also be used to interface the prototype board.
C. Clavel, T. Ehrette, and G. Richard, “Event detection for an audio-based surveillance system,” in IEEE International Conference on Multimedia and Expo, Amsterdam, July 2005.
S. Ntalampiras, I. Potamitis and N. Fakotakis, "On acoustic surveillance of hazardous situations", IEEE International Conference on Acoustics, Speech and Signal Processing, 2009. ICASSP 2009.
Jean-Luc Rouas, Jérôme Louradour and Sébastien Ambellouis, "Audio Events Detection in Public Transport Vehicle", in "9th International IEEE Conference on Intelligent Transportation Systems (ITSC'2006).
4.2System of Embedded System - SoES
Nowadays new interesting trends are characterizing the Embedded System field, in particular, they are even more physical and logically interconnected. These trends, of hyper connectivity, are driven by multiple needs, first is the gain of amount and the complexity of the services provided by the system, and second the growing interest around the distributed embedded. In such scenario we identify a new system typology commonly named System of Embedded System or simply Large System.
The SoES are usually large and composed by nodes that are heterogenic and independent. Such heterogeneity is mostly related to the nature of service provided by each node. The independency of nodes, instead, is strictly related to their autonomous evolution. Therefore, according to these considerations, the problem arises is about the design and the development of a large-scale system with specific SPD constraint and value. These two activities are extremely complex and the solution is generally very expansive. To simplify the design process, the idea is to use the concept of reusability and composability. We assume that is possible to put together components that are SPD compliant and obtain a System that is SPD compliant de facto. In this context, the main objective is to develop a new subcomponent that can be easily integrate into the every single node of the system without have to restructure architecture and ensuring the overall SPD properties of the SoES. This methodology is strongly related to pSHIELD project, where the composability criteria of the architecture design were defined. Therefore, following the identified mechanisms in pSHIELD, it will be developed a new methodology to be valid during the system design and system re-design.
Figure - Architecture Layers
A possible solution to create such gain factor is based on the development of a sub-component able to provide high functionality level of integration and that will be characterized by a set of properties that can confer Security, Privacy and Dependability to the entire architecture in which it will be integrated. This sub-component will be developed as a custom IP to simplify the integration and the adaptation of independent nodes at the lower level of the architecture.
Figure - Custom IP core
The custom IP will be providing functionalities and capabilities as follow:
Real time signal processing: to meet the stringent time constraint of which critical systems are characterized;
Abnormal event detection: to identify anomalous operations that can cause failures and/or can make the entire system insecure;
Automated system recovery: to resume correct system functionalities after a fault;
Secure service discovery: to provide the secure sub-components integration in the SPD node with a module identification;
Services balancing: to ensure the system reliability with a dynamic load distribution on more components;
Resources allocation: giving to the entire system the benefits of using more resources in the critical operations where a more computational power is required;
Low power consumption: to reduce the contribution of the power consumption;
Diagnostics: to provide a diagnostic service to the entire system with a status information exchanging;
Reconfiguration: providing different functionalities through a dynamic reconfiguration of the custom IP to manage different operating modes.
The integration of this custom IP in the pre-existing embedded system will be facilitated providing discovery and composability features. In addition, the Dependability of the network, which the IP is part of, will be improved by detecting abnormal events and recovering the system functionalities by a FPGA reconfiguration; the redundancy of hardware components will also assure the Dependability of the entire node architecture. Regarding the Security, the custom IP will receive encrypted data by others components and it will be able to decrypt it; another security aspect will be guaranteed by the diagnostic and self-reconfiguration property. The custom IP will be able to manages a dynamic resources allocation by a service balancing to assure a real time signal processing in every working condition, this is one of the most important requirements in some scenarios (i.e. the Avionic System), due to critical time constraints that have to be met.
At this architectural level, the heterogeneous nature of nodes makes the developing of a software support necessary for services exchanging with upper layer components. For this reason, in conjunction with the custom IP core, a software library will be developed to distribute SPD aspects as functionalities to microprocessor layer.