Streaming Analytics
Streaming Analytics Applications Using Ancelustm
Ancelus has developed a new way to address system architecture for high performance or compute intensive applications. The combination of extreme performance and efficient structure makes true streaming analytics practical for the first time.
Traditional Application Development
The traditional method of writing applications for high performance or compute intensive applications has been dictated by the characteristics and limitations of the data store. When Relational Databases (RDB) or indexed flat files are used for data storage there are severe restrictions on the developer's flexibility. High data input rates are given first priority to prevent overloading the insert functions. This necessarily restricts analytical activity. The difference between data rates and insert capacity are sometimes on the order of two to four orders of magnitude. To accommodate this disparity an intermediate journal step is often added to the architecture. The following shows an example of this data flow model.
The journal step is designed to be "Input Normalized" to prevent overload of the central database. The journal-to-datastore transfer is a batch process often performed in off hours precluding real time analysis.
The Ancelus Method
Ancelus is a completely different structural model with the explicit objective of supporting streaming analytics as an application development model. This approach is shown in the following data flow model:
The streaming analytics configuration allows exception alarming during data collection. It also allows the exception messages to become an integral part of the data record. This reduces the human analyst burden (pilots, intelligence analysts, scientists, manufacturing managers) assuring that these resources are focused on mission tasks.
Exemplar applications are:
Real-time statistical analysis is now possible. Traditional statistical analysis assumes that all the data is available before analysis starts. This architecture was imposed by the limits of RDBs. Now, streaming analytics are performed one measurement at a time producing a result that can be further processed. Statistical analysis can now occur as the data arrives.
Facial recognition scanning and other biometrics involve complex associative analysis and have been limited by the performance of the reference database. Their range of application can now be expanded by several orders of magnitude of complexity or speed.
Passport / Visa Scans have been useful for checking the documents against a single reference database, but the inability to concurrently support scans of other databases has led to several high-profile "misses."
Watch Lists suffer from a lack of effective validation which can be overcome by redesigning the system with Ancelus as the data integration hub. Cross function analysis can now occur in real-time to quickly clear the false positives.
Number, Word and Phrase Scanning have the potential for expanded associative and combinatorial tests allowing broader context analysis, expanded scan volume and reduced frequency of both false positives and false negatives.
Tracking and Traceability in real time enhances the ability to know the current location and movement of an entity (people, vehicles, containers, parts). This allows simultaneous, multi threaded data streams (drivers license, credit card, satellite links, passport, cell phone).
Instant History Retrieval allows the development of applications that can reconstruct traceability threads after the fact. Based on a detected pattern, the past history can be instantly retrieved and converted to a tracking thread for future monitoring and alarming.
Sensor Fusion applications are enhanced. One of the DARPA Grand Challenge entrants used an Ancelus prototype to integrate the 100 Hz real-time outputs from over 40 sensors into a space and time coherent model of position, route and obstacles.
Ancelus is a trade mark of Prevel Technology. US and International patents pending.