As Cognitive Systems aim to handle real world problems, which are highly uncertain and may be influenced by potentially unlimited number of different factors, quality and consistency of their results highly depends on the number of factors they consider while making the decision. That brings yet another technological trade-off as the complexity of the problem grows tremendously with the number of the data sources. Aggregating and integrating the data from different data sources and processing it in a unified way is also challenging. Here is where Apache Spark project comes into play providing distributed and highly efficient tool that covers most of the present data processing and analytics routines. It also addresses another important requirement of having the most important data available for the real-time ad-hoc access while being able to reach a long track of historical data for better insight. A complex Cognitive system usually is a combination of multiple technologies that are glued together. As an example, Kafka, Spark, Elasticsearch, Cassandra, Hadoop, and other technologies can coexist in a single solution to handle in-memory ‘hot’ contextual data and a batch processing of massive ‘cold’ historical data. The science and art of designing such complex solutions now is a subject of cooperation between practitioners and academia. The SmartDecisions game , which simulates the software architecture design process in a format that makes the learning process more enjoyable for students, is a good example of such a cooperation that introduces gamification to solving comprehensive technological challenges in a fun way.
The principle of separation of mechanism and policy is the substantial difference between the philosophy of micro and monolithic kernels.   Here a mechanism is the support that allows the implementation of many different policies, while a policy is a particular "mode of operation". For instance, a mechanism may provide for user log-in attempts to call an authorization server to determine whether access should be granted; a policy may be for the authorization server to request a password and check it against an encrypted password stored in a database. Because the mechanism is generic, the policy could more easily be changed (. by requiring the use of a security token ) than if the mechanism and policy were integrated in the same module.