In this session we like to share our experiences from analyzing streams of Twitter data with Apache Spark Streaming in near real-time, leveraging Apache Kafka as a HA messaging backbone plus storing and searching for Tweets in Elasticsearch at a large scale. Key design aspects are short end-end processing delays, sub-second search responses and a highly available system that does not rely on hardware redundancy.