R-Storm: A Resource-Aware Scheduler for Storm , CS Department, UIUC
- The era of big data has led to the emergence of new systems for real-time distributed stream processing, e.g., Apache Storm is one of the most popular stream processing systems in industry today. However, Storm, like many other stream processing systems, lacks an intelligent scheduling mechanism. The default round-robin scheduling currently deployed in Storm, disregards resource demands and availability, and can therefore be inefficient at times. We present R-Storm (Resource Aware Storm), a system that implements resource aware scheduling within Storm. R-Storm is designed to maximize resource utilization while minimizing network latency. When scheduling tasks, R-Storm can satisfy both soft and hard resource constraints as well as minimizing network distance between components that communicate with each other. We evaluate R-Storm on set of micro-benchmark Storm applications as well as Storm applications used in production at Yahoo! Inc. From our experimental results we conclude that R-Storm achieves 30-47% higher throughput and 69-350% better CPU utilization than default Storm for the micro-benchmarks. For the Yahoo! Storm applications, R-Storm outperforms default Storm by around 50% based on overall throughput.
Stela: Enabling Stream Processing Systems to Scale-in and Scale-out On-demand , CS Department, UIUC
Official Website: http://dprg.cs.uiuc.edu/
This research supported in part by NSF CNS 1409416, NSF CNS 1319527, NSF CCF 0964471, and AFOSR/AFRL FA8750-11-2-0084
- The era of big data has led to the emergence of new real-time distributed stream processing engines like Apache Storm. We present Stela (STream processing ELAsticity), a system that supports scale-in and scale-out operations in an on-demand manner, i.e., when the user requests such a scaling operation. Stela meets two goals: 1) it optimizes post-scaling throughput, and 2) it minimizes interruption to the ongoing computation while the scaling operation is being carried out. We integrated Stela into Apache Storm. We present experimental results using both micro-benchmark Storm applications as well as Storm applications from Yahoo! Inc. Our experiments show that compared to Apache Storm's default scheduler, Stela's scale-out operation achieves throughput that is 45-120% higher, and interruption time that is smaller. Stela's scale-in operation chooses the right set of servers to remove and achieves 40-500% higher throughput than Storm's default strategy.
iTrust, ECE Department, UCSB
Official Website: www.itrust.ece.ucsb.edu
This research supported in part by NSF CNS 10-16193
- iTrust is a novel information distribution and retrieval system that operates over HTTP and the Internet that provides trustworthy access to information. iTrust is a completely distributed system, with no centralized mechanisms and no centralized control, that avoids subversion or censorship of information. Individuals submit information they wish to share to nodes on the Internet that distribute metadata to random participating nodes. Likewise, users submit requests containing metadata for information they wish to retrieve to random participating nodes. iTrust can effectively enable citizens to distribute and retrieve information over the Internet, even in the presence of subverted or non-operational nodes.
Kinects in Unity, Jet Propulsion Lab, NASA
Official Website: http://seapong.com/capstone/
Web-Based Graphing and Data Analytic Untilities, Gauchospace, UCSB
Web-based score viewer
- This utility was developed for Gauchospace, UCSB's main course management website. The utility allows the both the student user to graphically view his grade for a certain assignment or test graphically compared to other students in the class. The utility also displays various statistics such the average, high, low, and median score of an assignment or test. The utility displays the rank of the students score on this assignment or test. The utility also allows the user to adjust the bin sizes to view the data in a larger and smaller grade range.
- Click Here for demo
Web-based Graphical Activity Viewer
- This utility allows a course administrator to view the number of access to the course webpage or an item on the course webapge as a function of the date. This allows the course administrator to see how many students have viewed the course webpage or an item on the course webpage. For example, a professor can use this utility to determine if his or her students actually read a document that was assigned to them. This utility can also allow the course administrator to view at what time in the day the are the access to an item occuring. This function can let professors know at what times students are doing assignments. There are also lots more uses for this utility!
- Click Here for demo