Richard Searle's Blog

Thoughts about software

Archive for April, 2012

Legal consequences of map-reduce,shards and distributed hash table systems

Posted by eggsearle on April 14, 2012

The legal consequences of holding data can be significant: privacy regulations, financial disclosure, Patriot Act, etc.
These difficulties increase geometrically for a multinational system, particularly one that covers both the US and the EU.

Conventional system architecture, as exemplified by JEE and RDBMs, drive a centralized model where data is moved to the code. That directly leads to data for entities in one legal jurisdiction being stored in another jurisdiction.

Distributed Hash table systems, such as Riak, spread the system data across many servers. It is certainly feasible to arrange that data is placed on servers that fall in the appropriate jurisdiction. Such an approach is also possible (and arguably easier) with sharded data stores, including RDBMs.

Riak (and many other DHT systems) implement map/reduce for queries and data reduction across the servers. Map/reduce moves the code to the data. This should provide some level of legal protection since the sensitive data need never leave the server.

Of course IANAL and regulations are never quite as sensible and logical as one might hope.

Posted in Uncategorized | Leave a Comment »

Angular, SSE and modifying a list model

Posted by eggsearle on April 8, 2012

The previous example used a very simple model, consisting of a single String.

A more realistic implementation would add the strings to a list.


<body ng-app>
 <div ng:controller="Main">
 <ul>
 <li ng:repeat="item in items">
 {{item}}
 </li>
 </ul>
 </div>

<script type='text/javascript'>//<![CDATA[

var source = new EventSource('/events');

function Main($scope) {
 $scope.items = [];

source.addEventListener('right', function(e) {
 $scope.$apply(function() {
 $scope.items.push(e.data);
 });
 },false);
 }
 //]]>
 </script>
 </body>

Posted in angular | Leave a Comment »

Angular.js and Server Sent Events wiring

Posted by eggsearle on April 8, 2012

SSE provides a standardized HTML5 way to push data over an HTTP connection.
angular is an interesting Javascript MVC framework that uses binding to provide a very clean implementation.
The general Angular documentation covers interactions that are initiated from the browser. Implementing an external push into Angular was not clearly described until the 1.0 release.

The following example uses a snapshot from upcoming angular 1.0 release.

Strings received via /events, tagged with right, are displayed

</pre>
<!DOCTYPE html>
<html>
<head>
 <meta http-equiv="content-type" content="text/html; charset=UTF-8">
 <title>AngularJS and SSE</title>
 <script type='text/javascript' src="http://ci.angularjs.org/job/angular.js-angular-master/ws/build/angular.js"></script>
</head>
<body ng-app>
 <div ng:controller="Main">
Value {{data}}
</div>

<script type='text/javascript'>//<![CDATA[

var source = new EventSource('/events');

function Main($scope, $window) {
 source.addEventListener('right', function(e) {
 $scope.$apply(function() {
 $scope.data = e.data;
 });
 },false);
}
//]]>
</script>
</body>
</html>
<pre>

Posted in angular | 1 Comment »

The future and promise of Scala is it’s killer app

Posted by eggsearle on April 4, 2012

Essentially every framework and library for Scala is based on or provides extensive support for asynchronous processing.
The title is thus something of a pun, referencing the future and promise functionality that form the the basis of such processing.

NIO was introduced back in Java 1.4, but has remained a niche technology. Even the introduction of netty and grizzly did not bring it into the mainstream.

This progression essentially started with Tuxedo and CICS, very complex and expensive products. JEE containers, ESBs and other variants on the theme reduced the cost and complexity to some extent. It is perhaps hard to remember that Spring first came to life as a way of dealing with the complexity of J2EE. Scala has now reduced this complexity to a library, largely seamlessly integrated into the language.

What was previously a complex and specialized middleware project is now something that “mere” application developers incorporate as a matter of course.
The provision of flags in spec2 and sbt to turn off concurrent processing is an indication of the ubiquity of concurrency in the Scala ecosystem.

These facilities are arriving just in the nick of time to allow effective utilization of multicore processors.

Posted in Scala | Leave a Comment »