RASCAL Data Streaming Backend -- Which Technology to use?

Herb Wideman herb at yorku.ca
Tue Oct 7 15:17:29 UTC 2008

Hi David,

I'd like to echo Peter's call for the use of a secure communications 
protocol. From the research perspective, data security is critical as 
our research consent agreements with subjects as well as our ethics 
review submissions to the university and/or school boards typically 
include clauses stating that any data gathered will be kept confidential 
by the research team, with an exception only being made for external 
publication when this is done in such a way as to ensure subject anonymity.


Message: 3
Date: Mon, 6 Oct 2008 18:08:10 -0400
From: Peter Rowley <prowley at yorku.ca>
Subject: Re: RASCAL Data Streaming Backend -- Which Technology to use?
To: David Makalsky <dmakalsky at gmail.com>
Cc: Fluid Mailing List <fluid-work at fluidproject.org>
	<OFB2B98947.23F82F6E-ON852574DA.0076296B-852574DA.00799877 at yorku.ca>
Content-Type: text/plain; charset="us-ascii"

Hi David and everyone,

Following David's suggestion, I've converted an e-mail I sent to him into 
one for fluid-work, so everyone can get in on the discussion.

Based on some discussions within CNS at York, I'd suggest looking more 
closely at using HTTPS, i.e. HTTP over SSL.  HTTPS...

- due to the possibility that the screen recordings contain secure data 
and thus the need to transmit them securely to the server, 
- due to the fact that it encodes credentials used for authorization to 
deposit recordings on the server, and 
- due to its acceptability to sysadmins from a security perspective 
(stemming from the nature of the protocol and greater level of comfort 
with the security of Apache than a typical FTP server), 
- due to its ability to pass through most firewalls,
- due to it being built into Java SE, so licensing is not an issue

As for the scalability, HTTP upload is certainly used for massive sites 
such as Flickr.  Your approach to uploading the file in smaller (e.g. 1MB) 
chunks is probably a good one to continue; you'll need to build (or find) 
a simple protocol above HTTPS to keep track of which chunks still have to 
be transmitted.

As for alternatives, JDBC and JMS are overly complex for this, and SFTP 
has firewall issues in some situations (Google "sftp firewall"). 


David Makalsky <dmakalsky at gmail.com> 
Sent by: fluid-work-bounces at fluidproject.org
10/03/08 05:57 PM

Fluid Mailing List <fluid-work at fluidproject.org>

RASCAL Data Streaming Backend -- Which Technology to use?


We have a functional requirement in RASCAL for data to be streamed 
from the clients' browser (applet) to the server.  This is the data 
that will be converted into the video of the users' desktop (and audio 
from the microphone).  This functionality is essential, since without 
it, the data wouldn't get to the server.

Currently, this solution is implemented via ftp.  An ftp connection is 
established from the client to the server and the data is uploaded in 
a separate thread in 1 MB chunks.  We chose this solution for various 
reasons (discussion in the table below) but would like to open this up 
to the fluid community.  Which of the proposed solutions below would 
you suggest, or are there other completely different solutions that we 
should consider?


David Makalsky

> ****

More information about the fluid-work mailing list