RASCAL Data Streaming Backend -- Which Technology to use?

Peter Rowley prowley at yorku.ca
Tue Oct 7 03:16:16 UTC 2008


Hi Colin,

Thanks for your note.  Yes, our folks think it's a 10-20% penalty to use 
HTTPS, but processor horsepower is relatively cheap these days and if the 
interactions recorded contain any personal data, we have to use it to meet 
York's security standards.  Moreover, for the sake of not having to make 
that determination each time, for an install at York, we'd likely want to 
mandate use of SSL by default.  That would also help with research ethics 
issues, which sometimes involve the security of data (e.g. recordings) 
gathered about subjects.  If the upload must be done over HTTP, it would 
be helpful to have at least some security-through-obscurity in the 
higher-level protocol for the credentials used to get write access to the 
server (e.g. don't use basic authentication).

I like your point about making sure the higher-level protocol built on top 
of HTTP(S) is stateless and how that avoids having to worry about session 
replication.  Worrying about the possibility of retransmitting chunks 
makes for a *small* amount of state associated with the transfer (though 
not the session) and that could be handled by having the client, after 
sending what it thinks is the last chunk, GET a list of all missing 
chunks, which would then drive a "cleanup" phase of the transfer.

So, David, I guess there's a building consensus on HTTP.  I swear that I 
didn't consult with Colin on this!  If you'd like to get more technical 
feedback from the community, I think the next step would be to describe a 
proposed upload protocol.

Peter




Colin Clark <colin.clark at utoronto.ca> 
10/06/08 07:41 PM

To
Peter Rowley <prowley at yorku.ca>
cc
David Makalsky <dmakalsky at gmail.com>, Fluid Mailing List 
<fluid-work at fluidproject.org>
Subject
Re: RASCAL Data Streaming Backend -- Which Technology to use?






Hey Peter,

I missed your response before I sent along mine, but your point about 
HTTPS is a good one. It's certainly a fair bit slower and presents 
greater load on the server to use HTTPS, but the nice thing with an 
HTTP streaming approach is that the VULab user can choose whether or 
not they need the added security. The other alternatives involve an 
"all or nothing" approach, whereas it's simply a server-side 
configuration to choose to use HTTPS.

Colin


On 6-Oct-08, at 6:08 PM, Peter Rowley wrote:

>
> Hi David and everyone,
>
> Following David's suggestion, I've converted an e-mail I sent to him 
> into one for fluid-work, so everyone can get in on the discussion.
>
> Based on some discussions within CNS at York, I'd suggest looking 
> more closely at using HTTPS, i.e. HTTP over SSL.  HTTPS...
>
> - due to the possibility that the screen recordings contain secure 
> data and thus the need to transmit them securely to the server,
> - due to the fact that it encodes credentials used for authorization 
> to deposit recordings on the server, and
> - due to its acceptability to sysadmins from a security perspective 
> (stemming from the nature of the protocol and greater level of 
> comfort with the security of Apache than a typical FTP server),
> - due to its ability to pass through most firewalls,
> - due to it being built into Java SE, so licensing is not an issue
>
> As for the scalability, HTTP upload is certainly used for massive 
> sites such as Flickr.  Your approach to uploading the file in 
> smaller (e.g. 1MB) chunks is probably a good one to continue; you'll 
> need to build (or find) a simple protocol above HTTPS to keep track 
> of which chunks still have to be transmitted.
>
> As for alternatives, JDBC and JMS are overly complex for this, and 
> SFTP has firewall issues in some situations (Google "sftp firewall").
>
> Peter
>
>
>
> David Makalsky <dmakalsky at gmail.com>
> Sent by: fluid-work-bounces at fluidproject.org
> 10/03/08 05:57 PM
>
> To
> Fluid Mailing List <fluid-work at fluidproject.org>
> cc
> Subject
> RASCAL Data Streaming Backend -- Which Technology to use?
>
>
>
>
>
> Hi,
>
> We have a functional requirement in RASCAL for data to be streamed
> from the clients' browser (applet) to the server.  This is the data
> that will be converted into the video of the users' desktop (and audio
> from the microphone).  This functionality is essential, since without
> it, the data wouldn't get to the server.
>
> Currently, this solution is implemented via ftp.  An ftp connection is
> established from the client to the server and the data is uploaded in
> a separate thread in 1 MB chunks.  We chose this solution for various
> reasons (discussion in the table below) but would like to open this up
> to the fluid community.  Which of the proposed solutions below would
> you suggest, or are there other completely different solutions that we
> should consider?
>
> Regards,
>
> David Makalsky
>
> FTP
> ===
>
> Pros:
> - Already implemented
> - Many java libraries available (license compatible)
> - Only virtual user needs to be created with only write access granted
> - Simple to implement
>
> Cons:
> - Password is sent as plaintext
> - There is a maximum simultaneous connection limit
> - Not transactional
>
> SFTP
> ====
>
> Pros:
> - Password is encrypted
> - Simple to implement (if you have a good library)
>
> Cons:
> - No License Compatible java libraries
> - Same cons as FTP except password is encrypted
>
> Message Queue
> =============
>
> Pros:
> - Transactional
> - Secure
> - Easy to implement on the client side
> - Scalable
>
> Cons:
> - Significant administration and installation effort
> - Resource-intensive
>
> JDBC
> ====
>
> Pros:
> - Scalable
> - Secure
> - Easy to implement on both sides
> - Transactional
>
> Cons:
> - Database will grow quite big unless blobs are written out to files
> regularly and cleaned from database
>
>
> HTTP
> ====
>
> Pros:
> - transactional
> - fairly easy to implement
>
> Cons:
> - Bad for streaming large chunks of data
> - Puts a large strain on http server
> - not scalable easily (will have to cluster web servers with session
> affinity)
> _______________________________________________
> fluid-work mailing list
> fluid-work at fluidproject.org
> http://fluidproject.org/mailman/listinfo/fluid-work
>
> _______________________________________________
> fluid-work mailing list
> fluid-work at fluidproject.org
> http://fluidproject.org/mailman/listinfo/fluid-work

---
Colin Clark
Technical Lead, Fluid Project
Adaptive Technology Resource Centre, University of Toronto
http://fluidproject.org


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://fluidproject.org/pipermail/fluid-work/attachments/20081006/80f378d0/attachment.html>


More information about the fluid-work mailing list