RASCAL Data Streaming Backend -- Which Technology to use?

[Colin Clark]

Hey Peter,

I missed your response before I sent along mine, but your point about  
HTTPS is a good one. It's certainly a fair bit slower and presents  
greater load on the server to use HTTPS, but the nice thing with an  
HTTP streaming approach is that the VULab user can choose whether or  
not they need the added security. The other alternatives involve an  
"all or nothing" approach, whereas it's simply a server-side  
configuration to choose to use HTTPS.

Colin


On 6-Oct-08, at 6:08 PM, Peter Rowley wrote:

>
> Hi David and everyone,
>
> Following David's suggestion, I've converted an e-mail I sent to him  
> into one for fluid-work, so everyone can get in on the discussion.
>
> Based on some discussions within CNS at York, I'd suggest looking  
> more closely at using HTTPS, i.e. HTTP over SSL.  HTTPS...
>
> - due to the possibility that the screen recordings contain secure  
> data and thus the need to transmit them securely to the server,
> - due to the fact that it encodes credentials used for authorization  
> to deposit recordings on the server, and
> - due to its acceptability to sysadmins from a security perspective  
> (stemming from the nature of the protocol and greater level of  
> comfort with the security of Apache than a typical FTP server),
> - due to its ability to pass through most firewalls,
> - due to it being built into Java SE, so licensing is not an issue
>
> As for the scalability, HTTP upload is certainly used for massive  
> sites such as Flickr.  Your approach to uploading the file in  
> smaller (e.g. 1MB) chunks is probably a good one to continue; you'll  
> need to build (or find) a simple protocol above HTTPS to keep track  
> of which chunks still have to be transmitted.
>
> As for alternatives, JDBC and JMS are overly complex for this, and  
> SFTP has firewall issues in some situations (Google "sftp firewall").
>
> Peter
>
>
>
> David Makalsky <dmakalsky at gmail.com>
> Sent by: fluid-work-bounces at fluidproject.org
> 10/03/08 05:57 PM
>
> To
> Fluid Mailing List <fluid-work at fluidproject.org>
> cc
> Subject
> RASCAL Data Streaming Backend -- Which Technology to use?
>
>
>
>
>
> Hi,
>
> We have a functional requirement in RASCAL for data to be streamed
> from the clients' browser (applet) to the server.  This is the data
> that will be converted into the video of the users' desktop (and audio
> from the microphone).  This functionality is essential, since without
> it, the data wouldn't get to the server.
>
> Currently, this solution is implemented via ftp.  An ftp connection is
> established from the client to the server and the data is uploaded in
> a separate thread in 1 MB chunks.  We chose this solution for various
> reasons (discussion in the table below) but would like to open this up
> to the fluid community.  Which of the proposed solutions below would
> you suggest, or are there other completely different solutions that we
> should consider?
>
> Regards,
>
> David Makalsky
>
> FTP
> ===
>
> Pros:
> - Already implemented
> - Many java libraries available (license compatible)
> - Only virtual user needs to be created with only write access granted
> - Simple to implement
>
> Cons:
> - Password is sent as plaintext
> - There is a maximum simultaneous connection limit
> - Not transactional
>
> SFTP
> ====
>
> Pros:
> - Password is encrypted
> - Simple to implement (if you have a good library)
>
> Cons:
> - No License Compatible java libraries
> - Same cons as FTP except password is encrypted
>
> Message Queue
> =============
>
> Pros:
> - Transactional
> - Secure
> - Easy to implement on the client side
> - Scalable
>
> Cons:
> - Significant administration and installation effort
> - Resource-intensive
>
> JDBC
> ====
>
> Pros:
> - Scalable
> - Secure
> - Easy to implement on both sides
> - Transactional
>
> Cons:
> - Database will grow quite big unless blobs are written out to files
> regularly and cleaned from database
>
>
> HTTP
> ====
>
> Pros:
> - transactional
> - fairly easy to implement
>
> Cons:
> - Bad for streaming large chunks of data
> - Puts a large strain on http server
> - not scalable easily (will have to cluster web servers with session
> affinity)
> _______________________________________________
> fluid-work mailing list
> fluid-work at fluidproject.org
> http://fluidproject.org/mailman/listinfo/fluid-work
>
> _______________________________________________
> fluid-work mailing list
> fluid-work at fluidproject.org
> http://fluidproject.org/mailman/listinfo/fluid-work

---
Colin Clark
Technical Lead, Fluid Project
Adaptive Technology Resource Centre, University of Toronto
http://fluidproject.org