RASCAL Data Streaming Backend -- Which Technology to use?

[David Makalsky]

Hi,

We have a functional requirement in RASCAL for data to be streamed  
from the clients' browser (applet) to the server.  This is the data  
that will be converted into the video of the users' desktop (and audio  
from the microphone).  This functionality is essential, since without  
it, the data wouldn't get to the server.

Currently, this solution is implemented via ftp.  An ftp connection is  
established from the client to the server and the data is uploaded in  
a separate thread in 1 MB chunks.  We chose this solution for various  
reasons (discussion in the table below) but would like to open this up  
to the fluid community.  Which of the proposed solutions below would  
you suggest, or are there other completely different solutions that we  
should consider?

Regards,

David Makalsky

FTP
===

Pros:
- Already implemented
- Many java libraries available (license compatible)
- Only virtual user needs to be created with only write access granted
- Simple to implement

Cons:
- Password is sent as plaintext
- There is a maximum simultaneous connection limit
- Not transactional

SFTP
====

Pros:
- Password is encrypted
- Simple to implement (if you have a good library)

Cons:
- No License Compatible java libraries
- Same cons as FTP except password is encrypted

Message Queue
=============

Pros:
- Transactional
- Secure
- Easy to implement on the client side
- Scalable

Cons:
- Significant administration and installation effort
- Resource-intensive

JDBC
====

Pros:
- Scalable
- Secure
- Easy to implement on both sides
- Transactional

Cons:
- Database will grow quite big unless blobs are written out to files  
regularly and cleaned from database


HTTP
====

Pros:
- transactional
- fairly easy to implement

Cons:
- Bad for streaming large chunks of data
- Puts a large strain on http server
- not scalable easily (will have to cluster web servers with session  
affinity)