File transfer requirements are diversifying at a rate of knots with more products available than I care to count, but for me there are three distinct types of solutions I believe the majority of the larger corporate and blue chip customers are interested in. These are;

Enterprise File Transfer – making use of email to deliver a message to the end user that provides them with instructions on how to download the files(s) with the added functionality of tracking and reporting.  This method is great for the ad-hoc user, requiring little to no training.

Managed File Transfer -  relates to the secure delivery of files, in many cases making use of FTP based protocols also providing additional functionality such as reporting and monitoring.  These solutions are generally embedded processes that are not seen by the users and underpin internal/external business processes.

Fast File Transfer – with businesses needing to shift large volumes of data over increasing distances across the WAN or Internet traditional delivery protocols such as FTP have been superseded with UDP based delivery solutions, which have the ability to send files significantly faster.  With the cost of Internet connectivity as it is, WAN acceleration technologies are becoming more frequently used to maximise the throughput over those connections.

I’d be interested to hear from anyone who has any suggestions for areas that we may have missed, specifically if you’re a vendor of the solution and are looking for representation in the UK.

Bookmark It

Recently we seem to be getting quite a few enquiries from companies experiencing difficulties with high latency during the file transfer process.  It appears to be a wide spread issue, affecting businesses in a climate where everybody needs to get ahead of the game.  Companies want – no need – their files right here, right now!!!

For those of us unfamiliar with the term, latency is an expression for the period of time taken to send a data packet from a source to the intended destination.  The higher the latency, the slower the data transmission.  This incorporates all elements involved in the file sending process – including encoding, transmission, and decoding.

There are certain delivery protocols (such as FTP) that are particularly susceptible to latency.  This is because when sending packets of data to a remote site, the sending site waits for an acknowledgment that the packet has been received before sending the next one.  This makes the problem extremely time consuming in the event of high latency.  In extreme cases of latency, the time that it takes for the delivery of data and then the reply, can result in the data throughput levels dropping to a significantly low level, rendering the solution useless.

So, what can we do about it?  There are several ways to combat this problem – one being to utilise a multi-threaded TCP protocol – working in the same manner as above ONLY multiple packet transfer requests are made at any one time, increasing the throughput.  Another increasingly popular route is to adopt a UDP based delivery protocol, which applies a send and forget mentality i.e. they don’t wait for the acknowledgment receipt.  This can significantly speed up the delivery process BUT then you have to consider the fact that other features are required, and UDP out of the box won’t work for everyone.

How can you measure the latency on your connection?  Network tools like ping tests and traceroute measure latency by determining the time it takes a given network packet to travel from source to destination and back, the so-called round-trip time. Round-trip time is not the only way to specify latency, but it is the most common.

To test the latency on your Internet connection between 100’s of test servers you can use a tool like Speedtest.net, where you can test your bandwidth and latency against a local (London) server and say one in Bangkok.

On DSL or cable Internet connections, latencies of less than 100 milliseconds (ms) are typical and less than 25 ms desired. Satellite Internet connections, on the other hand, average 500 ms or higher latency.

So if you suffer from latency problems when it comes to file transfer, please feel free to contact Pro2col to discuss.  Alternatively, you could have a look at the fast file transfer section on our website that details some solutions that may help.

Bookmark It

So why FTP and what’s so great about it?  Well to be honest this isn’t necessarily a blog to evangelise FTP but more the way in which it works, lets call it ’sending files’.  With many businesses looking to adopt Managed File Transfer solutions, I thought it might be worth redressing the balance and putting things into perspective.  Managed File Transfer solutions have many good features but in the case of email based ones, sending files isn’t one of them.  In many cases the Managed File Transfer solution doesn’t actually send anything, rather it asks the company email server to send an email to a particular recipient.  The person receiving the email clicks on a link within the email to download the file or goes to a web site to log-in and manually download the file – so you see the responsibility is on the recipient to download the file and given this, there is no guarantee that the file will get there.  In fact there’s no guarantee the email is going to get there at all, asking the recipient to download the file(s).  Whilst Managed File Transfer solutions cater for the majority of ‘file transfer’ uses it is certainly not the right solution for every scenario.

So what do I mean by ’sending files’.   Well, historically the majority of solutions used to send files required a connection to be created between two sites and the files to be pushed/transferred to the receiving site using the appropriate delivery protocol for the connection method, e.g. Modem, ISDN or IP.   A typical example that many people would be able to relate to is FTP.  A user with an FTP client enters the details for the server, connects, selects the files to transfer, drags them over to the ‘remote server’ window (in many FTP client softwares) and the transferring of files starts straight away.  Once all of the files have been transferred you can see them on the remote server, they are there without question, the files have been delivered.

In contrast, Managed File Transfer solutions that use email messaging to deliver a message to request the download of the files, has several potential points of failure.  You’ve got to rely on two email servers to be happy to deliver the message and not overburdened with other requests, you have to ensure that SPAM filters don’t whisk away your all important message and probably most importantly – someone has to be there to open, read and perform the manual process of downloading the file.

In short FTP file transfer has a place in the enterprise.  If you want to be able to push data to a location with or without manual intervention, then FTP or another file transfer protocol with similar features will do.  Certain business to business situations will rely on data being sent from one location to the next e.g. a publisher to his printer, where time is of the essence and any doubt about the delivery of the data has to be avoided.

Finally it is possible to make FTP more functional and secure than many Managed File Transfer vendors make out, in fact some Managed File Transfer vendors have it built in.  Depending upon the solution you implement, you can get some great functionality to compliment this old delivery protocol and its also possible to integrate with workflow solutions, script integration and utilise API’s and SDK’s for complete integration.

Bookmark It

Lets face it file transfer isn’t the most interesting subject in the world, but its a necessity for many businesses to move data from one location to another in order for individuals or teams to carry out certain tasks.

For many businesses however file transfer is a tedious process requiring manual intervention.  Regularly staff are expected to manually create a job to send and watch the file being delivered to the remote site to ensure that its gone, a good example being the user of an FTP client.  Clearly there are instances where this scenario works and is the most economic way, e.g. the one off or infrequent transferring of files, but for companies that regularly need to send large files to the same location this approach is nothing short of a criminal waste of resources.

As businesses strive to succeed in these challenging times many more are looking towards automation of their internal processes, this is the case irrespective of industry sector and to some extent business size.  A small part of many businesses workflow process is the delivery of files/data to another location – whether its on the same LAN, across a WAN or via the Internet whatever route the premise is the same even if the conditions are not.

File transfer should in my opinion wherever possible be an embedded process, effectively seamless, with the user not even aware that its taking place.  There are many ways of achieving this result (which in itself is another blog post entirely), whether hot-folder initiated or integrated with existing applications using various SDK’s.  Then there are the considerations of which delivery protocol is most appropriate, e.g. TCP/IP (FTP) or UDP (MTP/IP) and whether encryption of the data should be included.

The bottom line is in fact the bottom line (of your P&L statement).  Businesses need to wake up to the fact that manually sending files is not a good use of resources and where automation is possible processes are streamlined, files are delivered faster and human error is eradicated.

If you or your company could benefit from embedded file transfer or file transfer automation Pro2col would be pleased to assist you.

Bookmark It