Forums » Software Development »
File transfer methods
Added by Mary Frantz about 11 years ago
What is the most efficient file transfer method that this distro supports? We expect to generate very large files (up to 1 GB) on a SATA drive. A host PC (Windows) will need to transfer the files from the target to the host over a network. So far we have used SCP, but it is very slow. Does this distro support tftpd? or some other protocol? We don't need any security as the PC and target are on an isolated network.
We are using the MDK_2012-08-10 kernel.
thanks,
Mary Frantz
Replies (15)
RE: File transfer methods - Added by Jonathan Cormier about 11 years ago
Mary,
There should be a tftp dameon package available for this platform.
Which device will be triggering this transfer? Meaning is the Windows computer going to be the server and the som the client or visa versa.
If you want to test this, the module should already have the tftp client on it. You can quickly setup a windows tftp server. http://www.solarwinds.com/products/freetools/free_tftp_server.aspx And try to see what speed you get when you tftp from the som to the windows machine.
-Jonathan
RE: File transfer methods - Added by Mary Frantz about 11 years ago
The Windows PC would be requesting the file. So, I think we want the som to be the server. Is this possible?
Mary
RE: File transfer methods - Added by Jonathan Cormier about 11 years ago
It is possible. But i would test the other direction first to see if you get the speeds your looking for.
-Jonathan
RE: File transfer methods - Added by Bob Duke about 11 years ago
Mary,
The angstrom distribution we use for our baseline does not include a TFTP server in its package repository.
If the TFTP speed meets your requirements, you will need to compile busybox and enable support for TFTPD.
Some tips for recompiling busybox can be found here:
http://support.criticallink.com/redmine/projects/arm9-platforms/wiki/Modifying_Busybox
-Bob
Adding link to a prior discussion:
http://support.criticallink.com/redmine/boards/10/topics/2276
RE: File transfer methods - Added by Mary Frantz about 11 years ago
Trying the Solar Winds tftp server. Installed it, ran it.
On my target:
root@mityomap:/mnt/sata# tftp -p -l testfile 192.168.1.82
tftp: timeout
root@mityomap:/mnt/sata# tftp -p -r testfile 192.168.1.82
tftp: timeout
Also tried running from the /boot directory .... same result.
What am I doing wrong?
Mary
RE: File transfer methods - Added by Jonathan Cormier about 11 years ago
My guess is that windows firewall is blocking the connection. You should be able to google how to allow an inbound connection through the firewall on port 69 for your specific version of windows.
-Jonathan
RE: File transfer methods - Added by Jonathan Cormier about 11 years ago
I switched to using the http://tftpd32.jounin.net/ program as the tftp server as I was unable to get the other one to give me enough data to determine transfer speed. This program is much simpler and will display the transfer speed of the incoming file.
I did have to open up the firewall in order to get it to work.
-Jonathan
RE: File transfer methods - Added by Jonathan Cormier about 11 years ago
I was able to transfer a 6MB file in about 13 seconds which is about 472 kB/s.
And the same transfer done over scp took about 6 seconds and reported an average speed of 1.0MB/s.
RE: File transfer methods - Added by Michael Williamson about 11 years ago
It might be easier to export the mitydsp filesystem via NFS or SAMBA and have windows access it directly.
Or have the mitydsp mount a windows share via SAMBA and do a copy.
You should be able to get 7 or 8 MB/sec transfer speeds, higher with a UDP based protocol on a good network.
-Mike
RE: File transfer methods - Added by Michael Williamson about 11 years ago
TFTP is not very efficient depending on how it is configured.
RE: File transfer methods - Added by Jonathan Cormier about 11 years ago
I built the latest busybox using instructions provided and ran the ftpd service as described below and got a transfer rate close to 6MB/s.
I copied the new busybox executable to the /home/root folder and ran it from there for the below commands:
$ ./busybox ftpd BusyBox v1.21.1 (2013-10-14 14:39:43 EDT) multi-call binary. Usage: ftpd [-wvS] [-t N] [-T N] [DIR] Anonymous FTP server ftpd should be used as an inetd service. ftpd's line for inetd.conf: 21 stream tcp nowait root ftpd ftpd /files/to/serve It also can be ran from tcpsvd: tcpsvd -vE 0.0.0.0 21 ftpd /files/to/serve -w Allow upload -v Log errors to stderr. -vv: verbose log -S Log errors to syslog. -SS: verbose log -t,-T Idle and absolute timeouts DIR Change root to this directory $ ./busybox tcpsvd BusyBox v1.21.1 (2013-10-14 14:39:43 EDT) multi-call binary. Usage: tcpsvd [-hEv] [-c N] [-C N[:MSG]] [-b N] [-u USER] [-l NAME] IP PORT PROG Create TCP socket, bind to IP:PORT and listen for incoming connection. Run PROG for each connection. IP IP to listen on, 0 = all PORT Port to listen on PROG ARGS Program to run -l NAME Local hostname (else looks up local hostname in DNS) -u USER[:GRP] Change to user/group after bind -c N Handle up to N connections simultaneously -b N Allow a backlog of approximately N TCP SYNs -C N[:MSG] Allow only up to N connections from the same IP New connections from this IP address are closed immediately. MSG is written to the peer before close -h Look up peer's hostname -E Don't set up environment variables -v Verbose $ ./busybox tcpsvd -vE 0.0.0.0 21 './busybox ftpd' /home/root/ tcpsvd: listening on 0.0.0.0:21, starting tcpsvd: start 1356 10.0.59.31:21-10.0.62.1:16562 tcpsvd: can't execute './busybox ftpd': No such file or directory tcpsvd: status 1/30 tcpsvd: end 1356 exit 127 tcpsvd: status 0/30 ^Ctcpsvd: got signal 2, exit $ ln busybox ftpd $ ./busybox tcpsvd -vE 0.0.0.0 21 ./ftpd /home/root/ tcpsvd: listening on 0.0.0.0:21, starting tcpsvd: start 1361 10.0.59.31:21-10.0.62.1:16578 tcpsvd: status 1/30 tcpsvd: start 1363 10.0.59.31:21-10.0.62.1:16590 tcpsvd: status 2/30 tcpsvd: end 1363 exit 0 tcpsvd: status 1/30 tcpsvd: end 1361 exit 1 tcpsvd: status 0/30
RE: File transfer methods - Added by Jonathan Cormier about 11 years ago
Note that all of my experiments have been on a mitydsp copying from nand and not sata.
RE: File transfer methods - Added by Mary Frantz about 11 years ago
I tried some tests as you suggested.
file size = 64 Mbytes
scp: 60 sec (using WinSCP)
sftp: 53 sec (using WinSCP)
tftp: 6 min (using TFTPServer)
netcat: 20 sec (Linux host)
I also tried compressing with gzip, but after 3 minutes of zipping I quit.
LinuxHost: nc –l 8888 | gzip –d | tar xf - -C /
Mitydsp#: tar –cf – testfile | gzip -1 | nc 192.168.1.72 8888
I also had a permission error on the host (receiving) end after running with a very small test file.
Mary
RE: File transfer methods - Added by Jonathan Cormier about 11 years ago
Mary what do you get from the ftp server?
RE: File transfer methods - Added by Mary Frantz about 11 years ago
I did not try a ftp transfer. We have decided to do the transfer within our application, by connecting to a socket and just sending the file data through. I would expect to get performance something like netcat.
Mary