Project

General

Profile

File transfer methods

Added by Mary Frantz about 11 years ago

What is the most efficient file transfer method that this distro supports? We expect to generate very large files (up to 1 GB) on a SATA drive. A host PC (Windows) will need to transfer the files from the target to the host over a network. So far we have used SCP, but it is very slow. Does this distro support tftpd? or some other protocol? We don't need any security as the PC and target are on an isolated network.

We are using the MDK_2012-08-10 kernel.

thanks,
Mary Frantz


Replies (15)

RE: File transfer methods - Added by Jonathan Cormier about 11 years ago

Mary,

There should be a tftp dameon package available for this platform.

Which device will be triggering this transfer? Meaning is the Windows computer going to be the server and the som the client or visa versa.

If you want to test this, the module should already have the tftp client on it. You can quickly setup a windows tftp server. http://www.solarwinds.com/products/freetools/free_tftp_server.aspx And try to see what speed you get when you tftp from the som to the windows machine.

-Jonathan

RE: File transfer methods - Added by Mary Frantz about 11 years ago

The Windows PC would be requesting the file. So, I think we want the som to be the server. Is this possible?
Mary

RE: File transfer methods - Added by Jonathan Cormier about 11 years ago

It is possible. But i would test the other direction first to see if you get the speeds your looking for.

-Jonathan

RE: File transfer methods - Added by Bob Duke about 11 years ago

Mary,

The angstrom distribution we use for our baseline does not include a TFTP server in its package repository.

If the TFTP speed meets your requirements, you will need to compile busybox and enable support for TFTPD.

Some tips for recompiling busybox can be found here:
http://support.criticallink.com/redmine/projects/arm9-platforms/wiki/Modifying_Busybox

-Bob

Adding link to a prior discussion:
http://support.criticallink.com/redmine/boards/10/topics/2276

RE: File transfer methods - Added by Mary Frantz about 11 years ago

Trying the Solar Winds tftp server. Installed it, ran it.

On my target:

root@mityomap:/mnt/sata# tftp -p -l testfile 192.168.1.82
tftp: timeout

root@mityomap:/mnt/sata# tftp -p -r testfile 192.168.1.82
tftp: timeout

Also tried running from the /boot directory .... same result.

What am I doing wrong?
Mary

RE: File transfer methods - Added by Jonathan Cormier about 11 years ago

My guess is that windows firewall is blocking the connection. You should be able to google how to allow an inbound connection through the firewall on port 69 for your specific version of windows.

-Jonathan

RE: File transfer methods - Added by Jonathan Cormier about 11 years ago

I switched to using the http://tftpd32.jounin.net/ program as the tftp server as I was unable to get the other one to give me enough data to determine transfer speed. This program is much simpler and will display the transfer speed of the incoming file.

I did have to open up the firewall in order to get it to work.

-Jonathan

RE: File transfer methods - Added by Jonathan Cormier about 11 years ago

I was able to transfer a 6MB file in about 13 seconds which is about 472 kB/s.

And the same transfer done over scp took about 6 seconds and reported an average speed of 1.0MB/s.

RE: File transfer methods - Added by Michael Williamson about 11 years ago

It might be easier to export the mitydsp filesystem via NFS or SAMBA and have windows access it directly.

Or have the mitydsp mount a windows share via SAMBA and do a copy.

You should be able to get 7 or 8 MB/sec transfer speeds, higher with a UDP based protocol on a good network.

-Mike

RE: File transfer methods - Added by Michael Williamson about 11 years ago

TFTP is not very efficient depending on how it is configured.

RE: File transfer methods - Added by Jonathan Cormier about 11 years ago

I built the latest busybox using instructions provided and ran the ftpd service as described below and got a transfer rate close to 6MB/s.

I copied the new busybox executable to the /home/root folder and ran it from there for the below commands:

$ ./busybox ftpd
BusyBox v1.21.1 (2013-10-14 14:39:43 EDT) multi-call binary.

Usage: ftpd [-wvS] [-t N] [-T N] [DIR]

Anonymous FTP server

ftpd should be used as an inetd service.
ftpd's line for inetd.conf:
        21 stream tcp nowait root ftpd ftpd /files/to/serve
It also can be ran from tcpsvd:
        tcpsvd -vE 0.0.0.0 21 ftpd /files/to/serve

        -w      Allow upload
        -v      Log errors to stderr. -vv: verbose log
        -S      Log errors to syslog. -SS: verbose log
        -t,-T   Idle and absolute timeouts
        DIR     Change root to this directory

$ ./busybox tcpsvd
BusyBox v1.21.1 (2013-10-14 14:39:43 EDT) multi-call binary.

Usage: tcpsvd [-hEv] [-c N] [-C N[:MSG]] [-b N] [-u USER] [-l NAME] IP PORT PROG

Create TCP socket, bind to IP:PORT and listen
for incoming connection. Run PROG for each connection.

        IP              IP to listen on, 0 = all
        PORT            Port to listen on
        PROG ARGS       Program to run
        -l NAME         Local hostname (else looks up local hostname in DNS)
        -u USER[:GRP]   Change to user/group after bind
        -c N            Handle up to N connections simultaneously
        -b N            Allow a backlog of approximately N TCP SYNs
        -C N[:MSG]      Allow only up to N connections from the same IP
                        New connections from this IP address are closed
                        immediately. MSG is written to the peer before close
        -h              Look up peer's hostname
        -E              Don't set up environment variables
        -v              Verbose

$ ./busybox tcpsvd -vE 0.0.0.0 21 './busybox ftpd' /home/root/
tcpsvd: listening on 0.0.0.0:21, starting
tcpsvd: start 1356 10.0.59.31:21-10.0.62.1:16562
tcpsvd: can't execute './busybox ftpd': No such file or directory
tcpsvd: status 1/30
tcpsvd: end 1356 exit 127
tcpsvd: status 0/30
^Ctcpsvd: got signal 2, exit

$ ln busybox ftpd
$ ./busybox tcpsvd -vE 0.0.0.0 21 ./ftpd /home/root/
tcpsvd: listening on 0.0.0.0:21, starting
tcpsvd: start 1361 10.0.59.31:21-10.0.62.1:16578
tcpsvd: status 1/30
tcpsvd: start 1363 10.0.59.31:21-10.0.62.1:16590
tcpsvd: status 2/30
tcpsvd: end 1363 exit 0
tcpsvd: status 1/30
tcpsvd: end 1361 exit 1
tcpsvd: status 0/30

RE: File transfer methods - Added by Jonathan Cormier about 11 years ago

Note that all of my experiments have been on a mitydsp copying from nand and not sata.

RE: File transfer methods - Added by Mary Frantz about 11 years ago

I tried some tests as you suggested.

file size = 64 Mbytes

scp: 60 sec (using WinSCP)
sftp: 53 sec (using WinSCP)
tftp: 6 min (using TFTPServer)
netcat: 20 sec (Linux host)

I also tried compressing with gzip, but after 3 minutes of zipping I quit.
LinuxHost: nc –l 8888 | gzip –d | tar xf - -C /
Mitydsp#: tar –cf – testfile | gzip -1 | nc 192.168.1.72 8888

I also had a permission error on the host (receiving) end after running with a very small test file.

Mary

RE: File transfer methods - Added by Jonathan Cormier about 11 years ago

Mary what do you get from the ftp server?

RE: File transfer methods - Added by Mary Frantz about 11 years ago

I did not try a ftp transfer. We have decided to do the transfer within our application, by connecting to a socket and just sending the file data through. I would expect to get performance something like netcat.

Mary

    (1-15/15)
    Go to top
    Add picture from clipboard (Maximum size: 1 GB)