Advertise on Bikeforums.net



User Tag List

Page 1 of 2 12 LastLast
Results 1 to 25 of 35
  1. #1
    J E R S E Y S B E S T Jerseysbest's Avatar
    Join Date
    Apr 2005
    Location
    DC
    Posts
    1,852
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    What's the most efficient way to transfer data?

    Something came up at work and we're wondering if we could, on about a weekly basis, transfer about 1.0 to 1.3 GB of data from one computer to about 5, or as many as 10, others, all of which are in different locations.

    Do you think this could be done in a reasonable amount of time? I guess the cheapest solution would be USB 2.0 flash drives or external hard drives?

    Anyone have experience doing this kind of stuff?

  2. #2
    riding once again jschen's Avatar
    Join Date
    Oct 2005
    Location
    San Diego, CA
    My Bikes
    '06 Cervelo R3, '05 Specialized Allez
    Posts
    7,360
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Efficient for the computer or efficient for the human? Different locations in the same building or geographically separated? If geographically separated, is the data sensitive? Why not just make a bunch of copies via file sharing? Sure, if your network and/or internet connection is slow it might take a while, but you can initiate the process at the end of a workday and just let it go.
    If you notice this notice then you will notice that this notice is not worth noticing.

  3. #3
    That darn Yankee TexasGuy's Avatar
    Join Date
    Jun 2005
    Location
    West West Fort Worth
    My Bikes
    Mongoose XR-100, Eros Bianchi
    Posts
    4,286
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I'd try using a network. Network should be tons times faster then USB and much less processor intensive. A 1gb file can transfer quite fast over a modern network (preferably wired)
    Life is about hanging onto what you think is important and finding out what really is important.
    "Stop Ruining my joke!", "No, a joke implies humor attached at no additional cost"
    So many sayings, so little sig space.

  4. #4
    Junior Member Glenfiddich_Man's Avatar
    Join Date
    Sep 2006
    Posts
    9
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Pretty much passing around a flash drive would be the quickest/cheapest

    http://www.amazon.com/SanDisk-Cruzer...&s=electronics

  5. #5
    That darn Yankee TexasGuy's Avatar
    Join Date
    Jun 2005
    Location
    West West Fort Worth
    My Bikes
    Mongoose XR-100, Eros Bianchi
    Posts
    4,286
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Flash drives are slow. The fast ones are expensive and they have a limited # of writes.
    I would never use a Flash Disk system for something like this. Unless there was no way to run network cables or run wireless as wireless should be faster then USB, is not limited by write speed and does not wear down after x writes.
    Life is about hanging onto what you think is important and finding out what really is important.
    "Stop Ruining my joke!", "No, a joke implies humor attached at no additional cost"
    So many sayings, so little sig space.

  6. #6
    Dances With Cars TRaffic Jammer's Avatar
    Join Date
    Jan 2006
    Location
    Toronto, Canada
    My Bikes
    TBL Onyx Pro(ss converted), Pake SS (starting to look kinda pimped)
    Posts
    10,527
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    When you say different locations...different cities?
    Publish once to FTP/ or intranet site (if offices are linked with one)...let branch office download at their leisure. Or write to their respective sites at the same time (overnight lan admin operation). You network administrator will have your solution where they write to the locations all at once.

    If it's within the same building/wired network, simply post to network volume and it's accessible.

    If it's Sensitive with capitol S burn and ship DVD's

  7. #7
    J E R S E Y S B E S T Jerseysbest's Avatar
    Join Date
    Apr 2005
    Location
    DC
    Posts
    1,852
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    All the computers are in different geographic locations, meaning, not in the same building.

    I thought about the internet, but I'd guess that the max download speed would be 1 mb/sec, so that would take forever, and since we'd like to not have the data get into anyone else's hands.

    One other idea I thought about are burning it onto DVD's. Have the people who will be bringing the data to the remote machines get the data off our network, and the burn it. For something that would happen on a weekly basis, I guess it could work.

  8. #8
    riding once again jschen's Avatar
    Join Date
    Oct 2005
    Location
    San Diego, CA
    My Bikes
    '06 Cervelo R3, '05 Specialized Allez
    Posts
    7,360
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Proper encryption (which would be necessary even with the DVDs) would help protect sensitive information. So I don't see going over the Internet via some sort of VPN as being a problem. I know of companies that transfer huge amounts of extremely sensitive data that way. As for time, well, at 1 MB/s, you'd copy those files in about 20 minutes. Let's say it takes 30 minutes. You really think you could burn a DVD, drive it over, and then upload it faster than that? And the electronic transfer takes far less work. Even if you just make transfers all from the same computer (you could do it faster if you script it so that after the first transfer, that computer starts transferring, too), and it takes 30 minutes per transfer, that only takes 5 hours max. Even if it takes an hour per transfer, that's 10 hours. Surely you have a 10 hour window over the weekend? And importantly, you don't have to be there to get the job done. It could be scripted to run automatically. Or you could just start the transfers at the end of the work week and let them go.
    If you notice this notice then you will notice that this notice is not worth noticing.

  9. #9
    Dances With Cars TRaffic Jammer's Avatar
    Join Date
    Jan 2006
    Location
    Toronto, Canada
    My Bikes
    TBL Onyx Pro(ss converted), Pake SS (starting to look kinda pimped)
    Posts
    10,527
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    you can get blistering fast data transfers... I have three fibre lines to our press centre at the other end of the city, but it's full all day everyday. A weekly burn and ship seems fair inexpensive in comparison

  10. #10
    Banned.
    Join Date
    Sep 2003
    Location
    Home alone
    My Bikes
    Trek 4300 X 2. Trek 1000, Trek 6000
    Posts
    6,021
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Dvd Ram

  11. #11
    Senior Member DannoXYZ's Avatar
    Join Date
    Jul 2005
    Location
    Saratoga, CA
    Posts
    11,495
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Why aren't all your buildings on a network? Sneaker-net went out of style decades ago... You can set up a private WiFi network using directional antennaes with ranges of about a mile.

    jschen's got the right idea. It's simplest to use the network someone else has developed. Saves you time, effort and money. You can even set up a sftp-server and have the files posted for all them to download. Encryption will ensure security. Compression can probably crunch that gig of data down quite a bit depending upon what type of data it is.

    If it's something that you do on a regular basis, you can even write a script that'll upload to the server and automatically download to each of the client stations.

  12. #12
    On my TARDIScycle! KingTermite's Avatar
    Join Date
    Jun 2005
    Location
    Eastside Seattlite Termite Mound
    My Bikes
    Trek 520, Trek Navigator 300, Peugeot Versailles PE10DE
    Posts
    3,928
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by TexasGuy
    I'd try using a network. Network should be tons times faster then USB and much less processor intensive. A 1gb file can transfer quite fast over a modern network (preferably wired)
    +1

    If its the same data, you could even automate it with a batch file or script.
    Quote Originally Posted by coffeecake View Post
    - it's pretty well established that Hitler was an *******.

  13. #13
    That darn Yankee TexasGuy's Avatar
    Join Date
    Jun 2005
    Location
    West West Fort Worth
    My Bikes
    Mongoose XR-100, Eros Bianchi
    Posts
    4,286
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by TRaffic Jammer
    you can get blistering fast data transfers... I have three fibre lines to our press centre at the other end of the city, but it's full all day everyday. A weekly burn and ship seems fair inexpensive in comparison
    I wonder why printing press and likees networks are always bogged down
    Life is about hanging onto what you think is important and finding out what really is important.
    "Stop Ruining my joke!", "No, a joke implies humor attached at no additional cost"
    So many sayings, so little sig space.

  14. #14
    Dances With Cars TRaffic Jammer's Avatar
    Join Date
    Jan 2006
    Location
    Toronto, Canada
    My Bikes
    TBL Onyx Pro(ss converted), Pake SS (starting to look kinda pimped)
    Posts
    10,527
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    ^^ NOT that our ISP runs through those channels, TG smar-ty

  15. #15
    Senior Member russiankdi's Avatar
    Join Date
    Feb 2006
    Location
    PA
    My Bikes
    many
    Posts
    213
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    If all the computers are hooked up to one main network, you can setup up so that all the computers can be shared, that way files can be transfered from one computer to as many as you want. On my home network i have 3 computers hooked up, and i can transfer data really easily, up to 100mbp/s.
    1978 Schwinn Varsity Single Speed

  16. #16
    That darn Yankee TexasGuy's Avatar
    Join Date
    Jun 2005
    Location
    West West Fort Worth
    My Bikes
    Mongoose XR-100, Eros Bianchi
    Posts
    4,286
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by russiankdi
    If all the computers are hooked up to one main network, you can setup up so that all the computers can be shared, that way files can be transfered from one computer to as many as you want. On my home network i have 3 computers hooked up, and i can transfer data really easily, up to 100mbp/s.
    that aspect of the problem has already been delineated. They're not. They are in different geogrpahical locations.
    Life is about hanging onto what you think is important and finding out what really is important.
    "Stop Ruining my joke!", "No, a joke implies humor attached at no additional cost"
    So many sayings, so little sig space.

  17. #17
    hide not your essence TRACKMAN's Avatar
    Join Date
    Oct 2006
    Location
    Dodge City
    My Bikes
    Airborne,Guerciotti,Jamis,Merckx,Paris Sport,Raleigh,Vitus
    Posts
    156
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    you could do a point to point; some browsers, i.e. DEEPNET enable this and you can set passwords.
    FTP is archaic and slower, and an open share on a network means ANYONE that can see it can snatch it.
    of course, you could set passwords on that share as well but throughput would degregate depending on amount of traffic in that subnet. Not to mention if you were leaving that net you would be subjugated to the router or switches ability to provide bandwith as well. Regardless of how much bandwidth you have, you are still limited by the slowest hop along the path between two nodes.
    May you find the joy and peace you long for.
    Life is a journey ... NOT a guided tour.
    .......__O
    .......\<,
    ....( )/ ( )...

  18. #18
    riding once again jschen's Avatar
    Join Date
    Oct 2005
    Location
    San Diego, CA
    My Bikes
    '06 Cervelo R3, '05 Specialized Allez
    Posts
    7,360
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Passwords alone don't do you much good if you're transmitting unencrypted data in the clear. If the data is at all sensitive, encryption is necessary. Whether you do this at the file level or at the network level is up to you. I'd do it at the network level to avoid having to encrypt and decrypt files over and over.
    If you notice this notice then you will notice that this notice is not worth noticing.

  19. #19
    J E R S E Y S B E S T Jerseysbest's Avatar
    Join Date
    Apr 2005
    Location
    DC
    Posts
    1,852
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Thanks for all the input, I'm still trying to wrap my head around this and figure something out.

    There are a lot of variables, so I don't know if we'll be able to afford the time of downloading over the net, but that option is on the table.

    Because the person that will be going to the remote machine already has their own machine here in the office with a DVD burner, is on the network, and a batch file set up to retrieve the data from the main machine, I'm leaning towards having each person burn the data onto the DVD.

    This might all be done sorta on the fly, so weekend downloads probably aren't an option, and I think we're already getting flack for our bandwidth use.

  20. #20
    riding once again jschen's Avatar
    Join Date
    Oct 2005
    Location
    San Diego, CA
    My Bikes
    '06 Cervelo R3, '05 Specialized Allez
    Posts
    7,360
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    How far are the buildings? Transfering data by shuffling people back and forth is truly inefficient. If these transfers are made "on the fly", how urgent is the data?

    If you must shuffle people, I'd go with portable hard drives (Firewire or USB2). Having lots of DVDs sitting around with sensitive data is potentially problematic. You'd have to ensure each of those DVDs is destroyed after use. And hard drive transfers are far faster.
    If you notice this notice then you will notice that this notice is not worth noticing.

  21. #21
    That darn Yankee TexasGuy's Avatar
    Join Date
    Jun 2005
    Location
    West West Fort Worth
    My Bikes
    Mongoose XR-100, Eros Bianchi
    Posts
    4,286
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by jschen
    How far are the buildings? Transfering data by shuffling people back and forth is truly inefficient. If these transfers are made "on the fly", how urgent is the data?

    If you must shuffle people, I'd go with portable hard drives (Firewire or USB2). Having lots of DVDs sitting around with sensitive data is potentially problematic. You'd have to ensure each of those DVDs is destroyed after use. And hard drive transfers are far faster.
    The same issue happens with USB also. Some janitor picks up usb, takes it home . bam. Information is compromised. I'd be tempted -

    if they are going from laptop to computer why not just use a wireless network.
    Life is about hanging onto what you think is important and finding out what really is important.
    "Stop Ruining my joke!", "No, a joke implies humor attached at no additional cost"
    So many sayings, so little sig space.

  22. #22
    J E R S E Y S B E S T Jerseysbest's Avatar
    Join Date
    Apr 2005
    Location
    DC
    Posts
    1,852
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by jschen
    How far are the buildings? Transfering data by shuffling people back and forth is truly inefficient. If these transfers are made "on the fly", how urgent is the data?

    If you must shuffle people, I'd go with portable hard drives (Firewire or USB2). Having lots of DVDs sitting around with sensitive data is potentially problematic. You'd have to ensure each of those DVDs is destroyed after use. And hard drive transfers are far faster.
    Well, I'd assume we'd destroy the data DVD once it was transfered, and the cost isn't prohibitive.

    And the the other computers are too far to utilitize a wireless network.

  23. #23
    The AVatar Ninja SaabFan's Avatar
    Join Date
    Aug 2006
    Posts
    609
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    How about a shared workspace application like Groove?

    Install it on each machine, leave them connected to the internet and logged in to a shared Groove workspace. The files in the workspace are automatically distributed to each computer over the internet, encrypted, in the background. No human interaction required. It'll even do automatic backups of everything, and lets you use version tracking if the files are changed over time by multiple users.
    Q: How many surrealists does it take to change a light bulb?
    A: FISH!

  24. #24
    Senior Member russiankdi's Avatar
    Join Date
    Feb 2006
    Location
    PA
    My Bikes
    many
    Posts
    213
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by TexasGuy
    that aspect of the problem has already been delineated. They're not. They are in different geogrpahical locations.
    In that case external USB hard drive, have one of those as well, works great.
    1978 Schwinn Varsity Single Speed

  25. #25
    DEADBEEF khuon's Avatar
    Join Date
    Aug 2002
    Location
    Catching his breath alongside a road near Seattle, WA USA
    My Bikes
    1999 K2 OzM, 2001 Aegis Aro Svelte
    Posts
    12,155
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I think everyone forgot to ask the critical question here...

    What's your budget?

    Also: What's the distance? Can the data be streamed and/or updated incrimentally? Is it one big file or multiple files? What's the nature of the data? Text? Binary?
    1999 K2 OzM 2001 Aegis Aro Svelte OCP Club Member
    "Be liberal in what you accept, and conservative in what you send." -- Jon Postel, RFC1122

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •